2024-07-29T00:12:25.034 INFO:root:teuthology version: 0.0.1.dev379+g53ce146 2024-07-29T00:12:25.070 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-29T00:12:25.157 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7823088' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: ba82afec3628bf830c0830558bf3d3c3aea66701 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: ba82afec3628bf830c0830558bf3d3c3aea66701 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ba82afec3628bf830c0830558bf3d3c3aea66701 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 743 sha1: ba82afec3628bf830c0830558bf3d3c3aea66701 sleep_before_teardown: 0 subset: 285/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ba82afec3628bf830c0830558bf3d3c3aea66701 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCoDkNDSRWwmdk9Ih8gsDFSkycZW9PEQQ/uNp+sTikbzEk3sWomGXTs9DB9jpFmlLLzbiKm9DftgJm16GGg6aSY= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDuyFPEmEtERtJpcY8KrPRZL3t0FZJwYnPFUo9LP0tWCwMvTgKjcW8x9FfKMsNL1jxskBlCvjADUT+r9lmiPBXE= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPd5oIS5Qotgn796CzxzZmB+R/oQg1Xkx2YS3DTZi1kniVLIGUIFwbtp6tVotS1DqsO8Kw/cAPXvPy569awUe9w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 53ce1462e129f6eb4071986336534c740fdebd31 timestamp: 2024-07-28_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-07-29T00:12:25.157 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa; will attempt to use it 2024-07-29T00:12:25.158 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks 2024-07-29T00:12:25.158 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-29T00:12:25.159 INFO:teuthology.task.internal:Checking packages... 2024-07-29T00:12:25.180 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ba82afec3628bf830c0830558bf3d3c3aea66701' 2024-07-29T00:12:25.180 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-29T00:12:25.180 INFO:teuthology.packaging:ref: None 2024-07-29T00:12:25.180 INFO:teuthology.packaging:tag: None 2024-07-29T00:12:25.180 INFO:teuthology.packaging:branch: squid 2024-07-29T00:12:25.180 INFO:teuthology.packaging:sha1: ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:12:25.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-07-29T00:12:25.404 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1150.gba82afec 2024-07-29T00:12:25.406 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-29T00:12:25.508 INFO:teuthology.task.internal:no buildpackages task found 2024-07-29T00:12:25.508 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-29T00:12:25.545 INFO:teuthology.task.internal:Saving configuration 2024-07-29T00:12:25.565 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-29T00:12:25.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-29T00:12:25.595 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-29 00:07:20.937171', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCoDkNDSRWwmdk9Ih8gsDFSkycZW9PEQQ/uNp+sTikbzEk3sWomGXTs9DB9jpFmlLLzbiKm9DftgJm16GGg6aSY='} 2024-07-29T00:12:25.616 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-29 00:07:20.935676', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDuyFPEmEtERtJpcY8KrPRZL3t0FZJwYnPFUo9LP0tWCwMvTgKjcW8x9FfKMsNL1jxskBlCvjADUT+r9lmiPBXE='} 2024-07-29T00:12:25.634 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-29 00:07:20.933937', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPd5oIS5Qotgn796CzxzZmB+R/oQg1Xkx2YS3DTZi1kniVLIGUIFwbtp6tVotS1DqsO8Kw/cAPXvPy569awUe9w='} 2024-07-29T00:12:25.634 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-29T00:12:25.641 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-07-29T00:12:25.641 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-07-29T00:12:25.641 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-07-29T00:12:25.641 INFO:teuthology.run_tasks:Running task console_log... 2024-07-29T00:12:25.725 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f59d0cd3be0>, signals=[15]) 2024-07-29T00:12:25.725 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-29T00:12:25.750 INFO:teuthology.task.internal:Opening connections... 2024-07-29T00:12:25.750 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:12:25.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:12:25.829 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-07-29T00:12:25.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:12:25.906 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-07-29T00:12:25.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:12:25.979 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-29T00:12:25.992 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-07-29T00:12:26.009 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-07-29T00:12:26.010 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-07-29T00:12:26.066 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-29T00:12:26.067 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-29T00:12:26.068 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-07-29T00:12:26.092 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-07-29T00:12:26.110 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-07-29T00:12:26.111 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-07-29T00:12:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-29T00:12:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-29T00:12:26.169 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-07-29T00:12:26.192 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-07-29T00:12:26.210 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-07-29T00:12:26.210 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-07-29T00:12:26.267 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-07-29T00:12:26.268 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-29T00:12:26.268 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-07-29T00:12:26.268 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-29T00:12:26.268 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-29T00:12:26.268 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-29T00:12:26.268 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-07-29T00:12:26.292 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-29T00:12:26.301 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-29T00:12:26.307 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-29T00:12:26.307 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-07-29T00:12:26.310 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-07-29T00:12:26.312 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-07-29T00:12:26.328 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-29T00:12:26.334 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-29T00:12:26.334 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-07-29T00:12:26.366 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-07-29T00:12:26.369 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-07-29T00:12:26.404 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-29T00:12:26.475 INFO:teuthology.run_tasks:Running task kernel... 2024-07-29T00:12:26.521 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-07-29T00:12:26.521 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-07-29T00:12:26.521 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-07-29T00:12:26.521 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-29T00:12:26.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:12:26.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-07-29T00:12:26.715 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3e9255e907100f88049f320ca33d977e29155d12 2024-07-29T00:12:26.715 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3e9255e907100f88049f320ca33d977e29155d12"... 2024-07-29T00:12:26.715 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-07-29T00:12:26.732 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-437.el9.x86_64 2024-07-29T00:12:26.733 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 3e9255e907100f88049f320ca33d977e29155d12 2024-07-29T00:12:26.733 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-07-29T00:12:26.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum upgrade -y linux-firmware 2024-07-29T00:12:29.804 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-07-29T00:12:34.004 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 20 MB 00:03 2024-07-29T00:12:38.905 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.5 MB 00:01 2024-07-29T00:12:40.597 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 18 kB 00:00 2024-07-29T00:12:41.442 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-07-29T00:12:46.815 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-29T00:12:48.896 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: linux-firmware noarch 20240716-145.el9 baseos 383 M 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 1 Package 2024-07-29T00:12:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:12:48.903 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 383 M 2024-07-29T00:12:48.903 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-07-29T00:12:56.358 INFO:teuthology.orchestra.run.smithi175.stdout:linux-firmware-20240716-145.el9.noarch.rpm 52 MB/s | 383 MB 00:07 2024-07-29T00:12:56.359 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-07-29T00:12:56.359 INFO:teuthology.orchestra.run.smithi175.stdout:Total 51 MB/s | 383 MB 00:07 2024-07-29T00:12:59.432 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-07-29T00:12:59.524 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-07-29T00:12:59.524 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-07-29T00:13:02.828 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-07-29T00:13:02.829 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-07-29T00:13:11.803 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-07-29T00:13:12.520 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : linux-firmware-20240716-145.el9.noarch 1/2 2024-07-29T00:13:12.602 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-29T00:13:13.407 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-07-29T00:13:13.407 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 1/2 2024-07-29T00:13:14.100 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-29T00:13:14.100 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:13:14.100 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-07-29T00:13:14.100 INFO:teuthology.orchestra.run.smithi175.stdout: linux-firmware-20240716-145.el9.noarch 2024-07-29T00:13:14.101 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:13:14.101 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-07-29T00:13:14.425 INFO:teuthology.task.kernel:Downloading kernel 3e9255e907100f88049f320ca33d977e29155d12 on client.0... 2024-07-29T00:13:14.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3e9255e907100f88049f320ca33d977e29155d12 2024-07-29T00:13:14.571 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3e9255e907100f88049f320ca33d977e29155d12/centos/9/flavors/default/x86_64/ 2024-07-29T00:13:14.571 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.10.0_g3e9255e90710-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/3e9255e907100f88049f320ca33d977e29155d12/centos/9/flavors/default/x86_64/ --input-file=- 2024-07-29T00:13:14.574 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi175.front.sepia.ceph.com to complete... 2024-07-29T00:13:20.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29 00:13:20 URL:https://1.chacra.ceph.com/r/kernel/testing/3e9255e907100f88049f320ca33d977e29155d12/centos/9/flavors/default/x86_64/kernel-6.10.0_g3e9255e90710-1.x86_64.rpm [358399635/358399635] -> "/tmp/kernel.x86_64.rpm" [1] 2024-07-29T00:13:20.867 INFO:teuthology.orchestra.run.smithi175.stderr:FINISHED --2024-07-29 00:13:20-- 2024-07-29T00:13:20.867 INFO:teuthology.orchestra.run.smithi175.stderr:Total wall clock time: 6.3s 2024-07-29T00:13:20.867 INFO:teuthology.orchestra.run.smithi175.stderr:Downloaded: 1 files, 342M in 6.0s (57.0 MB/s) 2024-07-29T00:13:20.870 INFO:teuthology.task.kernel:Installing kernel 3e9255e907100f88049f320ca33d977e29155d12 on client.0... 2024-07-29T00:13:20.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-07-29T00:13:22.927 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying... ######################################## 2024-07-29T00:13:23.053 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing... ######################################## 2024-07-29T00:13:23.103 INFO:teuthology.orchestra.run.smithi175.stdout:Updating / installing... 2024-07-29T00:13:31.128 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-6.10.0_g3e9255e90710-1 ######################################## 2024-07-29T00:14:10.547 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-07-29T00:14:10.547 DEBUG:teuthology.orchestra.run.smithi175:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-07-29T00:14:10.621 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/System.map-6.10.0-g3e9255e90710 2024-07-29T00:14:10.621 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/config-6.10.0-g3e9255e90710 2024-07-29T00:14:10.621 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/vmlinuz-6.10.0-g3e9255e90710 2024-07-29T00:14:10.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710 2024-07-29T00:14:10.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/System.map 2024-07-29T00:14:10.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/config 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-07-29T00:14:10.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/crc32c-intel.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/serpent-avx2.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-07-29T00:14:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kernel 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kernel/cpu 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kernel/cpu/mce 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kvm 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kvm/kvm-amd.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kvm/kvm-intel.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/arch/x86/kvm/kvm.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/ansi_cprng.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/anubis.ko 2024-07-29T00:14:10.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/arc4.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx/async_memcpy.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx/async_pq.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx/async_raid6_recov.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx/async_tx.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx/async_xor.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/async_tx/raid6test.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/authenc.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/authencesn.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/blake2b_generic.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/blowfish_common.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/blowfish_generic.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/camellia_generic.ko 2024-07-29T00:14:10.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/cast5_generic.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/cast6_generic.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/cast_common.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/ccm.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/cmac.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/crc32_generic.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/crypto_engine.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/crypto_user.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/cts.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/des_generic.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/dh_generic.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/ecc.ko 2024-07-29T00:14:10.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/ecdh_generic.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/echainiv.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/essiv.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/fcrypt.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/khazad.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/lz4.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/lz4hc.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/md4.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/michael_mic.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/pcbc.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/pcrypt.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/rmd160.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/seed.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/serpent_generic.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/tcrypt.ko 2024-07-29T00:14:10.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/tea.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/twofish_common.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/twofish_generic.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/vmac.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/wp512.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/xcbc.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/xor.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/crypto/xxhash_generic.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/acpi_ipmi.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/acpi_pad.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/ec_sys.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/platform_profile.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/sbs.ko 2024-07-29T00:14:10.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/sbshc.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/acpi/video.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/acard-ahci.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/ahci_platform.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/ata_generic.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/libahci_platform.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_acpi.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_ali.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_amd.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_artop.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_atiixp.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_atp867x.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_cmd640.ko 2024-07-29T00:14:10.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_cmd64x.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_cypress.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_efar.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_hpt366.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_hpt37x.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_hpt3x2n.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_hpt3x3.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_it8213.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_it821x.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_jmicron.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_marvell.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_mpiix.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_netcell.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_ninja32.ko 2024-07-29T00:14:10.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_ns87410.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_ns87415.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_oldpiix.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_opti.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_optidma.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_pcmcia.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_pdc2027x.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_piccolo.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_rdc.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_sch.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_serverworks.ko 2024-07-29T00:14:10.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_sil680.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_sis.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_sl82c105.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_triflex.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pata_via.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/pdc_adma.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_inic162x.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_mv.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_nv.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_promise.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_qstor.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_sil.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_sil24.ko 2024-07-29T00:14:10.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_sis.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_svw.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_sx4.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_uli.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_via.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ata/sata_vsc.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/auxdisplay 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/auxdisplay/ks0108.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/base 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/base/regmap 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/base/regmap/regmap-i2c.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/base/regmap/regmap-mmio.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/bcma 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/bcma/bcma.ko 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/aoe 2024-07-29T00:14:10.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/aoe/aoe.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/brd.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/drbd 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/drbd/drbd.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/floppy.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/loop.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/mtip32xx 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/nbd.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/null_blk 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/null_blk/null_blk.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/pktcdvd.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/rbd.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/virtio_blk.ko 2024-07-29T00:14:10.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/xen-blkback 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/xen-blkfront.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/zram 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/block/zram/zram.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hangcheck-timer.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hw_random 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hw_random/amd-rng.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hw_random/intel-rng.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hw_random/via-rng.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/hw_random/virtio-rng.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-07-29T00:14:10.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi/ipmi_si.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/lp.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/mwave 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/mwave/mwave.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/ppdev.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tlclk.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm/tpm.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm/tpm_atmel.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm/tpm_infineon.ko 2024-07-29T00:14:10.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm/tpm_nsc.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm/tpm_tis.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/uv_mmtimer.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/char/virtio_console.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq/p4-clockmod.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq/powernow-k8.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/cpufreq/speedstep-lib.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/ccp 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/ccp/ccp.ko 2024-07-29T00:14:10.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/intel 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/intel/qat 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/intel/qat/qat_common 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/padlock-aes.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/padlock-sha.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/virtio 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dca 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dca/dca.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/devfreq 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-07-29T00:14:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dma 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dma/dw 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dma/dw/dw_dmac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dma/ioat 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/dma/ioat/ioatdma.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/amd64_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/e752x_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/edac_mce_amd.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i3000_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i3200_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i5100_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i5400_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i7300_edac.ko 2024-07-29T00:14:10.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i7core_edac.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/i82975x_edac.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/ie31200_edac.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/sb_edac.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/edac/x38_edac.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/firmware 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/firmware/edd.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/firmware/iscsi_ibft.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/gpio 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/gpio/gpio-ich.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/gpio/gpio-viperboard.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-appleir.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-aureal.ko 2024-07-29T00:14:10.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-axff.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-dr.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-elecom.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-elo.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-emsff.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-gaff.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-gt683r.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-gyration.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-holtek-kbd.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-holtek-mouse.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-holtekff.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-hyperv.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-icade.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-keytouch.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-kye.ko 2024-07-29T00:14:10.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-lcpower.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-led.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-lenovo.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-logitech-dj.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-multitouch.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-ortek.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-penmount.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-petalynx.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-picolcd.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-pl.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-plantronics.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-primax.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-rmi.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-arvo.ko 2024-07-29T00:14:10.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-common.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-isku.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-kone.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-konepure.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-lua.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-pyra.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-ryos.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat-savu.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-roccat.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-saitek.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-samsung.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-sensor-hub.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-sjoy.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-sony.ko 2024-07-29T00:14:10.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-speedlink.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-steelseries.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-sunplus.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-thrustmaster.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-tivo.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-tmff.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-topseed.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-twinhan.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-uclogic.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-waltop.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-wiimote.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-xinmo.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-zpff.ko 2024-07-29T00:14:10.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/hid-zydacron.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/uhid.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hid/wacom.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hv 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hv/hv_balloon.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hv/hv_utils.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hv/hv_vmbus.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/abituguru.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/abituguru3.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/acpi_power_meter.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ad7414.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ad7418.ko 2024-07-29T00:14:10.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adc128d818.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adm1025.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adm1026.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adm1029.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adm1031.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adm9240.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ads7828.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adt7410.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adt7411.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adt7462.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adt7470.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adt7475.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/adt7x10.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/amc6821.ko 2024-07-29T00:14:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/applesmc.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/asb100.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/asc7621.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/asus_atk0110.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/atxp1.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/coretemp.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/dme1737.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ds1621.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ds620.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/emc1403.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/emc6w201.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/f71805f.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/f71882fg.ko 2024-07-29T00:14:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/f75375s.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/fam15h_power.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/fschmd.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/g760a.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/g762.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/gl518sm.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/gl520sm.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/hwmon-vid.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/i5500_temp.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/i5k_amb.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ibmaem.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ibmpex.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ina209.ko 2024-07-29T00:14:10.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ina2xx.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/it87.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/k10temp.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/k8temp.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lineage-pem.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm63.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm73.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm75.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm77.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm78.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm80.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm83.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm85.ko 2024-07-29T00:14:10.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm87.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm90.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm92.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm93.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm95234.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm95241.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/lm95245.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc2945.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc4151.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc4215.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc4222.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc4245.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc4260.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/ltc4261.ko 2024-07-29T00:14:10.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max16065.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max1619.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max1668.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max197.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max6639.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max6650.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/max6697.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/mcp3021.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/nct6683.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/nct6775-core.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/nct6775.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/nct7802.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pc87360.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pc87427.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pcf8591.ko 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus 2024-07-29T00:14:10.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/max16064.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/max34440.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/max8688.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/powr1220.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/sch5627.ko 2024-07-29T00:14:10.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/sch5636.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/sch56xx-common.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/sht15.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/sht21.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/shtc1.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/sis5595.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/smsc47b397.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/smsc47m1.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/smsc47m192.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/thmc50.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/tmp102.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/tmp103.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/tmp401.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/tmp421.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/via-cputemp.ko 2024-07-29T00:14:10.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/via686a.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/vt1211.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/vt8231.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83627ehf.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83627hf.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83781d.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83791d.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83792d.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83793.ko 2024-07-29T00:14:10.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83795.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83l785ts.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/hwmon/w83l786ng.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/algos 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-07-29T00:14:10.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-i801.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-isch.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-parport.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-via.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-07-29T00:14:10.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/i2c-dev.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/i2c-mux.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/i2c-smbus.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/i2c/i2c-stub.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/ib_cm.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/ib_core.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/ib_umad.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/iw_cm.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/rdma_cm.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-07-29T00:14:10.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/cxgb4 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/mlx4 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/mlx5 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/mthca 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/ipoib 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/iser 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-07-29T00:14:10.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/isert 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/srp 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/srpt 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/gameport 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/gameport/fm801-gp.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/gameport/gameport.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/joydev.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/keyboard 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/keyboard/gpio_keys.ko 2024-07-29T00:14:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/appletouch.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/bcm5974.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/cyapatp.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/elan_i2c.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/sermouse.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/synaptics_usb.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/mouse/vsxxxaa.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/rmi4 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/rmi4/rmi_core.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/serio 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/serio/altera_ps2.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/serio/arc_ps2.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-07-29T00:14:10.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/serio/serio_raw.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/input/sparse-keymap.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/leds-blinkm.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/leds-lm3530.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/leds-lp3944.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/leds-ss4200.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-07-29T00:14:10.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/bcache 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/bcache/bcache.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-bio-prison.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-cache-smq.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-cache.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-crypt.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-delay.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-flakey.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-log-userspace.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-multipath.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-queue-length.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-raid.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-round-robin.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-service-time.ko 2024-07-29T00:14:10.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-switch.ko 2024-07-29T00:14:10.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-thin-pool.ko 2024-07-29T00:14:10.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/dm-verity.ko 2024-07-29T00:14:10.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/persistent-data 2024-07-29T00:14:10.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/raid0.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/raid1.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/raid10.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/md/raid456.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptbase.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptctl.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptfc.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptlan.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptsas.ko 2024-07-29T00:14:10.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptscsih.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/message/fusion/mptspi.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/lpc_ich.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/lpc_sch.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/mfd-core.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/sm501.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/viperboard.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/vx855.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/mfd/wl1273-core.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/altera-stapl 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/apds9802als.ko 2024-07-29T00:14:10.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/apds990x.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/bh1770glc.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/cb710 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/cb710/cb710.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/eeprom 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/eeprom/at24.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/eeprom/max6875.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/enclosure.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/hpilo.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/isl29003.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/isl29020.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/lis3lv02d 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-07-29T00:14:10.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/mei 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/mei/mei-me.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/mei/mei-txe.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/mei/mei.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/sgi-gru 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/sgi-gru/gru.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/sgi-xp 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/sgi-xp/xp.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/sgi-xp/xpc.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/tifm_7xx1.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/tifm_core.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/tsl2550.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/vmw_balloon.ko 2024-07-29T00:14:10.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/vmw_vmci 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/bonding 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/bonding/bonding.ko 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa/b53 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa/b53/b53_common.ko 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa/bcm-sf2.ko 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa/mv88e6060.ko 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa/mv88e6xxx 2024-07-29T00:14:10.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/dummy.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/eql.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/3com 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/8390 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/8390/8390.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-07-29T00:14:10.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/adaptec 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/agere 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/agere/et131x.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/alteon 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/altera 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/amd 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/alx 2024-07-29T00:14:10.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atl1c 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atl1e 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atlx 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-07-29T00:14:10.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/genet 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/brocade 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/brocade/bna 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-07-29T00:14:10.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/cisco 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/cisco/enic 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-07-29T00:14:10.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dlink 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/dnet.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/emulex 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/emulex/benet 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/ethoc.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/fealnx.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/e100.ko 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/e1000 2024-07-29T00:14:10.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/e1000e 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/fm10k 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/i40e 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/iavf 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/igb 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/igbvf 2024-07-29T00:14:10.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/ixgbe 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/ixgbevf 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/libeth 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/libie 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/jme.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/marvell 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/marvell/skge.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-07-29T00:14:10.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox/mlx4 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox/mlx5 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/micrel 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/myricom 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/myricom/myri10ge 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/natsemi 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-07-29T00:14:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/neterion 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/nvidia 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/packetengines 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/qlogic 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/qlogic/netxen 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-07-29T00:14:10.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/rdc 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/realtek 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/realtek/atp.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/rocker 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sfc 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/silan 2024-07-29T00:14:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sis 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sis/sis190.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sis/sis900.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/smsc 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/stmicro 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/stmicro/stmmac 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sun 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sun/cassini.ko 2024-07-29T00:14:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sun/niu.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sun/sungem.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/tehuti 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/ti 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/ti/tlan.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/via 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/wiznet 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/xircom 2024-07-29T00:14:10.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/geneve.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/hyperv 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ieee802154 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ieee802154/fakelb.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ifb.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ipvlan 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ipvlan/ipvlan.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/macvlan.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/macvtap.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/mdio 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/mdio.ko 2024-07-29T00:14:10.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/mdio/mdio-regmap.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/mii.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/net_failover.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/netconsole.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/nlmon.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/pcs 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/pcs/pcs-lynx.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/amd.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-07-29T00:14:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/bcm7xxx.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/bcm87xx.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/broadcom.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/cicada.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/davicom.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/dp83640.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/et1011c.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/icplus.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/lxt.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/marvell.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/micrel.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/national.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/phylink.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/qsemi.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/realtek.ko 2024-07-29T00:14:10.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/smsc.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/ste10Xp.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/phy/vitesse.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/bsd_comp.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/ppp_async.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/ppp_deflate.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/ppp_generic.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/ppp_mppe.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/ppp_synctty.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/pppoe.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/pppox.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/ppp/pptp.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/slip 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/slip/slhc.ko 2024-07-29T00:14:10.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/slip/slip.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/sungem_phy.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/tap.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team/team.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team/team_mode_activebackup.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team/team_mode_broadcast.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team/team_mode_random.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/tun.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/veth.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/virtio_net.ko 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/vmxnet3 2024-07-29T00:14:10.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/vxlan 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/vxlan/vxlan.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/xen-netback 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/xen-netback/xen-netback.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/net/xen-netfront.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ntb 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ntb/ntb.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/nvme 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/nvme/host 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/nvme/host/nvme-core.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/nvme/host/nvme.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/parport 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/parport/parport.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/parport/parport_cs.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/parport/parport_pc.ko 2024-07-29T00:14:10.687 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/parport/parport_serial.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pci 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pci/hotplug 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pci/pcie 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pci/pcie/aer_inject.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pci/xen-pcifront.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pcmcia 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pcmcia/i82092.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pcmcia/pd6729.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pcmcia/yenta_socket.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pinctrl 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pinctrl/intel 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform 2024-07-29T00:14:10.688 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/acer-wmi.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/acerhdf.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/apple-gmux.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/asus-laptop.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/asus-wmi.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/classmate-laptop.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/hdaps.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/intel 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/intel_ips.ko 2024-07-29T00:14:10.689 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/msi-wmi.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/mxm-wmi.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/samsung-laptop.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/samsung-q10.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/topstar-laptop.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/toshiba_haps.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/wmi-bmof.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/platform/x86/wmi.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/powercap 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/powercap/intel_rapl_common.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/powercap/intel_rapl_msr.ko 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pps 2024-07-29T00:14:10.690 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pps/clients 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pps/clients/pps-gpio.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pps/clients/pps-ldisc.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pps/clients/pps_parport.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/pps/pps_core.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ptp 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ptp/ptp.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ptp/ptp_kvm.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ras 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ras/amd 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ras/amd/atl 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ras/amd/fmpm.ko 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc 2024-07-29T00:14:10.691 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-bq32k.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1286.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1307.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1374.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1511.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1553.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1672.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds1742.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds2404.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-ds3232.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-em3027.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-fm3130.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-isl12022.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-isl1208.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-m41t80.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-m48t35.ko 2024-07-29T00:14:10.692 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-m48t59.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-max6900.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-msm6242.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-pcf2127.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-pcf85063.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-pcf8523.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-pcf8563.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-pcf8583.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-rp5c01.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-rs5c372.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-rx8025.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-rx8581.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-07-29T00:14:10.693 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/rtc/rtc-x1205.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/3w-9xxx.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/3w-sas.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/3w-xxxx.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/BusLogic.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/a100u2w.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/aacraid 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/aacraid/aacraid.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/advansys.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/aic7xxx 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-07-29T00:14:10.694 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/am53c974.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/arcmsr 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/atp870u.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/be2iscsi 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/bfa 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/bfa/bfa.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/bnx2fc 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/bnx2i 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/ch.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/csiostor 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/csiostor/csiostor.ko 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/cxgbi 2024-07-29T00:14:10.695 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/cxgbi/cxgb3i 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/cxgbi/cxgb4i 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/dc395x.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/device_handler 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/dmx3191d.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/esas2r 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/esas2r/esas2r.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/esp_scsi.ko 2024-07-29T00:14:10.696 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/fcoe 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/fcoe/fcoe.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/fnic 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/fnic/fnic.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/hpsa.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/hptiop.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/hv_storvsc.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/initio.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/ipr.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/ips.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/isci 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/isci/isci.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-07-29T00:14:10.697 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/iscsi_tcp.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/libfc 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/libfc/libfc.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/libiscsi.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/libiscsi_tcp.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/libsas 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/libsas/libsas.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/lpfc 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/lpfc/lpfc.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/megaraid 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/megaraid.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-07-29T00:14:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/mpt3sas 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/mvsas 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/mvsas/mvsas.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/mvumi.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/pm8001 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/pmcraid.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/qla1280.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/qla2xxx 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/qla4xxx 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-07-29T00:14:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/raid_class.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/scsi_debug.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/scsi_transport_fc.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/scsi_transport_sas.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/scsi_transport_spi.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/scsi_transport_srp.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/ses.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/st.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/stex.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/sym53c8xx_2 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/virtio_scsi.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/vmw_pvscsi.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/scsi/wd719x.ko 2024-07-29T00:14:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ssb 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ssb/ssb.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/iscsi 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/loopback 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/loopback/tcm_loop.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/target_core_file.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/target_core_iblock.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/target_core_mod.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/target_core_pscsi.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/target_core_user.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/tcm_fc 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel 2024-07-29T00:14:10.701 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-07-29T00:14:10.702 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/ipwireless 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/n_gsm.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/n_hdlc.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/nozomi.ko 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/serial 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/serial/8250 2024-07-29T00:14:10.703 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/serial/arc_uart.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/serial/jsm 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/tty/serial/jsm/jsm.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ufs 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ufs/core 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ufs/core/ufshcd-core.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ufs/host 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/uio 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/uio/uio.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/uio/uio_aec.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/uio/uio_cif.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/uio/uio_pci_generic.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/uio/uio_sercos3.ko 2024-07-29T00:14:10.704 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/class 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/class/cdc-acm.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/class/cdc-wdm.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/class/usblp.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/class/usbtmc.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/host 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/host/sl811-hcd.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/image 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/image/mdc800.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/image/microtek.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/adutux.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/appledisplay.ko 2024-07-29T00:14:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/emi26.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/emi62.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/ezusb.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/idmouse.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/iowarrior.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/isight_firmware.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/ldusb.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/legousbtower.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/sisusbvga 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/trancevibrator.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/usb3503.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/usblcd.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/usbsevseg.ko 2024-07-29T00:14:10.706 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/uss720.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/misc/yurex.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/phy 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/phy/phy-generic.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/aircable.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ark3116.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/belkin_sa.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ch341.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/cp210x.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/cyberjack.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/cypress_m8.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/digi_acceleport.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/empeg.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ftdi_sio.ko 2024-07-29T00:14:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/garmin_gps.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/io_edgeport.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/io_ti.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ipaq.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ipw.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ir-usb.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/keyspan.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/keyspan_pda.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/kl5kusb105.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/kobil_sct.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/mct_u232.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/mos7720.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/mos7840.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/navman.ko 2024-07-29T00:14:10.708 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/omninet.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/opticon.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/option.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/oti6858.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/pl2303.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/qcaux.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/qcserial.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/quatech2.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/safe_serial.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/sierra.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/spcp8x5.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ssu100.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/symbolserial.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-07-29T00:14:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/usb_debug.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/usb_wwan.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/visor.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/whiteheat.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/serial/xsens_mt.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/uas.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-alauda.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-cypress.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-datafab.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-freecom.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-isd200.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-07-29T00:14:10.710 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-karma.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-onetouch.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-realtek.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-sddr09.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-sddr55.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/ums-usbat.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/storage/usb-storage.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/usbip 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/usbip/usbip-core.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/usbip/usbip-host.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vfio 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vfio/pci 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-07-29T00:14:10.711 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vfio/pci/vfio-pci.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vfio/vfio.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vhost 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vhost/vhost.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vhost/vhost_iotlb.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vhost/vhost_net.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/vhost/vhost_scsi.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/video 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/video/backlight 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/video/backlight/apple_bl.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/video/backlight/lcd.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/video/backlight/platform_lcd.ko 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio 2024-07-29T00:14:10.712 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio_balloon.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio_mmio.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio_pci.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/virtio/virtio_ring.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/masters 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/masters/ds2482.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/masters/ds2490.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-07-29T00:14:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_smem.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/slaves/w1_therm.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/w1/wire.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/alim1535_wdt.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/alim7101_wdt.ko 2024-07-29T00:14:10.714 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/f71808e_wdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/hpwdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/i6300esb.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/iTCO_wdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/ib700wdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/ibmasr.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/it8712f_wdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/it87_wdt.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/machzwd.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/nv_tco.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/pcwd_pci.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/pcwd_usb.ko 2024-07-29T00:14:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/sch311x_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/softdog.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/sp5100_tco.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/via_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/w83877f_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/w83977f_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/wdt_pci.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/watchdog/xen_wdt.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-acpi-processor.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-evtchn.ko 2024-07-29T00:14:10.716 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-gntalloc.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-gntdev.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-pciback 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xen-privcmd.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xenfs 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/drivers/xen/xenfs/xenfs.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/9p 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/9p/9p.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/affs 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/affs/affs.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/befs 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/befs/befs.ko 2024-07-29T00:14:10.717 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/binfmt_misc.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/btrfs 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/btrfs/btrfs.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/cachefiles 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/cachefiles/cachefiles.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ceph 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ceph/ceph.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/coda 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/coda/coda.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/cramfs 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/cramfs/cramfs.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/dlm 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/dlm/dlm.ko 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ecryptfs 2024-07-29T00:14:10.718 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ecryptfs/ecryptfs.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/f2fs 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/f2fs/f2fs.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fat 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fat/fat.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fat/msdos.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fat/vfat.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fuse 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fuse/cuse.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/fuse/fuse.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/gfs2 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/gfs2/gfs2.ko 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/hfs 2024-07-29T00:14:10.719 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/hfs/hfs.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/hfsplus 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/hfsplus/hfsplus.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/isofs 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/isofs/isofs.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/jfs 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/jfs/jfs.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/lockd 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/lockd/lockd.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/minix 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/minix/minix.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/netfs 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/netfs/netfs.ko 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs 2024-07-29T00:14:10.720 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/blocklayout 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/filelayout 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/flexfilelayout 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/nfs.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/nfsv3.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs/nfsv4.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs_common 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs_common/grace.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfs_common/nfs_acl.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfsd 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nfsd/nfsd.ko 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nilfs2 2024-07-29T00:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nilfs2/nilfs2.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-celtic.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-centeuro.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-croatian.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-cyrillic.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-gaelic.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-greek.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-iceland.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-inuit.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-roman.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-romanian.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/mac-turkish.ko 2024-07-29T00:14:10.722 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp1250.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp1251.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp1255.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp737.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp775.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp850.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp852.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp855.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp857.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp860.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp861.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp862.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp863.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp864.ko 2024-07-29T00:14:10.723 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp865.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp866.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp869.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp874.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp932.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp936.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp949.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_cp950.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_euc-jp.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-1.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-13.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-14.ko 2024-07-29T00:14:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-15.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-2.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-3.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-4.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-5.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-6.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-7.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_iso8859-9.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_koi8-r.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_koi8-ru.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_koi8-u.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_ucs2_utils.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/nls/nls_utf8.ko 2024-07-29T00:14:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/cluster 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/dlm 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/dlmfs 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/ocfs2.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/overlayfs 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/overlayfs/overlay.ko 2024-07-29T00:14:10.726 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/pstore 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/pstore/ramoops.ko 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/reiserfs 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/reiserfs/reiserfs.ko 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/romfs 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/romfs/romfs.ko 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/smb 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/smb/client 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/smb/client/cifs.ko 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/smb/common 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/smb/common/cifs_arc4.ko 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/smb/common/cifs_md4.ko 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/squashfs 2024-07-29T00:14:10.727 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/squashfs/squashfs.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/sysv 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/sysv/sysv.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/udf 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/udf/udf.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ufs 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/ufs/ufs.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/xfs 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/fs/xfs/xfs.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/kernel 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/kernel/rcu 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/kernel/rcu/rcutorture.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/kernel/torture.ko 2024-07-29T00:14:10.728 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/kernel/trace 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/asn1_encoder.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/crc-itu-t.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/crc8.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/crypto 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/crypto/libaescfb.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/crypto/libarc4.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/crypto/libdes.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/dim 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/dim/dimlib.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/libcrc32c.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/lru_cache.ko 2024-07-29T00:14:10.729 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/lz4 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/lz4/lz4_compress.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/lz4/lz4hc_compress.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/math 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/math/cordic.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/raid6 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/raid6/raid6_pq.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/reed_solomon 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/reed_solomon/reed_solomon.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/ts_bm.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/ts_fsm.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/ts_kmp.ko 2024-07-29T00:14:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/zstd 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/lib/zstd/zstd_compress.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/mm 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/mm/hwpoison-inject.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/6lowpan.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_dest.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_fragment.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_hop.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_ipv6.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_mobility.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_routing.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/6lowpan/nhc_udp.ko 2024-07-29T00:14:10.731 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/802 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/802/garp.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/802/mrp.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/802/p8022.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/802/psnap.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/802/stp.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/8021q 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/8021q/8021q.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/9p 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/9p/9pnet.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/9p/9pnet_fd.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/9p/9pnet_rdma.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/9p/9pnet_virtio.ko 2024-07-29T00:14:10.732 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/batman-adv 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/batman-adv/batman-adv.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/br_netfilter.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/bridge.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_among.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_arp.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_ip.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_limit.ko 2024-07-29T00:14:10.733 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_log.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_mark.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_snat.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_stp.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/ebtables.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-07-29T00:14:10.734 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ceph 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ceph/libceph.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/core 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/core/failover.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/core/pktgen.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dccp 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dccp/dccp.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dccp/dccp_diag.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dccp/dccp_ipv4.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dccp/dccp_ipv6.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dns_resolver 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dns_resolver/dns_resolver.ko 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dsa 2024-07-29T00:14:10.735 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dsa/dsa_core.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dsa/tag_brcm.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dsa/tag_dsa.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dsa/tag_none.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/dsa/tag_trailer.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ieee802154 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ieee802154/6lowpan 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ieee802154/ieee802154.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ieee802154/ieee802154_socket.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/ah4.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/esp4.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/fou.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/gre.ko 2024-07-29T00:14:10.736 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/inet_diag.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/ip_gre.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/ip_tunnel.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/ip_vti.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/ipcomp.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/ipip.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/arp_tables.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-07-29T00:14:10.737 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/iptable_security.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-07-29T00:14:10.738 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_bic.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_dctcp.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_diag.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_highspeed.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_htcp.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_hybla.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_illinois.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_lp.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_scalable.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_vegas.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_veno.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_westwood.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tcp_yeah.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/tunnel4.ko 2024-07-29T00:14:10.739 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/udp_diag.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/udp_tunnel.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv4/xfrm4_tunnel.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/ah6.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/esp6.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/fou6.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/ip6_tunnel.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/ip6_vti.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/ipcomp6.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-07-29T00:14:10.740 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-07-29T00:14:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/sit.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/tunnel6.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/ipv6/xfrm6_tunnel.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/key 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/key/af_key.ko 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp 2024-07-29T00:14:10.742 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_core.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_debugfs.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_eth.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_ip.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_ip6.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_netlink.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/l2tp/l2tp_ppp.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/llc 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/llc/llc.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/mac802154 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/mac802154/mac802154.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/mpls 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/mpls/mpls_gso.ko 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset 2024-07-29T00:14:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-07-29T00:14:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-07-29T00:14:10.745 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conncount.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_h323.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_irc.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_sane.ko 2024-07-29T00:14:10.746 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_sip.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_dup_netdev.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_flow_table.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_flow_table_inet.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_log_syslog.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_nat.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_nat_amanda.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_nat_ftp.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_nat_irc.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_nat_sip.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_nat_tftp.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_synproxy_core.ko 2024-07-29T00:14:10.747 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nf_tables.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nfnetlink.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nfnetlink_acct.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nfnetlink_log.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nfnetlink_osf.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nfnetlink_queue.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_chain_nat.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_compat.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_ct.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_dup_netdev.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_fib.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_fwd_netdev.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_hash.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_limit.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_log.ko 2024-07-29T00:14:10.748 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_masq.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_nat.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_numgen.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_queue.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_quota.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_redir.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_reject.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_reject_inet.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_socket.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/nft_tunnel.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_AUDIT.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_CHECKSUM.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_CLASSIFY.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-07-29T00:14:10.749 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_CT.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_DSCP.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_HL.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_HMARK.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_IDLETIMER.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_LED.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_LOG.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_MASQUERADE.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_NETMAP.ko 2024-07-29T00:14:10.750 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_NFLOG.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_NFQUEUE.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_RATEEST.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_REDIRECT.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_SECMARK.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_TCPMSS.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_TEE.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_TPROXY.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_TRACE.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_addrtype.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_bpf.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_cgroup.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_cluster.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_comment.ko 2024-07-29T00:14:10.751 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_connbytes.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_connlabel.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_connlimit.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_connmark.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_conntrack.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_cpu.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_dccp.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_devgroup.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_dscp.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_ecn.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_esp.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_hashlimit.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_helper.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_hl.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_ipcomp.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_iprange.ko 2024-07-29T00:14:10.752 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_ipvs.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_l2tp.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_length.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_limit.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_mac.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_mark.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_multiport.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_nat.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_nfacct.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_osf.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_owner.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_physdev.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_pkttype.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_policy.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_quota.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_rateest.ko 2024-07-29T00:14:10.753 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_realm.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_recent.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_sctp.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_set.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_socket.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_state.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_statistic.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_string.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_tcpmss.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_time.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netfilter/xt_u32.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netlink 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/netlink/netlink_diag.ko 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/nsh 2024-07-29T00:14:10.754 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/nsh/nsh.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/openvswitch 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/openvswitch/openvswitch.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/openvswitch/vport-geneve.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/openvswitch/vport-gre.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/openvswitch/vport-vxlan.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/packet 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/packet/af_packet_diag.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/rds 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/rds/rds.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/rds/rds_rdma.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/rds/rds_tcp.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_csum.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_gact.ko 2024-07-29T00:14:10.755 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_mirred.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_nat.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_pedit.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_police.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_simple.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_skbedit.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/act_vlan.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/cls_basic.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/cls_bpf.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/cls_flow.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/cls_fw.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/cls_route.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/cls_u32.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/em_cmp.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/em_ipset.ko 2024-07-29T00:14:10.756 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/em_meta.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/em_nbyte.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/em_text.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/em_u32.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_choke.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_codel.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_drr.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_fq.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_gred.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_hfsc.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_hhf.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_htb.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_ingress.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_mqprio.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_mqprio_lib.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_multiq.ko 2024-07-29T00:14:10.757 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_netem.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_pie.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_plug.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_prio.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_qfq.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_red.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_sfb.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_sfq.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_tbf.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sched/sch_teql.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sctp 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sctp/sctp.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sctp/sctp_diag.ko 2024-07-29T00:14:10.758 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc/auth_gss 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc/sunrpc.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc/xprtrdma 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/tipc 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/tipc/tipc.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/tipc/tipc_diag.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/unix 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/unix/unix_diag.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/vmw_vsock 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-07-29T00:14:10.759 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/vmw_vsock/vsock.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/vmw_vsock/vsock_diag.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/vmw_vsock/vsock_loopback.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/xfrm 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/net/xfrm/xfrm_ipcomp.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/security 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/security/keys 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/security/keys/encrypted-keys 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/security/keys/trusted-keys 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/kernel/security/keys/trusted-keys/trusted.ko 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.alias 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.alias.bin 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.builtin 2024-07-29T00:14:10.760 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.builtin.alias.bin 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.builtin.bin 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.builtin.modinfo 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.dep 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.dep.bin 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.devname 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.order 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.softdep 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.symbols 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/modules.symbols.bin 2024-07-29T00:14:10.761 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-g3e9255e90710/vmlinuz 2024-07-29T00:14:10.765 DEBUG:teuthology.task.kernel:get_image_version: 6.10.0-g3e9255e90710 2024-07-29T00:14:10.765 DEBUG:teuthology.orchestra.run.smithi175:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout:postinstall scriptlet (using /bin/sh): 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/bin/kernel-install add 6.10.0-g3e9255e90710 /lib/modules/6.10.0-g3e9255e90710/vmlinuz 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout:for file in vmlinuz System.map config; do 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout: if ! cmp --silent "/lib/modules/6.10.0-g3e9255e90710/${file}" "/boot/${file}-6.10.0-g3e9255e90710"; then 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout: cp "/lib/modules/6.10.0-g3e9255e90710/${file}" "/boot/${file}-6.10.0-g3e9255e90710" 2024-07-29T00:14:10.789 INFO:teuthology.orchestra.run.smithi175.stdout: fi 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -d "/lib/modules/6.10.0-g3e9255e90710/dtb" ] && \ 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout: ! diff -rq "/lib/modules/6.10.0-g3e9255e90710/dtb" "/boot/dtb-6.10.0-g3e9255e90710" >/dev/null 2>&1; then 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout: rm -rf "/boot/dtb-6.10.0-g3e9255e90710" 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout: cp -r "/lib/modules/6.10.0-g3e9255e90710/dtb" "/boot/dtb-6.10.0-g3e9255e90710" 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:if [ ! -e "/lib/modules/6.10.0-g3e9255e90710/modules.dep" ]; then 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/sbin/depmod 6.10.0-g3e9255e90710 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:preuninstall scriptlet (using /bin/sh): 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-install remove 6.10.0-g3e9255e90710 2024-07-29T00:14:10.790 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-29T00:14:10.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-07-29T00:14:10.864 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-07-29T00:14:10.864 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-07-29T00:14:10.864 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 68.el9 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8264254 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-07-29T00:14:10.865 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:14:10.866 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-07-29T00:14:10.867 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-29T00:14:10.867 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 6.10.0-g3e9255e90710 2024-07-29T00:14:10.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-29T00:14:11.375 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-07-29T00:14:12.742 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-29T00:14:12.773 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-07-29T00:14:12.777 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-07-29T00:14:12.792 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.wMDTrO1bh6 2024-07-29T00:14:12.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wMDTrO1bh6 2024-07-29T00:14:12.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.wMDTrO1bh6 2024-07-29T00:14:13.003 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.wMDTrO1bh6 is 6KB 2024-07-29T00:14:13.053 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.wMDTrO1bh6 2024-07-29T00:14:13.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-07-29T00:14:13.130 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-29T00:14:13.130 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-29T00:14:13.130 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-29T00:14:13.130 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.10.0-g3e9255e90710.conf 2024-07-29T00:14:13.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.10.0-g3e9255e90710 2024-07-29T00:14:13.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-07-29T00:14:43.280 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-29T00:14:43.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-29T00:14:43.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:15:43.285 DEBUG:teuthology.orchestra.remote:timed out 2024-07-29T00:15:52.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-29T00:15:52.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:16:08.107 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-07-29T00:16:08.719 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-29T00:16:08.720 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.10.0-g3e9255e90710"... 2024-07-29T00:16:08.720 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-07-29T00:16:08.758 INFO:teuthology.orchestra.run.smithi175.stdout:6.10.0-g3e9255e90710 2024-07-29T00:16:08.758 DEBUG:teuthology.task.kernel:current kernel version is 6.10.0-g3e9255e90710 vs 6.10.0-g3e9255e90710 2024-07-29T00:16:08.759 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-29T00:16:08.759 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-29T00:16:09.762 DEBUG:teuthology.parallel:result is None 2024-07-29T00:16:09.762 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-29T00:16:09.771 INFO:teuthology.task.internal:Creating test directory... 2024-07-29T00:16:09.772 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-29T00:16:09.776 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-29T00:16:09.779 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-29T00:16:09.811 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-29T00:16:09.819 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-29T00:16:09.832 INFO:teuthology.task.internal:Creating archive directory... 2024-07-29T00:16:09.832 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-29T00:16:09.855 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-29T00:16:09.857 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-29T00:16:09.948 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-29T00:16:09.955 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-29T00:16:09.955 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-29T00:16:09.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:16:09.971 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-29T00:16:09.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:16:09.988 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-29T00:16:09.993 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-29T00:16:10.013 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-29T00:16:10.056 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-29T00:16:10.059 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-29T00:16:10.082 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-29T00:16:10.084 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-29T00:16:10.146 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-29T00:16:10.205 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-29T00:16:10.209 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-29T00:16:10.219 INFO:teuthology.task.internal:Configuring sudo... 2024-07-29T00:16:10.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-29T00:16:10.224 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-29T00:16:10.226 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-29T00:16:10.286 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-29T00:16:10.295 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-29T00:16:10.296 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-29T00:16:10.301 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-29T00:16:10.303 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-29T00:16:10.334 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-29T00:16:10.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-29T00:16:10.532 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-29T00:16:10.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-29T00:16:10.677 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:16:10.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-29T00:16:10.740 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-29T00:16:10.781 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-29T00:16:10.849 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-29T00:16:10.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-29T00:16:10.933 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:16:10.933 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-29T00:16:10.998 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-29T00:16:11.042 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-29T00:16:11.105 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-29T00:16:11.147 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-29T00:16:11.190 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:16:11.190 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-29T00:16:11.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-07-29T00:16:11.263 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-07-29T00:16:11.266 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-07-29T00:16:11.298 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-29T00:16:11.333 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-29T00:16:11.353 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-29T00:16:11.867 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-29T00:16:12.099 INFO:teuthology.task.internal:Starting timer... 2024-07-29T00:16:12.099 INFO:teuthology.run_tasks:Running task pcp... 2024-07-29T00:16:12.110 INFO:teuthology.run_tasks:Running task selinux... 2024-07-29T00:16:12.118 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-29T00:16:12.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-07-29T00:16:12.238 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-07-29T00:16:12.242 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-07-29T00:16:12.325 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-07-29T00:16:12.327 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-07-29T00:16:12.383 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-07-29T00:16:12.385 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-29T00:16:12.385 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-07-29T00:16:12.426 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-07-29T00:16:12.426 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-07-29T00:16:12.451 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-07-29T00:16:12.451 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-07-29T00:16:12.471 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-07-29T00:16:12.471 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi175.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-07-29T00:16:12.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-29T00:16:12.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:16:12.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-29T00:16:12.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:16:12.559 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-29T00:16:12.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:16:12.585 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-29T00:16:12.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-07-29T00:16:12.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-07-29T00:16:12.669 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-07-29T00:16:12.696 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-29T00:16:12.706 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-29T00:16:12.718 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-29T00:16:12.719 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi175.front.sepia.ceph.com,smithi181.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-29T00:22:48.899 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi175.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-07-29T00:22:48.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-29T00:22:48.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:22:49.013 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-07-29T00:22:49.168 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-29T00:22:49.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-07-29T00:22:49.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:22:49.247 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-07-29T00:22:49.327 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-07-29T00:22:49.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-07-29T00:22:49.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-29T00:22:49.404 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-07-29T00:22:49.483 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-07-29T00:22:49.483 INFO:teuthology.run_tasks:Running task clock... 2024-07-29T00:22:49.500 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-29T00:22:49.500 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-29T00:22:49.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-29T00:22:49.508 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-29T00:22:49.508 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-29T00:22:49.510 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-29T00:22:49.510 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-29T00:22:49.536 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-29T00:22:49.550 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-29T00:22:49.553 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-29T00:22:49.567 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-29T00:22:49.572 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-07-29T00:22:49.582 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-07-29T00:22:49.591 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-07-29T00:22:49.594 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-29T00:22:49.595 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-29T00:22:49.602 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-07-29T00:22:49.608 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-29T00:22:49.615 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-29T00:22:49.629 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-29T00:22:49.640 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-29T00:22:49.665 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-07-29T00:22:49.669 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-29T00:22:49.669 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-07-29T00:22:49.669 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.669 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.669 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.670 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.683 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-07-29T00:22:49.687 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-29T00:22:49.687 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-07-29T00:22:49.687 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.687 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.687 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.687 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-07-29T00:22:49.747 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-07-29T00:22:49.786 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-29T00:22:49.828 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-29T00:22:49.969 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-07-29T00:22:49.978 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-29T00:22:49.978 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-07-29T00:22:49.978 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.978 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.978 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.978 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:22:49.979 INFO:teuthology.run_tasks:Running task install... 2024-07-29T00:22:49.992 DEBUG:teuthology.task.install:project ceph 2024-07-29T00:22:49.993 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701'}} 2024-07-29T00:22:49.993 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701'} 2024-07-29T00:22:49.993 INFO:teuthology.task.install:Using flavor: default 2024-07-29T00:22:49.998 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-29T00:22:49.998 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-29T00:22:49.999 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701', 'tag': None, 'wait_for_package': False} 2024-07-29T00:22:49.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:22:50.001 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701', 'tag': None, 'wait_for_package': False} 2024-07-29T00:22:50.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:22:50.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701', 'tag': None, 'wait_for_package': False} 2024-07-29T00:22:50.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:22:50.158 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/ 2024-07-29T00:22:50.158 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1150.gba82afec 2024-07-29T00:22:50.162 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/ 2024-07-29T00:22:50.162 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1150.gba82afec 2024-07-29T00:22:50.170 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/ 2024-07-29T00:22:50.170 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1150.gba82afec 2024-07-29T00:22:50.275 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-29T00:22:50.276 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:22:50.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-29T00:22:50.284 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-29T00:22:50.285 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:22:50.285 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-29T00:22:50.292 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-29T00:22:50.292 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:22:50.292 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-29T00:22:50.312 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-29T00:22:50.313 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba82afec3628bf830c0830558bf3d3c3aea66701/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-29T00:22:50.320 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-29T00:22:50.320 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba82afec3628bf830c0830558bf3d3c3aea66701/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-29T00:22:50.369 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-29T00:22:50.369 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba82afec3628bf830c0830558bf3d3c3aea66701/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-29T00:22:50.379 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-29T00:22:50.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-29T00:22:50.454 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-29T00:22:50.462 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-29T00:22:50.497 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-29T00:22:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-07-29T00:22:50.518 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-07-29T00:22:50.527 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-07-29T00:22:50.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-07-29T00:22:50.607 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-29T00:22:50.688 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-07-29T00:22:50.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-07-29T00:22:50.743 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-07-29T00:22:50.755 INFO:teuthology.orchestra.run.smithi181.stdout:45 files removed 2024-07-29T00:22:50.770 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-29T00:22:50.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-29T00:22:51.007 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-07-29T00:22:51.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-29T00:22:51.349 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 267 kB/s | 84 kB 00:00 2024-07-29T00:22:51.380 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 250 kB/s | 84 kB 00:00 2024-07-29T00:22:51.683 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-07-29T00:22:51.739 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-07-29T00:22:51.849 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 218 kB/s | 84 kB 00:00 2024-07-29T00:22:51.991 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-07-29T00:22:52.064 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-07-29T00:22:52.316 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-07-29T00:22:52.766 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-07-29T00:22:52.858 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.2 MB 00:00 2024-07-29T00:22:53.548 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-07-29T00:22:53.689 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.2 MB 00:01 2024-07-29T00:22:55.574 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-07-29T00:23:00.166 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-07-29T00:23:00.765 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 20 MB 00:05 2024-07-29T00:23:01.784 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 18 kB 00:00 2024-07-29T00:23:02.923 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-07-29T00:23:06.007 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-07-29T00:23:07.758 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 18 kB 00:00 2024-07-29T00:23:08.595 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-07-29T00:23:09.235 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-07-29T00:23:11.783 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-29T00:23:11.784 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-29T00:23:11.803 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-29T00:23:11.804 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-29T00:23:11.806 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-29T00:23:11.810 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-29T00:23:11.810 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-29T00:23:11.813 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-29T00:23:11.814 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-29T00:23:11.885 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-07-29T00:23:11.904 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-29T00:23:11.904 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-07-29T00:23:11.904 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-29T00:23:11.904 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-07-29T00:23:11.904 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-29T00:23:11.904 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.1.0-1150.gba82afec.el9 ceph 6.6 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.1.0-1150.gba82afec.el9 ceph 5.4 M 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 896 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1150.gba82afec.el9 ceph 145 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.1.0-1150.gba82afec.el9 ceph 1.5 M 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 146 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 3.6 M 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 7.4 M 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 49 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.1.0-1150.gba82afec.el9 ceph 11 M 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.1.0-1150.gba82afec.el9 ceph 49 M 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 274 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 765 k 2024-07-29T00:23:11.905 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.1.0-1150.gba82afec.el9 ceph 225 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.1.0-1150.gba82afec.el9 ceph 27 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 18 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.1.0-1150.gba82afec.el9 ceph 32 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 753 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.1.0-1150.gba82afec.el9 ceph 127 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-29T00:23:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.1.0-1150.gba82afec.el9 ceph 188 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.1.0-1150.gba82afec.el9 ceph 371 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 349 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.1.0-1150.gba82afec.el9 ceph 116 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 85 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.1 M 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 170 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-29T00:23:11.907 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.4 M 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.2 M 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-29T00:23:11.908 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.1.0-1150.gba82afec.el9 ceph 21 M 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 32 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.1.0-1150.gba82afec.el9 ceph 2.4 M 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 249 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.1.0-1150.gba82afec.el9 ceph 4.7 M 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 17 M 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 16 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.1.0-1150.gba82afec.el9 ceph 25 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-29T00:23:11.909 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.1.0-1150.gba82afec.el9 ceph 163 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-29T00:23:11.910 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 507 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 5.4 M 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-29T00:23:11.911 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 45 k 2024-07-29T00:23:11.912 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.1.0-1150.gba82afec.el9 ceph 136 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-29T00:23:11.913 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-29T00:23:11.914 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-29T00:23:11.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-29T00:23:11.916 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-29T00:23:11.917 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 12 Packages 2024-07-29T00:23:11.918 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:23:11.919 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 228 M 2024-07-29T00:23:11.919 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-07-29T00:23:12.183 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-07-29T00:23:12.770 INFO:teuthology.orchestra.run.smithi175.stdout:(1/160): ceph-19.1.0-1150.gba82afec.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-07-29T00:23:13.336 INFO:teuthology.orchestra.run.smithi175.stdout:(2/160): ceph-fuse-19.1.0-1150.gba82afec.el9.x8 1.5 MB/s | 896 kB 00:00 2024-07-29T00:23:13.470 INFO:teuthology.orchestra.run.smithi175.stdout:(3/160): ceph-immutable-object-cache-19.1.0-115 1.1 MB/s | 145 kB 00:00 2024-07-29T00:23:14.042 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-07-29T00:23:14.345 INFO:teuthology.orchestra.run.smithi175.stdout:(4/160): ceph-mds-19.1.0-1150.gba82afec.el9.x86 2.7 MB/s | 2.4 MB 00:00 2024-07-29T00:23:14.562 INFO:teuthology.orchestra.run.smithi175.stdout:(5/160): ceph-base-19.1.0-1150.gba82afec.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-07-29T00:23:14.821 INFO:teuthology.orchestra.run.smithi175.stdout:(6/160): ceph-mgr-19.1.0-1150.gba82afec.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-07-29T00:23:15.463 INFO:teuthology.orchestra.run.smithi175.stdout:(7/160): ceph-mon-19.1.0-1150.gba82afec.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-07-29T00:23:16.271 INFO:teuthology.orchestra.run.smithi175.stdout:(8/160): ceph-common-19.1.0-1150.gba82afec.el9. 5.6 MB/s | 21 MB 00:03 2024-07-29T00:23:16.293 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-29T00:23:16.294 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-29T00:23:16.305 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-29T00:23:16.306 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-29T00:23:16.307 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-29T00:23:16.308 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-29T00:23:16.309 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-29T00:23:16.312 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-29T00:23:16.313 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-29T00:23:16.329 INFO:teuthology.orchestra.run.smithi175.stdout:(9/160): ceph-selinux-19.1.0-1150.gba82afec.el9 441 kB/s | 25 kB 00:00 2024-07-29T00:23:16.378 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-07-29T00:23:16.401 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.1.0-1150.gba82afec.el9 ceph 6.6 k 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.1.0-1150.gba82afec.el9 ceph 5.4 M 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 896 k 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1150.gba82afec.el9 ceph 145 k 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.1.0-1150.gba82afec.el9 ceph 1.5 M 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 146 k 2024-07-29T00:23:16.402 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 3.6 M 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 7.4 M 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 49 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.1.0-1150.gba82afec.el9 ceph 11 M 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.1.0-1150.gba82afec.el9 ceph 49 M 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 274 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 765 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.1.0-1150.gba82afec.el9 ceph 225 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell x86_64 2:19.1.0-1150.gba82afec.el9 ceph 27 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 18 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.1.0-1150.gba82afec.el9 ceph 32 k 2024-07-29T00:23:16.403 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 753 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.1.0-1150.gba82afec.el9 ceph 127 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.1.0-1150.gba82afec.el9 ceph 188 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.1.0-1150.gba82afec.el9 ceph 371 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 349 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.1.0-1150.gba82afec.el9 ceph 116 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 85 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.1 M 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 170 k 2024-07-29T00:23:16.404 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.4 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.2 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-29T00:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.1.0-1150.gba82afec.el9 ceph 21 M 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 32 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.1.0-1150.gba82afec.el9 ceph 2.4 M 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 249 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.1.0-1150.gba82afec.el9 ceph 4.7 M 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 17 M 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 16 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.1.0-1150.gba82afec.el9 ceph 25 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-29T00:23:16.406 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.1.0-1150.gba82afec.el9 ceph 163 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-29T00:23:16.407 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 507 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 5.4 M 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-29T00:23:16.408 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 45 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.1.0-1150.gba82afec.el9 ceph 136 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-29T00:23:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-29T00:23:16.410 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-29T00:23:16.411 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-29T00:23:16.412 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-29T00:23:16.413 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout:Install 148 Packages 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 12 Packages 2024-07-29T00:23:16.414 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:23:16.415 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 228 M 2024-07-29T00:23:16.415 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-07-29T00:23:16.667 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-07-29T00:23:16.788 INFO:teuthology.orchestra.run.smithi175.stdout:(10/160): ceph-radosgw-19.1.0-1150.gba82afec.el 7.9 MB/s | 11 MB 00:01 2024-07-29T00:23:16.872 INFO:teuthology.orchestra.run.smithi175.stdout:(11/160): cephfs-mirror-19.1.0-1150.gba82afec.e 2.7 MB/s | 225 kB 00:00 2024-07-29T00:23:16.947 INFO:teuthology.orchestra.run.smithi175.stdout:(12/160): cephfs-shell-19.1.0-1150.gba82afec.el 364 kB/s | 27 kB 00:00 2024-07-29T00:23:17.022 INFO:teuthology.orchestra.run.smithi175.stdout:(13/160): libcephfs-devel-19.1.0-1150.gba82afec 427 kB/s | 32 kB 00:00 2024-07-29T00:23:17.173 INFO:teuthology.orchestra.run.smithi175.stdout:(14/160): libcephfs2-19.1.0-1150.gba82afec.el9. 4.9 MB/s | 753 kB 00:00 2024-07-29T00:23:17.352 INFO:teuthology.orchestra.run.smithi181.stdout:(1/160): ceph-19.1.0-1150.gba82afec.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-07-29T00:23:17.381 INFO:teuthology.orchestra.run.smithi175.stdout:(15/160): ceph-osd-19.1.0-1150.gba82afec.el9.x8 6.6 MB/s | 17 MB 00:02 2024-07-29T00:23:17.415 INFO:teuthology.orchestra.run.smithi175.stdout:(16/160): libcephsqlite-19.1.0-1150.gba82afec.e 678 kB/s | 163 kB 00:00 2024-07-29T00:23:17.448 INFO:teuthology.orchestra.run.smithi175.stdout:(17/160): librados-devel-19.1.0-1150.gba82afec. 1.9 MB/s | 127 kB 00:00 2024-07-29T00:23:17.515 INFO:teuthology.orchestra.run.smithi175.stdout:(18/160): libradosstriper1-19.1.0-1150.gba82afe 5.0 MB/s | 507 kB 00:00 2024-07-29T00:23:17.599 INFO:teuthology.orchestra.run.smithi175.stdout:(19/160): python3-ceph-argparse-19.1.0-1150.gba 548 kB/s | 45 kB 00:00 2024-07-29T00:23:17.708 INFO:teuthology.orchestra.run.smithi175.stdout:(20/160): python3-ceph-common-19.1.0-1150.gba82 1.2 MB/s | 136 kB 00:00 2024-07-29T00:23:17.800 INFO:teuthology.orchestra.run.smithi175.stdout:(21/160): python3-cephfs-19.1.0-1150.gba82afec. 2.0 MB/s | 188 kB 00:00 2024-07-29T00:23:17.900 INFO:teuthology.orchestra.run.smithi175.stdout:(22/160): python3-rados-19.1.0-1150.gba82afec.e 3.7 MB/s | 371 kB 00:00 2024-07-29T00:23:18.000 INFO:teuthology.orchestra.run.smithi175.stdout:(23/160): python3-rbd-19.1.0-1150.gba82afec.el9 3.4 MB/s | 349 kB 00:00 2024-07-29T00:23:18.092 INFO:teuthology.orchestra.run.smithi175.stdout:(24/160): python3-rgw-19.1.0-1150.gba82afec.el9 1.2 MB/s | 116 kB 00:00 2024-07-29T00:23:18.160 INFO:teuthology.orchestra.run.smithi181.stdout:(2/160): ceph-fuse-19.1.0-1150.gba82afec.el9.x8 1.1 MB/s | 896 kB 00:00 2024-07-29T00:23:18.184 INFO:teuthology.orchestra.run.smithi175.stdout:(25/160): rbd-fuse-19.1.0-1150.gba82afec.el9.x8 939 kB/s | 85 kB 00:00 2024-07-29T00:23:18.310 INFO:teuthology.orchestra.run.smithi181.stdout:(3/160): ceph-immutable-object-cache-19.1.0-115 967 kB/s | 145 kB 00:00 2024-07-29T00:23:18.326 INFO:teuthology.orchestra.run.smithi175.stdout:(26/160): librgw2-19.1.0-1150.gba82afec.el9.x86 6.1 MB/s | 5.4 MB 00:00 2024-07-29T00:23:18.426 INFO:teuthology.orchestra.run.smithi175.stdout:(27/160): rbd-nbd-19.1.0-1150.gba82afec.el9.x86 1.7 MB/s | 170 kB 00:00 2024-07-29T00:23:18.518 INFO:teuthology.orchestra.run.smithi175.stdout:(28/160): ceph-grafana-dashboards-19.1.0-1150.g 351 kB/s | 32 kB 00:00 2024-07-29T00:23:18.618 INFO:teuthology.orchestra.run.smithi175.stdout:(29/160): ceph-mgr-cephadm-19.1.0-1150.gba82afe 1.4 MB/s | 146 kB 00:00 2024-07-29T00:23:18.710 INFO:teuthology.orchestra.run.smithi175.stdout:(30/160): rbd-mirror-19.1.0-1150.gba82afec.el9. 5.9 MB/s | 3.1 MB 00:00 2024-07-29T00:23:18.811 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 18 kB 00:00 2024-07-29T00:23:19.074 INFO:teuthology.orchestra.run.smithi181.stdout:(4/160): ceph-base-19.1.0-1150.gba82afec.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-07-29T00:23:19.252 INFO:teuthology.orchestra.run.smithi175.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1150.gba82a 5.6 MB/s | 3.6 MB 00:00 2024-07-29T00:23:19.352 INFO:teuthology.orchestra.run.smithi175.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1150.gba 2.5 MB/s | 249 kB 00:00 2024-07-29T00:23:19.353 INFO:teuthology.orchestra.run.smithi181.stdout:(5/160): ceph-mgr-19.1.0-1150.gba82afec.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-07-29T00:23:19.453 INFO:teuthology.orchestra.run.smithi175.stdout:(33/160): ceph-mgr-rook-19.1.0-1150.gba82afec.e 498 kB/s | 49 kB 00:00 2024-07-29T00:23:19.478 INFO:teuthology.orchestra.run.smithi181.stdout:(6/160): ceph-mds-19.1.0-1150.gba82afec.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-07-29T00:23:19.545 INFO:teuthology.orchestra.run.smithi175.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1150.gb 181 kB/s | 16 kB 00:00 2024-07-29T00:23:19.653 INFO:teuthology.orchestra.run.smithi175.stdout:(35/160): ceph-volume-19.1.0-1150.gba82afec.el9 2.5 MB/s | 274 kB 00:00 2024-07-29T00:23:19.756 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-07-29T00:23:19.804 INFO:teuthology.orchestra.run.smithi175.stdout:(36/160): ceph-mgr-diskprediction-local-19.1.0- 6.8 MB/s | 7.4 MB 00:01 2024-07-29T00:23:19.870 INFO:teuthology.orchestra.run.smithi175.stdout:(37/160): cephadm-19.1.0-1150.gba82afec.el9.noa 3.5 MB/s | 765 kB 00:00 2024-07-29T00:23:19.904 INFO:teuthology.orchestra.run.smithi175.stdout:(38/160): cephfs-top-19.1.0-1150.gba82afec.el9. 183 kB/s | 18 kB 00:00 2024-07-29T00:23:19.978 INFO:teuthology.orchestra.run.smithi181.stdout:(7/160): ceph-mon-19.1.0-1150.gba82afec.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-07-29T00:23:20.330 INFO:teuthology.orchestra.run.smithi175.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-07-29T00:23:20.605 INFO:teuthology.orchestra.run.smithi175.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 712 kB/s | 195 kB 00:00 2024-07-29T00:23:20.664 INFO:teuthology.orchestra.run.smithi175.stdout:(41/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-07-29T00:23:20.805 INFO:teuthology.orchestra.run.smithi175.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-07-29T00:23:20.914 INFO:teuthology.orchestra.run.smithi175.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 986 kB/s | 106 kB 00:00 2024-07-29T00:23:20.964 INFO:teuthology.orchestra.run.smithi175.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-07-29T00:23:20.979 INFO:teuthology.orchestra.run.smithi181.stdout:(8/160): ceph-radosgw-19.1.0-1150.gba82afec.el9 11 MB/s | 11 MB 00:00 2024-07-29T00:23:21.023 INFO:teuthology.orchestra.run.smithi175.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-07-29T00:23:21.045 INFO:teuthology.orchestra.run.smithi181.stdout:(9/160): ceph-selinux-19.1.0-1150.gba82afec.el9 379 kB/s | 25 kB 00:00 2024-07-29T00:23:21.263 INFO:teuthology.orchestra.run.smithi181.stdout:(10/160): ceph-common-19.1.0-1150.gba82afec.el9 5.1 MB/s | 21 MB 00:04 2024-07-29T00:23:21.329 INFO:teuthology.orchestra.run.smithi181.stdout:(11/160): cephfs-mirror-19.1.0-1150.gba82afec.e 3.3 MB/s | 225 kB 00:00 2024-07-29T00:23:21.388 INFO:teuthology.orchestra.run.smithi181.stdout:(12/160): cephfs-shell-19.1.0-1150.gba82afec.el 464 kB/s | 27 kB 00:00 2024-07-29T00:23:21.447 INFO:teuthology.orchestra.run.smithi181.stdout:(13/160): libcephfs-devel-19.1.0-1150.gba82afec 542 kB/s | 32 kB 00:00 2024-07-29T00:23:21.523 INFO:teuthology.orchestra.run.smithi175.stdout:(46/160): ceph-test-19.1.0-1150.gba82afec.el9.x 9.4 MB/s | 49 MB 00:05 2024-07-29T00:23:21.557 INFO:teuthology.orchestra.run.smithi175.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 211 kB/s | 125 kB 00:00 2024-07-29T00:23:21.580 INFO:teuthology.orchestra.run.smithi181.stdout:(14/160): libcephfs2-19.1.0-1150.gba82afec.el9. 5.5 MB/s | 753 kB 00:00 2024-07-29T00:23:21.590 INFO:teuthology.orchestra.run.smithi175.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 379 kB/s | 215 kB 00:00 2024-07-29T00:23:21.656 INFO:teuthology.orchestra.run.smithi181.stdout:(15/160): libcephsqlite-19.1.0-1150.gba82afec.e 2.1 MB/s | 163 kB 00:00 2024-07-29T00:23:21.722 INFO:teuthology.orchestra.run.smithi181.stdout:(16/160): librados-devel-19.1.0-1150.gba82afec. 1.9 MB/s | 127 kB 00:00 2024-07-29T00:23:21.774 INFO:teuthology.orchestra.run.smithi175.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-07-29T00:23:21.799 INFO:teuthology.orchestra.run.smithi175.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 516 kB/s | 107 kB 00:00 2024-07-29T00:23:21.806 INFO:teuthology.orchestra.run.smithi181.stdout:(17/160): libradosstriper1-19.1.0-1150.gba82afe 6.0 MB/s | 507 kB 00:00 2024-07-29T00:23:21.833 INFO:teuthology.orchestra.run.smithi175.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 375 kB/s | 22 kB 00:00 2024-07-29T00:23:21.883 INFO:teuthology.orchestra.run.smithi175.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-07-29T00:23:21.909 INFO:teuthology.orchestra.run.smithi175.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 439 kB/s | 33 kB 00:00 2024-07-29T00:23:21.942 INFO:teuthology.orchestra.run.smithi175.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-07-29T00:23:21.976 INFO:teuthology.orchestra.run.smithi175.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 274 kB/s | 18 kB 00:00 2024-07-29T00:23:22.001 INFO:teuthology.orchestra.run.smithi175.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-07-29T00:23:22.035 INFO:teuthology.orchestra.run.smithi175.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-07-29T00:23:22.060 INFO:teuthology.orchestra.run.smithi175.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-07-29T00:23:22.118 INFO:teuthology.orchestra.run.smithi175.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-07-29T00:23:22.211 INFO:teuthology.orchestra.run.smithi175.stdout:(60/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-07-29T00:23:22.236 INFO:teuthology.orchestra.run.smithi175.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 219 kB/s | 25 kB 00:00 2024-07-29T00:23:22.298 INFO:teuthology.orchestra.run.smithi181.stdout:(18/160): ceph-osd-19.1.0-1150.gba82afec.el9.x8 6.0 MB/s | 17 MB 00:02 2024-07-29T00:23:22.301 INFO:teuthology.orchestra.run.smithi175.stdout:(62/160): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-07-29T00:23:22.336 INFO:teuthology.orchestra.run.smithi175.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 380 kB/s | 47 kB 00:00 2024-07-29T00:23:22.370 INFO:teuthology.orchestra.run.smithi175.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-07-29T00:23:22.374 INFO:teuthology.orchestra.run.smithi181.stdout:(19/160): librgw2-19.1.0-1150.gba82afec.el9.x86 9.4 MB/s | 5.4 MB 00:00 2024-07-29T00:23:22.395 INFO:teuthology.orchestra.run.smithi175.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-07-29T00:23:22.398 INFO:teuthology.orchestra.run.smithi181.stdout:(20/160): python3-ceph-argparse-19.1.0-1150.gba 450 kB/s | 45 kB 00:00 2024-07-29T00:23:22.420 INFO:teuthology.orchestra.run.smithi175.stdout:(66/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-07-29T00:23:22.424 INFO:teuthology.orchestra.run.smithi181.stdout:(21/160): python3-ceph-common-19.1.0-1150.gba82 2.6 MB/s | 136 kB 00:00 2024-07-29T00:23:22.449 INFO:teuthology.orchestra.run.smithi181.stdout:(22/160): python3-cephfs-19.1.0-1150.gba82afec. 3.7 MB/s | 188 kB 00:00 2024-07-29T00:23:22.491 INFO:teuthology.orchestra.run.smithi181.stdout:(23/160): python3-rados-19.1.0-1150.gba82afec.e 5.4 MB/s | 371 kB 00:00 2024-07-29T00:23:22.516 INFO:teuthology.orchestra.run.smithi181.stdout:(24/160): python3-rbd-19.1.0-1150.gba82afec.el9 5.1 MB/s | 349 kB 00:00 2024-07-29T00:23:22.529 INFO:teuthology.orchestra.run.smithi175.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-07-29T00:23:22.541 INFO:teuthology.orchestra.run.smithi181.stdout:(25/160): python3-rgw-19.1.0-1150.gba82afec.el9 2.3 MB/s | 116 kB 00:00 2024-07-29T00:23:22.567 INFO:teuthology.orchestra.run.smithi181.stdout:(26/160): rbd-fuse-19.1.0-1150.gba82afec.el9.x8 1.7 MB/s | 85 kB 00:00 2024-07-29T00:23:22.625 INFO:teuthology.orchestra.run.smithi181.stdout:(27/160): rbd-nbd-19.1.0-1150.gba82afec.el9.x86 2.8 MB/s | 170 kB 00:00 2024-07-29T00:23:22.638 INFO:teuthology.orchestra.run.smithi175.stdout:(68/160): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-07-29T00:23:22.709 INFO:teuthology.orchestra.run.smithi181.stdout:(28/160): rbd-mirror-19.1.0-1150.gba82afec.el9. 19 MB/s | 3.1 MB 00:00 2024-07-29T00:23:22.726 INFO:teuthology.orchestra.run.smithi181.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1150.g 318 kB/s | 32 kB 00:00 2024-07-29T00:23:22.738 INFO:teuthology.orchestra.run.smithi175.stdout:(69/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-07-29T00:23:22.776 INFO:teuthology.orchestra.run.smithi181.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1150.gba82afe 2.9 MB/s | 146 kB 00:00 2024-07-29T00:23:22.780 INFO:teuthology.orchestra.run.smithi175.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 944 kB/s | 133 kB 00:00 2024-07-29T00:23:22.839 INFO:teuthology.orchestra.run.smithi175.stdout:(71/160): postgresql-contrib-13.14-1.el9.x86_64 2.9 MB/s | 911 kB 00:00 2024-07-29T00:23:22.906 INFO:teuthology.orchestra.run.smithi175.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-07-29T00:23:22.931 INFO:teuthology.orchestra.run.smithi175.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 959 kB/s | 87 kB 00:00 2024-07-29T00:23:22.935 INFO:teuthology.orchestra.run.smithi181.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1150.gba82a 17 MB/s | 3.6 MB 00:00 2024-07-29T00:23:22.985 INFO:teuthology.orchestra.run.smithi181.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1150.gba 4.9 MB/s | 249 kB 00:00 2024-07-29T00:23:22.990 INFO:teuthology.orchestra.run.smithi175.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-07-29T00:23:23.044 INFO:teuthology.orchestra.run.smithi181.stdout:(33/160): ceph-mgr-rook-19.1.0-1150.gba82afec.e 845 kB/s | 49 kB 00:00 2024-07-29T00:23:23.056 INFO:teuthology.orchestra.run.smithi175.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-07-29T00:23:23.105 INFO:teuthology.orchestra.run.smithi181.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1150.gb 282 kB/s | 16 kB 00:00 2024-07-29T00:23:23.165 INFO:teuthology.orchestra.run.smithi175.stdout:(76/160): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-07-29T00:23:23.169 INFO:teuthology.orchestra.run.smithi181.stdout:(35/160): ceph-volume-19.1.0-1150.gba82afec.el9 4.0 MB/s | 274 kB 00:00 2024-07-29T00:23:23.199 INFO:teuthology.orchestra.run.smithi175.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-07-29T00:23:23.224 INFO:teuthology.orchestra.run.smithi175.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 444 kB/s | 26 kB 00:00 2024-07-29T00:23:23.253 INFO:teuthology.orchestra.run.smithi181.stdout:(36/160): cephadm-19.1.0-1150.gba82afec.el9.noa 9.0 MB/s | 765 kB 00:00 2024-07-29T00:23:23.258 INFO:teuthology.orchestra.run.smithi175.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 331 kB/s | 19 kB 00:00 2024-07-29T00:23:23.291 INFO:teuthology.orchestra.run.smithi175.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-07-29T00:23:23.312 INFO:teuthology.orchestra.run.smithi181.stdout:(37/160): cephfs-top-19.1.0-1150.gba82afec.el9. 313 kB/s | 18 kB 00:00 2024-07-29T00:23:23.316 INFO:teuthology.orchestra.run.smithi175.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 601 kB/s | 35 kB 00:00 2024-07-29T00:23:23.454 INFO:teuthology.orchestra.run.smithi181.stdout:(38/160): ceph-mgr-diskprediction-local-19.1.0- 11 MB/s | 7.4 MB 00:00 2024-07-29T00:23:23.464 INFO:teuthology.orchestra.run.smithi175.stdout:(82/160): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-07-29T00:23:23.500 INFO:teuthology.orchestra.run.smithi175.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-07-29T00:23:23.504 INFO:teuthology.orchestra.run.smithi181.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 415 kB/s | 80 kB 00:00 2024-07-29T00:23:23.534 INFO:teuthology.orchestra.run.smithi175.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-07-29T00:23:23.559 INFO:teuthology.orchestra.run.smithi175.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-07-29T00:23:23.593 INFO:teuthology.orchestra.run.smithi175.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 727 kB/s | 42 kB 00:00 2024-07-29T00:23:23.627 INFO:teuthology.orchestra.run.smithi175.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-07-29T00:23:23.660 INFO:teuthology.orchestra.run.smithi175.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-07-29T00:23:23.694 INFO:teuthology.orchestra.run.smithi175.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-07-29T00:23:23.719 INFO:teuthology.orchestra.run.smithi175.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 882 kB/s | 51 kB 00:00 2024-07-29T00:23:23.753 INFO:teuthology.orchestra.run.smithi175.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 929 kB/s | 54 kB 00:00 2024-07-29T00:23:23.811 INFO:teuthology.orchestra.run.smithi175.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 724 kB/s | 42 kB 00:00 2024-07-29T00:23:23.920 INFO:teuthology.orchestra.run.smithi175.stdout:(93/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.1 MB/s | 5.1 MB 00:00 2024-07-29T00:23:23.945 INFO:teuthology.orchestra.run.smithi175.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 332 kB/s | 44 kB 00:00 2024-07-29T00:23:23.954 INFO:teuthology.orchestra.run.smithi181.stdout:(40/160): ceph-test-19.1.0-1150.gba82afec.el9.x 17 MB/s | 49 MB 00:02 2024-07-29T00:23:23.979 INFO:teuthology.orchestra.run.smithi175.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-07-29T00:23:23.979 INFO:teuthology.orchestra.run.smithi181.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 411 kB/s | 195 kB 00:00 2024-07-29T00:23:24.004 INFO:teuthology.orchestra.run.smithi175.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 620 kB/s | 36 kB 00:00 2024-07-29T00:23:24.063 INFO:teuthology.orchestra.run.smithi175.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 574 kB/s | 47 kB 00:00 2024-07-29T00:23:24.105 INFO:teuthology.orchestra.run.smithi175.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-07-29T00:23:24.130 INFO:teuthology.orchestra.run.smithi181.stdout:(42/160): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-07-29T00:23:24.130 INFO:teuthology.orchestra.run.smithi175.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-07-29T00:23:24.155 INFO:teuthology.orchestra.run.smithi181.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-07-29T00:23:24.164 INFO:teuthology.orchestra.run.smithi175.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-07-29T00:23:24.180 INFO:teuthology.orchestra.run.smithi181.stdout:(44/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-07-29T00:23:24.189 INFO:teuthology.orchestra.run.smithi175.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-07-29T00:23:24.205 INFO:teuthology.orchestra.run.smithi181.stdout:(45/160): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-07-29T00:23:24.222 INFO:teuthology.orchestra.run.smithi181.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-07-29T00:23:24.223 INFO:teuthology.orchestra.run.smithi175.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-07-29T00:23:24.248 INFO:teuthology.orchestra.run.smithi181.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-07-29T00:23:24.273 INFO:teuthology.orchestra.run.smithi181.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-07-29T00:23:24.281 INFO:teuthology.orchestra.run.smithi175.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-07-29T00:23:24.407 INFO:teuthology.orchestra.run.smithi175.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-07-29T00:23:24.448 INFO:teuthology.orchestra.run.smithi175.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-07-29T00:23:24.482 INFO:teuthology.orchestra.run.smithi175.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 590 kB/s | 19 kB 00:00 2024-07-29T00:23:24.524 INFO:teuthology.orchestra.run.smithi175.stdout:(107/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-07-29T00:23:24.633 INFO:teuthology.orchestra.run.smithi175.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 242 kB/s | 26 kB 00:00 2024-07-29T00:23:24.658 INFO:teuthology.orchestra.run.smithi175.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-29T00:23:24.692 INFO:teuthology.orchestra.run.smithi175.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-07-29T00:23:24.725 INFO:teuthology.orchestra.run.smithi175.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-07-29T00:23:24.801 INFO:teuthology.orchestra.run.smithi175.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-07-29T00:23:24.859 INFO:teuthology.orchestra.run.smithi175.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-07-29T00:23:24.876 INFO:teuthology.orchestra.run.smithi175.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.7 MB/s | 43 kB 00:00 2024-07-29T00:23:24.893 INFO:teuthology.orchestra.run.smithi175.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 2.0 MB/s | 32 kB 00:00 2024-07-29T00:23:24.910 INFO:teuthology.orchestra.run.smithi175.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 889 kB/s | 14 kB 00:00 2024-07-29T00:23:24.952 INFO:teuthology.orchestra.run.smithi175.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-07-29T00:23:25.003 INFO:teuthology.orchestra.run.smithi175.stdout:(118/160): python3-cherrypy-18.6.1-2.el9.noarch 7.1 MB/s | 358 kB 00:00 2024-07-29T00:23:25.053 INFO:teuthology.orchestra.run.smithi175.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-07-29T00:23:25.065 INFO:teuthology.orchestra.run.smithi181.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 135 kB/s | 107 kB 00:00 2024-07-29T00:23:25.137 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-29T00:23:25.199 INFO:teuthology.orchestra.run.smithi181.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 706 kB/s | 689 kB 00:00 2024-07-29T00:23:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:(51/160): bison-3.7.4-5.el9.x86_64.rpm 927 kB/s | 943 kB 00:01 2024-07-29T00:23:25.278 INFO:teuthology.orchestra.run.smithi175.stdout:(120/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-07-29T00:23:25.378 INFO:teuthology.orchestra.run.smithi175.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 665 kB/s | 216 kB 00:00 2024-07-29T00:23:25.412 INFO:teuthology.orchestra.run.smithi175.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.9 MB/s | 144 kB 00:00 2024-07-29T00:23:25.437 INFO:teuthology.orchestra.run.smithi175.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 438 kB/s | 11 kB 00:00 2024-07-29T00:23:25.463 INFO:teuthology.orchestra.run.smithi175.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 729 kB/s | 18 kB 00:00 2024-07-29T00:23:25.488 INFO:teuthology.orchestra.run.smithi175.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 959 kB/s | 23 kB 00:00 2024-07-29T00:23:25.513 INFO:teuthology.orchestra.run.smithi175.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 795 kB/s | 19 kB 00:00 2024-07-29T00:23:25.539 INFO:teuthology.orchestra.run.smithi175.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 811 kB/s | 20 kB 00:00 2024-07-29T00:23:25.664 INFO:teuthology.orchestra.run.smithi175.stdout:(128/160): libarrow-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-07-29T00:23:25.674 INFO:teuthology.orchestra.run.smithi181.stdout:(52/160): elfutils-libelf-devel-0.191-4.el9.x86 36 kB/s | 22 kB 00:00 2024-07-29T00:23:25.698 INFO:teuthology.orchestra.run.smithi175.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-07-29T00:23:25.732 INFO:teuthology.orchestra.run.smithi175.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.4 MB/s | 79 kB 00:00 2024-07-29T00:23:25.782 INFO:teuthology.orchestra.run.smithi175.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-07-29T00:23:25.832 INFO:teuthology.orchestra.run.smithi175.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-07-29T00:23:25.874 INFO:teuthology.orchestra.run.smithi175.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 402 kB/s | 16 kB 00:00 2024-07-29T00:23:25.895 INFO:teuthology.orchestra.run.smithi181.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-07-29T00:23:25.908 INFO:teuthology.orchestra.run.smithi175.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-07-29T00:23:25.933 INFO:teuthology.orchestra.run.smithi175.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-29T00:23:25.958 INFO:teuthology.orchestra.run.smithi175.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-07-29T00:23:25.983 INFO:teuthology.orchestra.run.smithi175.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 622 kB/s | 31 kB 00:00 2024-07-29T00:23:26.017 INFO:teuthology.orchestra.run.smithi175.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-07-29T00:23:26.043 INFO:teuthology.orchestra.run.smithi175.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-07-29T00:23:26.076 INFO:teuthology.orchestra.run.smithi175.stdout:(140/160): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-07-29T00:23:26.083 INFO:teuthology.orchestra.run.smithi181.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 355 kB/s | 313 kB 00:00 2024-07-29T00:23:26.102 INFO:teuthology.orchestra.run.smithi175.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 881 kB/s | 51 kB 00:00 2024-07-29T00:23:26.126 INFO:teuthology.orchestra.run.smithi175.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-07-29T00:23:26.160 INFO:teuthology.orchestra.run.smithi175.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-07-29T00:23:26.185 INFO:teuthology.orchestra.run.smithi175.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 825 kB/s | 20 kB 00:00 2024-07-29T00:23:26.236 INFO:teuthology.orchestra.run.smithi175.stdout:(145/160): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-07-29T00:23:26.269 INFO:teuthology.orchestra.run.smithi175.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-07-29T00:23:26.303 INFO:teuthology.orchestra.run.smithi175.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-07-29T00:23:26.387 INFO:teuthology.orchestra.run.smithi175.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-07-29T00:23:26.500 INFO:teuthology.orchestra.run.smithi181.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 30 kB/s | 18 kB 00:00 2024-07-29T00:23:26.650 INFO:teuthology.orchestra.run.smithi181.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-07-29T00:23:26.679 INFO:teuthology.orchestra.run.smithi175.stdout:(149/160): librbd1-19.1.0-1150.gba82afec.el9.x8 8.4 MB/s | 3.2 MB 00:00 2024-07-29T00:23:26.762 INFO:teuthology.orchestra.run.smithi175.stdout:(150/160): librados2-19.1.0-1150.gba82afec.el9. 7.0 MB/s | 3.4 MB 00:00 2024-07-29T00:23:26.850 INFO:teuthology.orchestra.run.smithi181.stdout:(57/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-07-29T00:23:27.029 INFO:teuthology.orchestra.run.smithi175.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 153 kB/s | 41 kB 00:00 2024-07-29T00:23:27.351 INFO:teuthology.orchestra.run.smithi181.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 65 kB/s | 45 kB 00:00 2024-07-29T00:23:27.376 INFO:teuthology.orchestra.run.smithi181.stdout:(59/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 183 kB/s | 160 kB 00:00 2024-07-29T00:23:27.402 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-29T00:23:27.403 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-29T00:23:27.413 INFO:teuthology.orchestra.run.smithi175.stdout:(152/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-07-29T00:23:27.413 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-29T00:23:27.414 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-29T00:23:27.415 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-29T00:23:27.416 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-29T00:23:27.418 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-29T00:23:27.420 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-29T00:23:27.421 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-29T00:23:27.455 INFO:teuthology.orchestra.run.smithi175.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 582 kB/s | 247 kB 00:00 2024-07-29T00:23:27.483 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-07-29T00:23:27.501 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-07-29T00:23:27.501 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-07-29T00:23:27.501 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-07-29T00:23:27.501 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-07-29T00:23:27.501 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.1.0-1150.gba82afec.el9 ceph 6.6 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.1.0-1150.gba82afec.el9 ceph 5.4 M 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 896 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1150.gba82afec.el9 ceph 145 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.1.0-1150.gba82afec.el9 ceph 1.5 M 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 146 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 3.6 M 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 7.4 M 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 49 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.1.0-1150.gba82afec.el9 ceph 11 M 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.1.0-1150.gba82afec.el9 ceph 49 M 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 274 k 2024-07-29T00:23:27.502 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 765 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.1.0-1150.gba82afec.el9 ceph 225 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.1.0-1150.gba82afec.el9 ceph 27 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 18 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.1.0-1150.gba82afec.el9 ceph 32 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 753 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.1.0-1150.gba82afec.el9 ceph 127 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-29T00:23:27.503 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.1.0-1150.gba82afec.el9 ceph 188 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.1.0-1150.gba82afec.el9 ceph 371 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 349 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.1.0-1150.gba82afec.el9 ceph 116 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 85 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.1 M 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 170 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-07-29T00:23:27.504 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.4 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 3.2 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-07-29T00:23:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.1.0-1150.gba82afec.el9 ceph 21 M 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 32 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.1.0-1150.gba82afec.el9 ceph 2.4 M 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 249 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.1.0-1150.gba82afec.el9 ceph 4.7 M 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.1.0-1150.gba82afec.el9 ceph 17 M 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1150.gba82afec.el9 ceph-noarch 16 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.1.0-1150.gba82afec.el9 ceph 25 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-29T00:23:27.506 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.1.0-1150.gba82afec.el9 ceph 163 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 507 k 2024-07-29T00:23:27.507 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.1.0-1150.gba82afec.el9 ceph 5.4 M 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-29T00:23:27.508 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.1.0-1150.gba82afec.el9 ceph 45 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.1.0-1150.gba82afec.el9 ceph 136 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-29T00:23:27.509 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-29T00:23:27.510 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-29T00:23:27.511 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-29T00:23:27.512 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-29T00:23:27.513 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 12 Packages 2024-07-29T00:23:27.514 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:23:27.515 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 228 M 2024-07-29T00:23:27.515 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-07-29T00:23:27.522 INFO:teuthology.orchestra.run.smithi175.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 339 kB/s | 37 kB 00:00 2024-07-29T00:23:27.564 INFO:teuthology.orchestra.run.smithi175.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-07-29T00:23:27.639 INFO:teuthology.orchestra.run.smithi175.stdout:(156/160): NetworkManager-libnm-1.48.2-2.el9.x8 1.9 MB/s | 1.8 MB 00:00 2024-07-29T00:23:27.673 INFO:teuthology.orchestra.run.smithi175.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.7 MB/s | 260 kB 00:00 2024-07-29T00:23:27.806 INFO:teuthology.orchestra.run.smithi175.stdout:(158/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 8.2 MB/s | 1.1 MB 00:00 2024-07-29T00:23:27.884 INFO:teuthology.orchestra.run.smithi181.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 642 kB/s | 662 kB 00:01 2024-07-29T00:23:27.957 INFO:teuthology.orchestra.run.smithi175.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 7.6 MB/s | 2.4 MB 00:00 2024-07-29T00:23:27.985 INFO:teuthology.orchestra.run.smithi181.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-07-29T00:23:28.006 INFO:teuthology.orchestra.run.smithi175.stdout:(160/160): openssl-3.2.2-2.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-07-29T00:23:28.019 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-07-29T00:23:28.019 INFO:teuthology.orchestra.run.smithi175.stdout:Total 14 MB/s | 228 MB 00:16 2024-07-29T00:23:28.263 INFO:teuthology.orchestra.run.smithi204.stdout:(1/160): ceph-19.1.0-1150.gba82afec.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-07-29T00:23:28.346 INFO:teuthology.orchestra.run.smithi181.stdout:(62/160): libtool-2.4.6-46.el9.x86_64.rpm 585 kB/s | 581 kB 00:00 2024-07-29T00:23:28.455 INFO:teuthology.orchestra.run.smithi204.stdout:(2/160): ceph-fuse-19.1.0-1150.gba82afec.el9.x8 4.6 MB/s | 896 kB 00:00 2024-07-29T00:23:28.513 INFO:teuthology.orchestra.run.smithi204.stdout:(3/160): ceph-immutable-object-cache-19.1.0-115 2.4 MB/s | 145 kB 00:00 2024-07-29T00:23:28.605 INFO:teuthology.orchestra.run.smithi204.stdout:(4/160): ceph-base-19.1.0-1150.gba82afec.el9.x8 10 MB/s | 5.4 MB 00:00 2024-07-29T00:23:28.635 INFO:teuthology.orchestra.run.smithi181.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 63 kB/s | 47 kB 00:00 2024-07-29T00:23:28.730 INFO:teuthology.orchestra.run.smithi204.stdout:(5/160): ceph-mgr-19.1.0-1150.gba82afec.el9.x86 12 MB/s | 1.5 MB 00:00 2024-07-29T00:23:28.772 INFO:teuthology.orchestra.run.smithi204.stdout:(6/160): ceph-mds-19.1.0-1150.gba82afec.el9.x86 9.2 MB/s | 2.4 MB 00:00 2024-07-29T00:23:28.911 INFO:teuthology.orchestra.run.smithi181.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 316 kB/s | 292 kB 00:00 2024-07-29T00:23:29.006 INFO:teuthology.orchestra.run.smithi204.stdout:(7/160): ceph-mon-19.1.0-1150.gba82afec.el9.x86 17 MB/s | 4.7 MB 00:00 2024-07-29T00:23:29.228 INFO:teuthology.orchestra.run.smithi181.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 213 kB/s | 188 kB 00:00 2024-07-29T00:23:29.328 INFO:teuthology.orchestra.run.smithi181.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 51 kB/s | 35 kB 00:00 2024-07-29T00:23:29.615 INFO:teuthology.orchestra.run.smithi204.stdout:(8/160): ceph-common-19.1.0-1150.gba82afec.el9. 14 MB/s | 21 MB 00:01 2024-07-29T00:23:29.740 INFO:teuthology.orchestra.run.smithi204.stdout:(9/160): ceph-radosgw-19.1.0-1150.gba82afec.el9 14 MB/s | 11 MB 00:00 2024-07-29T00:23:29.765 INFO:teuthology.orchestra.run.smithi204.stdout:(10/160): ceph-selinux-19.1.0-1150.gba82afec.el 167 kB/s | 25 kB 00:00 2024-07-29T00:23:29.974 INFO:teuthology.orchestra.run.smithi204.stdout:(11/160): ceph-osd-19.1.0-1150.gba82afec.el9.x8 14 MB/s | 17 MB 00:01 2024-07-29T00:23:29.999 INFO:teuthology.orchestra.run.smithi204.stdout:(12/160): cephfs-mirror-19.1.0-1150.gba82afec.e 961 kB/s | 225 kB 00:00 2024-07-29T00:23:30.075 INFO:teuthology.orchestra.run.smithi204.stdout:(13/160): cephfs-shell-19.1.0-1150.gba82afec.el 270 kB/s | 27 kB 00:00 2024-07-29T00:23:30.100 INFO:teuthology.orchestra.run.smithi204.stdout:(14/160): libcephfs-devel-19.1.0-1150.gba82afec 316 kB/s | 32 kB 00:00 2024-07-29T00:23:30.133 INFO:teuthology.orchestra.run.smithi204.stdout:(15/160): libcephfs2-19.1.0-1150.gba82afec.el9. 13 MB/s | 753 kB 00:00 2024-07-29T00:23:30.159 INFO:teuthology.orchestra.run.smithi204.stdout:(16/160): libcephsqlite-19.1.0-1150.gba82afec.e 2.7 MB/s | 163 kB 00:00 2024-07-29T00:23:30.184 INFO:teuthology.orchestra.run.smithi204.stdout:(17/160): librados-devel-19.1.0-1150.gba82afec. 2.5 MB/s | 127 kB 00:00 2024-07-29T00:23:30.186 INFO:teuthology.orchestra.run.smithi181.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-07-29T00:23:30.209 INFO:teuthology.orchestra.run.smithi204.stdout:(18/160): libradosstriper1-19.1.0-1150.gba82afe 9.9 MB/s | 507 kB 00:00 2024-07-29T00:23:30.268 INFO:teuthology.orchestra.run.smithi204.stdout:(19/160): python3-ceph-argparse-19.1.0-1150.gba 772 kB/s | 45 kB 00:00 2024-07-29T00:23:30.352 INFO:teuthology.orchestra.run.smithi204.stdout:(20/160): python3-ceph-common-19.1.0-1150.gba82 1.6 MB/s | 136 kB 00:00 2024-07-29T00:23:30.435 INFO:teuthology.orchestra.run.smithi204.stdout:(21/160): librgw2-19.1.0-1150.gba82afec.el9.x86 21 MB/s | 5.4 MB 00:00 2024-07-29T00:23:30.445 INFO:teuthology.orchestra.run.smithi181.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-07-29T00:23:30.460 INFO:teuthology.orchestra.run.smithi204.stdout:(22/160): python3-cephfs-19.1.0-1150.gba82afec. 1.7 MB/s | 188 kB 00:00 2024-07-29T00:23:30.486 INFO:teuthology.orchestra.run.smithi204.stdout:(23/160): python3-rados-19.1.0-1150.gba82afec.e 7.2 MB/s | 371 kB 00:00 2024-07-29T00:23:30.511 INFO:teuthology.orchestra.run.smithi204.stdout:(24/160): python3-rbd-19.1.0-1150.gba82afec.el9 6.8 MB/s | 349 kB 00:00 2024-07-29T00:23:30.528 INFO:teuthology.orchestra.run.smithi204.stdout:(25/160): python3-rgw-19.1.0-1150.gba82afec.el9 2.7 MB/s | 116 kB 00:00 2024-07-29T00:23:30.545 INFO:teuthology.orchestra.run.smithi181.stdout:(69/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-07-29T00:23:30.545 INFO:teuthology.orchestra.run.smithi204.stdout:(26/160): rbd-fuse-19.1.0-1150.gba82afec.el9.x8 2.5 MB/s | 85 kB 00:00 2024-07-29T00:23:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:(27/160): rbd-nbd-19.1.0-1150.gba82afec.el9.x86 2.5 MB/s | 170 kB 00:00 2024-07-29T00:23:30.688 INFO:teuthology.orchestra.run.smithi204.stdout:(28/160): rbd-mirror-19.1.0-1150.gba82afec.el9. 19 MB/s | 3.1 MB 00:00 2024-07-29T00:23:30.713 INFO:teuthology.orchestra.run.smithi204.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1150.g 317 kB/s | 32 kB 00:00 2024-07-29T00:23:30.738 INFO:teuthology.orchestra.run.smithi204.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1150.gba82afe 2.9 MB/s | 146 kB 00:00 2024-07-29T00:23:31.014 INFO:teuthology.orchestra.run.smithi204.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1150.gba82a 12 MB/s | 3.6 MB 00:00 2024-07-29T00:23:31.114 INFO:teuthology.orchestra.run.smithi204.stdout:(32/160): ceph-mgr-diskprediction-local-19.1.0- 20 MB/s | 7.4 MB 00:00 2024-07-29T00:23:31.139 INFO:teuthology.orchestra.run.smithi204.stdout:(33/160): ceph-mgr-modules-core-19.1.0-1150.gba 1.9 MB/s | 249 kB 00:00 2024-07-29T00:23:31.164 INFO:teuthology.orchestra.run.smithi204.stdout:(34/160): ceph-mgr-rook-19.1.0-1150.gba82afec.e 984 kB/s | 49 kB 00:00 2024-07-29T00:23:31.181 INFO:teuthology.orchestra.run.smithi204.stdout:(35/160): ceph-prometheus-alerts-19.1.0-1150.gb 391 kB/s | 16 kB 00:00 2024-07-29T00:23:31.215 INFO:teuthology.orchestra.run.smithi204.stdout:(36/160): ceph-volume-19.1.0-1150.gba82afec.el9 5.3 MB/s | 274 kB 00:00 2024-07-29T00:23:31.254 INFO:teuthology.orchestra.run.smithi181.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 854 kB/s | 911 kB 00:01 2024-07-29T00:23:31.265 INFO:teuthology.orchestra.run.smithi204.stdout:(37/160): cephadm-19.1.0-1150.gba82afec.el9.noa 9.0 MB/s | 765 kB 00:00 2024-07-29T00:23:31.282 INFO:teuthology.orchestra.run.smithi204.stdout:(38/160): cephfs-top-19.1.0-1150.gba82afec.el9. 272 kB/s | 18 kB 00:00 2024-07-29T00:23:31.310 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-07-29T00:23:31.321 INFO:teuthology.orchestra.run.smithi181.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 157 kB/s | 133 kB 00:00 2024-07-29T00:23:31.416 INFO:teuthology.orchestra.run.smithi204.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-07-29T00:23:31.508 INFO:teuthology.orchestra.run.smithi204.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-07-29T00:23:31.533 INFO:teuthology.orchestra.run.smithi204.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-07-29T00:23:31.575 INFO:teuthology.orchestra.run.smithi204.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-07-29T00:23:31.597 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-07-29T00:23:31.597 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-07-29T00:23:31.846 INFO:teuthology.orchestra.run.smithi181.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-07-29T00:23:32.026 INFO:teuthology.orchestra.run.smithi204.stdout:(43/160): ceph-test-19.1.0-1150.gba82afec.el9.x 21 MB/s | 49 MB 00:02 2024-07-29T00:23:32.051 INFO:teuthology.orchestra.run.smithi204.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 224 kB/s | 106 kB 00:00 2024-07-29T00:23:32.085 INFO:teuthology.orchestra.run.smithi204.stdout:(45/160): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-07-29T00:23:32.110 INFO:teuthology.orchestra.run.smithi204.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-07-29T00:23:32.112 INFO:teuthology.orchestra.run.smithi181.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 110 kB/s | 87 kB 00:00 2024-07-29T00:23:32.143 INFO:teuthology.orchestra.run.smithi204.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-07-29T00:23:32.178 INFO:teuthology.orchestra.run.smithi204.stdout:(48/160): python3-pycparser-2.20-6.el9.noarch.r 896 kB/s | 135 kB 00:00 2024-07-29T00:23:32.346 INFO:teuthology.orchestra.run.smithi181.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 946 kB/s | 1.0 MB 00:01 2024-07-29T00:23:32.378 INFO:teuthology.orchestra.run.smithi204.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-07-29T00:23:32.403 INFO:teuthology.orchestra.run.smithi204.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 475 kB/s | 107 kB 00:00 2024-07-29T00:23:32.437 INFO:teuthology.orchestra.run.smithi204.stdout:(51/160): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-07-29T00:23:32.462 INFO:teuthology.orchestra.run.smithi204.stdout:(52/160): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-07-29T00:23:32.487 INFO:teuthology.orchestra.run.smithi204.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-07-29T00:23:32.512 INFO:teuthology.orchestra.run.smithi204.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-07-29T00:23:32.538 INFO:teuthology.orchestra.run.smithi204.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-07-29T00:23:32.563 INFO:teuthology.orchestra.run.smithi204.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-07-29T00:23:32.596 INFO:teuthology.orchestra.run.smithi204.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-07-29T00:23:32.622 INFO:teuthology.orchestra.run.smithi204.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-07-29T00:23:32.655 INFO:teuthology.orchestra.run.smithi204.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-07-29T00:23:32.706 INFO:teuthology.orchestra.run.smithi204.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-07-29T00:23:32.731 INFO:teuthology.orchestra.run.smithi204.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-07-29T00:23:32.765 INFO:teuthology.orchestra.run.smithi204.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-07-29T00:23:32.790 INFO:teuthology.orchestra.run.smithi204.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-07-29T00:23:32.824 INFO:teuthology.orchestra.run.smithi204.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-07-29T00:23:32.846 INFO:teuthology.orchestra.run.smithi181.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 87 kB/s | 61 kB 00:00 2024-07-29T00:23:32.849 INFO:teuthology.orchestra.run.smithi204.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 596 kB/s | 35 kB 00:00 2024-07-29T00:23:32.900 INFO:teuthology.orchestra.run.smithi204.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.8 MB/s | 3.0 MB 00:00 2024-07-29T00:23:33.050 INFO:teuthology.orchestra.run.smithi204.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-07-29T00:23:33.117 INFO:teuthology.orchestra.run.smithi204.stdout:(68/160): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-07-29T00:23:33.175 INFO:teuthology.orchestra.run.smithi204.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-07-29T00:23:33.201 INFO:teuthology.orchestra.run.smithi204.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-07-29T00:23:33.205 INFO:teuthology.orchestra.run.smithi181.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-07-29T00:23:33.230 INFO:teuthology.orchestra.run.smithi181.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 276 kB/s | 244 kB 00:00 2024-07-29T00:23:33.243 INFO:teuthology.orchestra.run.smithi204.stdout:(71/160): postgresql-13.14-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-07-29T00:23:33.276 INFO:teuthology.orchestra.run.smithi204.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-07-29T00:23:33.302 INFO:teuthology.orchestra.run.smithi204.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-07-29T00:23:33.369 INFO:teuthology.orchestra.run.smithi204.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 921 kB/s | 61 kB 00:00 2024-07-29T00:23:33.444 INFO:teuthology.orchestra.run.smithi204.stdout:(75/160): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-07-29T00:23:33.469 INFO:teuthology.orchestra.run.smithi204.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-07-29T00:23:33.544 INFO:teuthology.orchestra.run.smithi204.stdout:(77/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-07-29T00:23:33.569 INFO:teuthology.orchestra.run.smithi204.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-07-29T00:23:33.595 INFO:teuthology.orchestra.run.smithi204.stdout:(79/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-07-29T00:23:33.620 INFO:teuthology.orchestra.run.smithi204.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 253 kB/s | 19 kB 00:00 2024-07-29T00:23:33.645 INFO:teuthology.orchestra.run.smithi204.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-07-29T00:23:33.671 INFO:teuthology.orchestra.run.smithi204.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-07-29T00:23:33.713 INFO:teuthology.orchestra.run.smithi204.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-07-29T00:23:33.738 INFO:teuthology.orchestra.run.smithi204.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-07-29T00:23:33.747 INFO:teuthology.orchestra.run.smithi181.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 276 kB/s | 248 kB 00:00 2024-07-29T00:23:33.797 INFO:teuthology.orchestra.run.smithi181.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-07-29T00:23:33.813 INFO:teuthology.orchestra.run.smithi204.stdout:(85/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-07-29T00:23:33.831 INFO:teuthology.orchestra.run.smithi181.stdout:(80/160): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 26 kB 00:00 2024-07-29T00:23:33.839 INFO:teuthology.orchestra.run.smithi204.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-07-29T00:23:33.864 INFO:teuthology.orchestra.run.smithi204.stdout:(87/160): python3-packaging-20.9-5.el9.noarch.r 512 kB/s | 77 kB 00:00 2024-07-29T00:23:33.889 INFO:teuthology.orchestra.run.smithi204.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-07-29T00:23:33.914 INFO:teuthology.orchestra.run.smithi204.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-07-29T00:23:33.940 INFO:teuthology.orchestra.run.smithi204.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-07-29T00:23:33.965 INFO:teuthology.orchestra.run.smithi204.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-07-29T00:23:33.990 INFO:teuthology.orchestra.run.smithi204.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-07-29T00:23:34.024 INFO:teuthology.orchestra.run.smithi204.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 707 kB/s | 42 kB 00:00 2024-07-29T00:23:34.050 INFO:teuthology.orchestra.run.smithi204.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 751 kB/s | 44 kB 00:00 2024-07-29T00:23:34.083 INFO:teuthology.orchestra.run.smithi204.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-07-29T00:23:34.109 INFO:teuthology.orchestra.run.smithi204.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-07-29T00:23:34.142 INFO:teuthology.orchestra.run.smithi204.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-07-29T00:23:34.167 INFO:teuthology.orchestra.run.smithi204.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-07-29T00:23:34.201 INFO:teuthology.orchestra.run.smithi204.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-07-29T00:23:34.226 INFO:teuthology.orchestra.run.smithi204.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-07-29T00:23:34.260 INFO:teuthology.orchestra.run.smithi204.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-07-29T00:23:34.265 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-07-29T00:23:34.265 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-07-29T00:23:34.285 INFO:teuthology.orchestra.run.smithi204.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-07-29T00:23:34.411 INFO:teuthology.orchestra.run.smithi204.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-07-29T00:23:34.440 INFO:teuthology.orchestra.run.smithi181.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 54 kB/s | 35 kB 00:00 2024-07-29T00:23:34.478 INFO:teuthology.orchestra.run.smithi204.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-07-29T00:23:34.503 INFO:teuthology.orchestra.run.smithi204.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-07-29T00:23:34.520 INFO:teuthology.orchestra.run.smithi204.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-07-29T00:23:34.590 INFO:teuthology.orchestra.run.smithi181.stdout:(82/160): python3-mako-1.1.4-6.el9.noarch.rpm 204 kB/s | 172 kB 00:00 2024-07-29T00:23:34.628 INFO:teuthology.orchestra.run.smithi204.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-07-29T00:23:34.654 INFO:teuthology.orchestra.run.smithi204.stdout:(108/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-07-29T00:23:34.671 INFO:teuthology.orchestra.run.smithi204.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 621 kB/s | 26 kB 00:00 2024-07-29T00:23:34.721 INFO:teuthology.orchestra.run.smithi204.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-29T00:23:34.754 INFO:teuthology.orchestra.run.smithi204.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.8 MB/s | 151 kB 00:00 2024-07-29T00:23:34.780 INFO:teuthology.orchestra.run.smithi204.stdout:(112/160): liboath-2.6.7-2.el9.x86_64.rpm 386 kB/s | 49 kB 00:00 2024-07-29T00:23:34.822 INFO:teuthology.orchestra.run.smithi204.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-07-29T00:23:34.847 INFO:teuthology.orchestra.run.smithi204.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-29T00:23:34.873 INFO:teuthology.orchestra.run.smithi204.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-29T00:23:34.898 INFO:teuthology.orchestra.run.smithi204.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-07-29T00:23:34.923 INFO:teuthology.orchestra.run.smithi204.stdout:(117/160): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-07-29T00:23:34.949 INFO:teuthology.orchestra.run.smithi204.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-07-29T00:23:34.982 INFO:teuthology.orchestra.run.smithi204.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-07-29T00:23:35.010 INFO:teuthology.orchestra.run.smithi204.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 4.1 MB/s | 358 kB 00:00 2024-07-29T00:23:35.035 INFO:teuthology.orchestra.run.smithi204.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.0 MB/s | 216 kB 00:00 2024-07-29T00:23:35.060 INFO:teuthology.orchestra.run.smithi204.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-07-29T00:23:35.086 INFO:teuthology.orchestra.run.smithi204.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-07-29T00:23:35.111 INFO:teuthology.orchestra.run.smithi204.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-07-29T00:23:35.123 INFO:teuthology.orchestra.run.smithi181.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-07-29T00:23:35.153 INFO:teuthology.orchestra.run.smithi204.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-07-29T00:23:35.178 INFO:teuthology.orchestra.run.smithi204.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 773 kB/s | 19 kB 00:00 2024-07-29T00:23:35.204 INFO:teuthology.orchestra.run.smithi204.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-07-29T00:23:35.254 INFO:teuthology.orchestra.run.smithi204.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-07-29T00:23:35.279 INFO:teuthology.orchestra.run.smithi204.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-29T00:23:35.298 INFO:teuthology.orchestra.run.smithi181.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 292 kB/s | 250 kB 00:00 2024-07-29T00:23:35.304 INFO:teuthology.orchestra.run.smithi204.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-29T00:23:35.330 INFO:teuthology.orchestra.run.smithi204.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-29T00:23:35.363 INFO:teuthology.orchestra.run.smithi204.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-07-29T00:23:35.380 INFO:teuthology.orchestra.run.smithi204.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 987 kB/s | 16 kB 00:00 2024-07-29T00:23:35.406 INFO:teuthology.orchestra.run.smithi204.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-07-29T00:23:35.431 INFO:teuthology.orchestra.run.smithi204.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-07-29T00:23:35.448 INFO:teuthology.orchestra.run.smithi204.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 673 kB/s | 28 kB 00:00 2024-07-29T00:23:35.449 INFO:teuthology.orchestra.run.smithi181.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 259 kB/s | 222 kB 00:00 2024-07-29T00:23:35.465 INFO:teuthology.orchestra.run.smithi204.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 914 kB/s | 31 kB 00:00 2024-07-29T00:23:35.482 INFO:teuthology.orchestra.run.smithi204.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-07-29T00:23:35.507 INFO:teuthology.orchestra.run.smithi204.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-07-29T00:23:35.524 INFO:teuthology.orchestra.run.smithi204.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 851 kB/s | 36 kB 00:00 2024-07-29T00:23:35.541 INFO:teuthology.orchestra.run.smithi204.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-07-29T00:23:35.575 INFO:teuthology.orchestra.run.smithi204.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-07-29T00:23:35.600 INFO:teuthology.orchestra.run.smithi204.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-07-29T00:23:35.625 INFO:teuthology.orchestra.run.smithi204.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-07-29T00:23:35.651 INFO:teuthology.orchestra.run.smithi204.stdout:(145/160): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-07-29T00:23:35.676 INFO:teuthology.orchestra.run.smithi204.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-29T00:23:35.727 INFO:teuthology.orchestra.run.smithi204.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-07-29T00:23:35.874 INFO:teuthology.orchestra.run.smithi181.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 103 kB/s | 77 kB 00:00 2024-07-29T00:23:35.950 INFO:teuthology.orchestra.run.smithi181.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 65 kB/s | 42 kB 00:00 2024-07-29T00:23:36.177 INFO:teuthology.orchestra.run.smithi204.stdout:(148/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.3 MB/s | 16 MB 00:02 2024-07-29T00:23:36.260 INFO:teuthology.orchestra.run.smithi204.stdout:(149/160): librados2-19.1.0-1150.gba82afec.el9. 5.9 MB/s | 3.4 MB 00:00 2024-07-29T00:23:36.325 INFO:teuthology.orchestra.run.smithi181.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 308 kB/s | 269 kB 00:00 2024-07-29T00:23:36.330 INFO:teuthology.orchestra.run.smithi204.stdout:(150/160): librbd1-19.1.0-1150.gba82afec.el9.x8 5.3 MB/s | 3.2 MB 00:00 2024-07-29T00:23:36.378 INFO:teuthology.orchestra.run.smithi204.stdout:(151/160): NetworkManager-1.48.2-2.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-07-29T00:23:36.403 INFO:teuthology.orchestra.run.smithi204.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 541 kB/s | 41 kB 00:00 2024-07-29T00:23:36.428 INFO:teuthology.orchestra.run.smithi204.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-07-29T00:23:36.454 INFO:teuthology.orchestra.run.smithi204.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 732 kB/s | 37 kB 00:00 2024-07-29T00:23:36.482 INFO:teuthology.orchestra.run.smithi204.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-07-29T00:23:36.529 INFO:teuthology.orchestra.run.smithi204.stdout:(156/160): NetworkManager-libnm-1.48.2-2.el9.x8 6.7 MB/s | 1.8 MB 00:00 2024-07-29T00:23:36.554 INFO:teuthology.orchestra.run.smithi204.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.5 MB/s | 260 kB 00:00 2024-07-29T00:23:36.597 INFO:teuthology.orchestra.run.smithi204.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-07-29T00:23:36.647 INFO:teuthology.orchestra.run.smithi204.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-07-29T00:23:36.680 INFO:teuthology.orchestra.run.smithi204.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-07-29T00:23:36.684 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-07-29T00:23:36.685 INFO:teuthology.orchestra.run.smithi204.stdout:Total 25 MB/s | 228 MB 00:09 2024-07-29T00:23:36.716 INFO:teuthology.orchestra.run.smithi181.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 189 kB/s | 159 kB 00:00 2024-07-29T00:23:36.842 INFO:teuthology.orchestra.run.smithi181.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 313 kB/s | 279 kB 00:00 2024-07-29T00:23:37.026 INFO:teuthology.orchestra.run.smithi181.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 73 kB/s | 51 kB 00:00 2024-07-29T00:23:37.409 INFO:teuthology.orchestra.run.smithi181.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 77 kB/s | 54 kB 00:00 2024-07-29T00:23:37.751 INFO:teuthology.orchestra.run.smithi181.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 57 kB/s | 42 kB 00:00 2024-07-29T00:23:37.864 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-07-29T00:23:38.056 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-07-29T00:23:38.056 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-07-29T00:23:38.076 INFO:teuthology.orchestra.run.smithi181.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 66 kB/s | 44 kB 00:00 2024-07-29T00:23:38.427 INFO:teuthology.orchestra.run.smithi181.stdout:(95/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-07-29T00:23:38.735 INFO:teuthology.orchestra.run.smithi181.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 224 kB/s | 221 kB 00:00 2024-07-29T00:23:38.860 INFO:teuthology.orchestra.run.smithi181.stdout:(97/160): qatlib-service-24.02.0-1.el9.x86_64.r 46 kB/s | 36 kB 00:00 2024-07-29T00:23:39.185 INFO:teuthology.orchestra.run.smithi181.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-07-29T00:23:39.577 INFO:teuthology.orchestra.run.smithi181.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 81 kB/s | 58 kB 00:00 2024-07-29T00:23:39.711 INFO:teuthology.orchestra.run.smithi181.stdout:(100/160): socat-1.7.4.1-6.el9.x86_64.rpm 320 kB/s | 304 kB 00:00 2024-07-29T00:23:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-07-29T00:23:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-07-29T00:23:40.061 INFO:teuthology.orchestra.run.smithi181.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 109 kB/s | 96 kB 00:00 2024-07-29T00:23:40.178 INFO:teuthology.orchestra.run.smithi181.stdout:(102/160): lua-devel-5.4.4-4.el9.x86_64.rpm 249 kB/s | 22 kB 00:00 2024-07-29T00:23:40.337 INFO:teuthology.orchestra.run.smithi181.stdout:(103/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 84 kB/s | 64 kB 00:00 2024-07-29T00:23:40.370 INFO:teuthology.orchestra.run.smithi181.stdout:(104/160): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-07-29T00:23:40.471 INFO:teuthology.orchestra.run.smithi181.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-07-29T00:23:40.487 INFO:teuthology.orchestra.run.smithi181.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-07-29T00:23:40.521 INFO:teuthology.orchestra.run.smithi181.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-07-29T00:23:40.546 INFO:teuthology.orchestra.run.smithi181.stdout:(108/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 54 kB/s | 45 kB 00:00 2024-07-29T00:23:40.572 INFO:teuthology.orchestra.run.smithi181.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 527 kB/s | 26 kB 00:00 2024-07-29T00:23:40.622 INFO:teuthology.orchestra.run.smithi181.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-07-29T00:23:40.656 INFO:teuthology.orchestra.run.smithi181.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-07-29T00:23:40.681 INFO:teuthology.orchestra.run.smithi181.stdout:(112/160): liboath-2.6.7-2.el9.x86_64.rpm 363 kB/s | 49 kB 00:00 2024-07-29T00:23:40.723 INFO:teuthology.orchestra.run.smithi181.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-07-29T00:23:40.748 INFO:teuthology.orchestra.run.smithi181.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-29T00:23:40.773 INFO:teuthology.orchestra.run.smithi181.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-29T00:23:40.798 INFO:teuthology.orchestra.run.smithi181.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-07-29T00:23:40.841 INFO:teuthology.orchestra.run.smithi181.stdout:(117/160): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-07-29T00:23:40.866 INFO:teuthology.orchestra.run.smithi181.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-07-29T00:23:40.899 INFO:teuthology.orchestra.run.smithi181.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-07-29T00:23:40.925 INFO:teuthology.orchestra.run.smithi181.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-07-29T00:23:40.950 INFO:teuthology.orchestra.run.smithi181.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-07-29T00:23:41.034 INFO:teuthology.orchestra.run.smithi181.stdout:(122/160): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-07-29T00:23:41.059 INFO:teuthology.orchestra.run.smithi181.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-07-29T00:23:41.084 INFO:teuthology.orchestra.run.smithi181.stdout:(124/160): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-07-29T00:23:41.109 INFO:teuthology.orchestra.run.smithi181.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-07-29T00:23:41.135 INFO:teuthology.orchestra.run.smithi181.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-07-29T00:23:41.146 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-07-29T00:23:41.160 INFO:teuthology.orchestra.run.smithi181.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-07-29T00:23:41.185 INFO:teuthology.orchestra.run.smithi181.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-07-29T00:23:41.211 INFO:teuthology.orchestra.run.smithi181.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-29T00:23:41.236 INFO:teuthology.orchestra.run.smithi181.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-29T00:23:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-29T00:23:41.261 INFO:teuthology.orchestra.run.smithi181.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-29T00:23:41.303 INFO:teuthology.orchestra.run.smithi181.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-07-29T00:23:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-29T00:23:41.329 INFO:teuthology.orchestra.run.smithi181.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-07-29T00:23:41.354 INFO:teuthology.orchestra.run.smithi181.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-07-29T00:23:41.379 INFO:teuthology.orchestra.run.smithi181.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-29T00:23:41.405 INFO:teuthology.orchestra.run.smithi181.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-07-29T00:23:41.430 INFO:teuthology.orchestra.run.smithi181.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-07-29T00:23:41.436 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-29T00:23:41.455 INFO:teuthology.orchestra.run.smithi181.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-07-29T00:23:41.480 INFO:teuthology.orchestra.run.smithi181.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-07-29T00:23:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-29T00:23:41.505 INFO:teuthology.orchestra.run.smithi181.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-07-29T00:23:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-29T00:23:41.531 INFO:teuthology.orchestra.run.smithi181.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-29T00:23:41.556 INFO:teuthology.orchestra.run.smithi181.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-07-29T00:23:41.607 INFO:teuthology.orchestra.run.smithi181.stdout:(143/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-07-29T00:23:41.620 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-29T00:23:41.632 INFO:teuthology.orchestra.run.smithi181.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-07-29T00:23:41.649 INFO:teuthology.orchestra.run.smithi181.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-07-29T00:23:41.674 INFO:teuthology.orchestra.run.smithi181.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 7.6 MB/s | 192 kB 00:00 2024-07-29T00:23:41.699 INFO:teuthology.orchestra.run.smithi181.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-07-29T00:23:41.750 INFO:teuthology.orchestra.run.smithi181.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-07-29T00:23:41.950 INFO:teuthology.orchestra.run.smithi181.stdout:(149/160): librbd1-19.1.0-1150.gba82afec.el9.x8 13 MB/s | 3.2 MB 00:00 2024-07-29T00:23:42.036 INFO:teuthology.orchestra.run.smithi181.stdout:(150/160): librados2-19.1.0-1150.gba82afec.el9. 9.5 MB/s | 3.4 MB 00:00 2024-07-29T00:23:42.066 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-29T00:23:42.169 INFO:teuthology.orchestra.run.smithi181.stdout:(151/160): NetworkManager-1.48.2-2.el9.x86_64.r 5.5 MB/s | 2.3 MB 00:00 2024-07-29T00:23:42.179 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:23:42.194 INFO:teuthology.orchestra.run.smithi181.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 257 kB/s | 41 kB 00:00 2024-07-29T00:23:42.228 INFO:teuthology.orchestra.run.smithi181.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 4.1 MB/s | 247 kB 00:00 2024-07-29T00:23:42.278 INFO:teuthology.orchestra.run.smithi181.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 439 kB/s | 37 kB 00:00 2024-07-29T00:23:42.304 INFO:teuthology.orchestra.run.smithi181.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.7 MB/s | 207 kB 00:00 2024-07-29T00:23:42.357 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:23:42.388 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:23:42.395 INFO:teuthology.orchestra.run.smithi181.stdout:(156/160): openssl-3.2.2-2.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-07-29T00:23:42.421 INFO:teuthology.orchestra.run.smithi181.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.8 MB/s | 260 kB 00:00 2024-07-29T00:23:42.530 INFO:teuthology.orchestra.run.smithi181.stdout:(158/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-07-29T00:23:42.596 INFO:teuthology.orchestra.run.smithi181.stdout:(159/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.8 MB/s | 1.8 MB 00:00 2024-07-29T00:23:42.630 INFO:teuthology.orchestra.run.smithi181.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 5.2 MB/s | 1.1 MB 00:00 2024-07-29T00:23:42.636 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-07-29T00:23:42.636 INFO:teuthology.orchestra.run.smithi181.stdout:Total 8.7 MB/s | 228 MB 00:26 2024-07-29T00:23:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:23:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 10/172 2024-07-29T00:23:42.859 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-29T00:23:43.142 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-29T00:23:43.221 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-29T00:23:43.251 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:23:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:23:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 15/172 2024-07-29T00:23:43.538 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 15/172 2024-07-29T00:23:43.570 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-29T00:23:43.659 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-29T00:23:43.841 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-07-29T00:23:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-29T00:23:44.030 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-07-29T00:23:44.030 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-07-29T00:23:44.114 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-29T00:23:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-29T00:23:44.645 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-29T00:23:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-29T00:23:44.868 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:44.981 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:45.420 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-29T00:23:45.540 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-29T00:23:45.579 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-07-29T00:23:45.624 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-07-29T00:23:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-07-29T00:23:45.768 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-07-29T00:23:45.807 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-07-29T00:23:45.836 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-07-29T00:23:45.837 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-07-29T00:23:45.893 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-07-29T00:23:45.962 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-07-29T00:23:45.995 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-07-29T00:23:46.110 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-07-29T00:23:46.136 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-07-29T00:23:46.178 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-07-29T00:23:46.201 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-29T00:23:46.213 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-07-29T00:23:46.239 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-29T00:23:46.268 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-07-29T00:23:46.293 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-29T00:23:46.327 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-29T00:23:46.353 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-29T00:23:46.378 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-07-29T00:23:46.396 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-29T00:23:46.416 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.1.0-1150.gba82afec.el9. 36/172 2024-07-29T00:23:46.463 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.1.0-1150.gba82afec.el 37/172 2024-07-29T00:23:46.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_6 38/172 2024-07-29T00:23:46.689 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-07-29T00:23:46.728 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-29T00:23:46.752 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-07-29T00:23:46.846 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:23:46.849 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-07-29T00:23:46.999 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:23:47.019 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-07-29T00:23:47.071 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:23:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-07-29T00:23:47.167 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:23:47.192 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 10/172 2024-07-29T00:23:47.225 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-29T00:23:47.413 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-29T00:23:47.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-29T00:23:47.494 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:23:47.580 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:23:47.604 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 15/172 2024-07-29T00:23:47.649 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 15/172 2024-07-29T00:23:47.671 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-29T00:23:47.691 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-29T00:23:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-07-29T00:23:47.887 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-29T00:23:47.928 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-07-29T00:23:47.973 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-07-29T00:23:47.989 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-29T00:23:48.011 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-07-29T00:23:48.075 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-07-29T00:23:48.121 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.1.0-1150.gba82afec.el9.x86_6 49/172 2024-07-29T00:23:48.177 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-29T00:23:48.192 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-07-29T00:23:48.321 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-29T00:23:48.365 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-29T00:23:48.428 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-07-29T00:23:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:48.494 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-07-29T00:23:48.550 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:48.572 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-07-29T00:23:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-29T00:23:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-29T00:23:48.751 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:48.758 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-29T00:23:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-29T00:23:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-29T00:23:48.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-29T00:23:48.963 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-29T00:23:49.010 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-29T00:23:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-29T00:23:49.047 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-07-29T00:23:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-07-29T00:23:49.156 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-07-29T00:23:49.200 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-07-29T00:23:49.237 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-29T00:23:49.240 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-07-29T00:23:49.285 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-07-29T00:23:49.301 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-29T00:23:49.316 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-07-29T00:23:49.344 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-07-29T00:23:49.356 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-29T00:23:49.413 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-07-29T00:23:49.467 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-07-29T00:23:49.497 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-07-29T00:23:49.499 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-29T00:23:49.532 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-07-29T00:23:49.550 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-29T00:23:49.609 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-07-29T00:23:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-29T00:23:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-29T00:23:49.660 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.1.0-1150.gba82afec.el9. 36/172 2024-07-29T00:23:49.692 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-29T00:23:49.705 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.1.0-1150.gba82afec.el 37/172 2024-07-29T00:23:49.732 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_6 38/172 2024-07-29T00:23:49.803 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-07-29T00:23:49.847 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-07-29T00:23:49.881 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-29T00:23:49.921 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-07-29T00:23:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-29T00:23:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-07-29T00:23:50.114 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-29T00:23:50.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-07-29T00:23:50.151 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-29T00:23:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-29T00:23:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-29T00:23:50.318 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-29T00:23:50.367 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-29T00:23:50.414 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-29T00:23:50.464 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-29T00:23:50.540 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-29T00:23:50.596 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-29T00:23:50.655 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-07-29T00:23:50.738 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-07-29T00:23:50.739 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-29T00:23:50.766 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-07-29T00:23:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-07-29T00:23:50.802 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-29T00:23:50.844 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-29T00:23:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-07-29T00:23:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.1.0-1150.gba82afec.el9.x86_6 49/172 2024-07-29T00:23:50.932 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-07-29T00:23:51.008 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-29T00:23:51.081 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-29T00:23:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-07-29T00:23:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-07-29T00:23:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-07-29T00:23:51.270 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-29T00:23:51.324 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-29T00:23:51.338 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-29T00:23:51.385 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-29T00:23:51.421 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-29T00:23:51.464 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-29T00:23:51.506 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-29T00:23:51.644 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-29T00:23:51.691 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-29T00:23:51.733 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-29T00:23:51.832 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-29T00:23:51.867 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-29T00:23:51.907 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-29T00:23:51.932 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-29T00:23:51.957 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-29T00:23:52.032 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-29T00:23:52.046 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-29T00:23:52.098 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-29T00:23:52.139 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-29T00:23:52.214 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-07-29T00:23:52.261 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-29T00:23:52.314 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-29T00:23:52.323 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-29T00:23:52.359 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-29T00:23:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-29T00:23:52.375 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-29T00:23:52.412 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-29T00:23:52.423 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-29T00:23:52.446 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-29T00:23:52.451 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-29T00:23:52.472 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-29T00:23:52.485 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-29T00:23:52.485 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-29T00:23:52.514 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-29T00:23:52.524 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-29T00:23:52.525 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-29T00:23:52.556 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-29T00:23:52.586 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-29T00:23:52.620 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-29T00:23:52.654 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-29T00:23:52.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-29T00:23:52.746 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-29T00:23:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-29T00:23:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-29T00:23:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-29T00:23:52.844 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-29T00:23:52.868 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-29T00:23:52.900 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-29T00:23:52.916 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-29T00:23:52.947 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:23:53.003 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-29T00:23:53.050 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-29T00:23:53.079 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:23:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:23:53.194 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:23:53.219 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 10/172 2024-07-29T00:23:53.244 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-29T00:23:53.419 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-29T00:23:53.476 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-29T00:23:53.504 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:23:53.600 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:23:53.622 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 15/172 2024-07-29T00:23:53.685 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 15/172 2024-07-29T00:23:53.706 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-29T00:23:53.716 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-29T00:23:53.725 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-29T00:23:53.788 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-29T00:23:53.831 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-29T00:23:53.852 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-29T00:23:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-29T00:23:53.932 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-29T00:23:53.968 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-29T00:23:54.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-29T00:23:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-29T00:23:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-29T00:23:54.163 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-29T00:23:54.202 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-29T00:23:54.210 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-29T00:23:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-29T00:23:54.398 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-29T00:23:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-29T00:23:54.442 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-29T00:23:54.486 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-29T00:23:54.540 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:54.618 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:54.815 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-29T00:23:54.923 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-29T00:23:55.019 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-29T00:23:55.057 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-07-29T00:23:55.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-07-29T00:23:55.137 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-29T00:23:55.157 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-07-29T00:23:55.202 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-07-29T00:23:55.233 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-07-29T00:23:55.249 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-29T00:23:55.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-07-29T00:23:55.318 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-07-29T00:23:55.346 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-07-29T00:23:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-07-29T00:23:55.469 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-07-29T00:23:55.470 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-29T00:23:55.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-07-29T00:23:55.534 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-07-29T00:23:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-29T00:23:55.607 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-29T00:23:55.610 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-07-29T00:23:55.642 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-29T00:23:55.645 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.1.0-1150.gba82afec.el9. 36/172 2024-07-29T00:23:55.690 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.1.0-1150.gba82afec.el 37/172 2024-07-29T00:23:55.717 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_6 38/172 2024-07-29T00:23:55.787 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-07-29T00:23:55.840 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-07-29T00:23:55.914 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-07-29T00:23:56.032 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-29T00:23:56.056 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-07-29T00:23:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 104/172 2024-07-29T00:23:56.110 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-07-29T00:23:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 104/172 2024-07-29T00:23:56.249 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 105/172 2024-07-29T00:23:56.337 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-29T00:23:56.502 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-29T00:23:56.605 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-29T00:23:56.633 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-07-29T00:23:56.640 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-29T00:23:56.673 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-29T00:23:56.714 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-07-29T00:23:56.751 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-07-29T00:23:56.787 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-07-29T00:23:56.836 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-07-29T00:23:56.874 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.1.0-1150.gba82afec.el9.x86_6 49/172 2024-07-29T00:23:56.885 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-29T00:23:56.916 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-07-29T00:23:56.920 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 108/172 2024-07-29T00:23:56.982 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-29T00:23:57.021 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 108/172 2024-07-29T00:23:57.028 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 104/172 2024-07-29T00:23:57.046 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 109/172 2024-07-29T00:23:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-07-29T00:23:57.106 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 104/172 2024-07-29T00:23:57.127 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-07-29T00:23:57.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 105/172 2024-07-29T00:23:57.182 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-07-29T00:23:57.255 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-29T00:23:57.286 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-29T00:23:57.323 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-29T00:23:57.328 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-29T00:23:57.377 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-29T00:23:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-29T00:23:57.448 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-29T00:23:57.490 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-29T00:23:57.627 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-29T00:23:57.676 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-29T00:23:57.717 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-29T00:23:57.807 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-29T00:23:57.842 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-29T00:23:57.882 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-29T00:23:57.908 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-29T00:23:57.933 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-29T00:23:57.958 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-29T00:23:57.998 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 108/172 2024-07-29T00:23:58.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-29T00:23:58.054 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 108/172 2024-07-29T00:23:58.083 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 109/172 2024-07-29T00:23:58.135 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-29T00:23:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-29T00:23:58.193 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-29T00:23:58.228 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-29T00:23:58.258 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-29T00:23:58.290 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-29T00:23:58.320 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-29T00:23:58.355 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-29T00:23:58.466 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-29T00:23:58.635 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-29T00:23:58.764 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-29T00:23:58.850 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-29T00:23:58.898 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-29T00:23:58.942 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-29T00:23:59.037 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-29T00:23:59.084 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-29T00:23:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:23:59.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:23:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-29T00:23:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-29T00:23:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-29T00:23:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-29T00:24:00.037 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-29T00:24:00.082 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-29T00:24:00.161 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-29T00:24:00.227 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-29T00:24:00.324 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:00.359 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:00.365 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:00.405 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 111/172 2024-07-29T00:24:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-29T00:24:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-29T00:24:00.518 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-29T00:24:00.542 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 111/172 2024-07-29T00:24:00.542 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-29T00:24:00.542 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-29T00:24:00.542 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:00.589 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 112/172 2024-07-29T00:24:00.908 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:00.944 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 111/172 2024-07-29T00:24:01.044 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 111/172 2024-07-29T00:24:01.044 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-29T00:24:01.044 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-29T00:24:01.044 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:01.078 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 112/172 2024-07-29T00:24:01.163 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-29T00:24:01.272 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-29T00:24:02.502 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-29T00:24:02.588 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-29T00:24:02.624 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-29T00:24:02.656 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-29T00:24:02.967 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-29T00:24:03.012 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 104/172 2024-07-29T00:24:03.089 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 104/172 2024-07-29T00:24:03.117 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 105/172 2024-07-29T00:24:03.152 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-29T00:24:03.642 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-29T00:24:03.667 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 108/172 2024-07-29T00:24:03.740 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 108/172 2024-07-29T00:24:03.769 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 109/172 2024-07-29T00:24:06.080 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:06.110 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:06.653 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 110/172 2024-07-29T00:24:06.688 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 111/172 2024-07-29T00:24:06.780 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 111/172 2024-07-29T00:24:06.781 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-29T00:24:06.781 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-29T00:24:06.781 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:06.823 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 112/172 2024-07-29T00:24:12.752 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 112/172 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-07-29T00:24:12.753 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:12.928 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 113/172 2024-07-29T00:24:12.943 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 113/172 2024-07-29T00:24:12.943 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:12.943 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-29T00:24:12.943 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:13.336 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 112/172 2024-07-29T00:24:13.336 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-07-29T00:24:13.336 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-07-29T00:24:13.337 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-07-29T00:24:13.337 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-07-29T00:24:13.337 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-07-29T00:24:13.337 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-07-29T00:24:13.337 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-07-29T00:24:13.337 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:13.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 113/172 2024-07-29T00:24:13.623 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 113/172 2024-07-29T00:24:13.623 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:13.623 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-29T00:24:13.623 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:14.440 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 114/172 2024-07-29T00:24:14.466 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 114/172 2024-07-29T00:24:14.466 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:14.466 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-29T00:24:14.466 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-29T00:24:14.466 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-29T00:24:14.466 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:14.691 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 115/172 2024-07-29T00:24:14.714 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 115/172 2024-07-29T00:24:14.714 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:14.714 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-29T00:24:14.714 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-29T00:24:14.714 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-29T00:24:14.714 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:15.150 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 116/172 2024-07-29T00:24:15.172 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 116/172 2024-07-29T00:24:15.172 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:15.172 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-29T00:24:15.172 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-29T00:24:15.172 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-29T00:24:15.172 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:15.262 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 114/172 2024-07-29T00:24:15.280 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-29T00:24:15.341 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 114/172 2024-07-29T00:24:15.341 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:15.341 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-29T00:24:15.341 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-29T00:24:15.342 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-29T00:24:15.342 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:15.345 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-29T00:24:15.401 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-29T00:24:15.589 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-29T00:24:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 115/172 2024-07-29T00:24:15.634 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-29T00:24:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 115/172 2024-07-29T00:24:15.662 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:15.662 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-29T00:24:15.662 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-29T00:24:15.662 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-29T00:24:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:15.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-29T00:24:15.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-29T00:24:15.951 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-29T00:24:15.984 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-29T00:24:16.016 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-29T00:24:16.052 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-29T00:24:16.146 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 116/172 2024-07-29T00:24:16.217 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 116/172 2024-07-29T00:24:16.217 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:16.217 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-29T00:24:16.217 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-29T00:24:16.217 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-29T00:24:16.217 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:16.346 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-29T00:24:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-29T00:24:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-29T00:24:16.555 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.1.0-1150.gba82afec.el9.noarch 128/172 2024-07-29T00:24:16.584 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.1.0-1150.gba82afec.el9.noarch 128/172 2024-07-29T00:24:16.615 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.e 129/172 2024-07-29T00:24:16.678 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1150.gba82afec. 130/172 2024-07-29T00:24:16.707 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 131/172 2024-07-29T00:24:16.772 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-29T00:24:16.833 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-29T00:24:16.917 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-29T00:24:17.059 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-29T00:24:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-29T00:24:17.274 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-29T00:24:17.322 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-29T00:24:17.348 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-29T00:24:17.750 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 131/172 2024-07-29T00:24:17.784 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 132/172 2024-07-29T00:24:17.980 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.1.0-1150.gba82afec.el9.noarch 128/172 2024-07-29T00:24:18.017 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.1.0-1150.gba82afec.el9.noarch 128/172 2024-07-29T00:24:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.e 129/172 2024-07-29T00:24:18.210 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1150.gba82afec. 130/172 2024-07-29T00:24:18.236 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 131/172 2024-07-29T00:24:18.572 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 112/172 2024-07-29T00:24:18.572 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-07-29T00:24:18.572 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-07-29T00:24:18.572 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-07-29T00:24:18.573 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-07-29T00:24:18.573 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-07-29T00:24:18.573 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-07-29T00:24:18.573 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-07-29T00:24:18.573 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:18.740 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 132/172 2024-07-29T00:24:18.746 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 113/172 2024-07-29T00:24:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 113/172 2024-07-29T00:24:18.762 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:18.762 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-29T00:24:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:18.768 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 133/172 2024-07-29T00:24:18.870 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 133/172 2024-07-29T00:24:19.038 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el 134/172 2024-07-29T00:24:19.070 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 135/172 2024-07-29T00:24:19.094 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 135/172 2024-07-29T00:24:19.094 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:19.094 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-29T00:24:19.094 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-29T00:24:19.094 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-29T00:24:19.094 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:19.146 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 136/172 2024-07-29T00:24:19.157 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 136/172 2024-07-29T00:24:19.257 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.1.0-1150.gba82afec.el9.x86_64 137/172 2024-07-29T00:24:19.304 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 138/172 2024-07-29T00:24:19.542 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 131/172 2024-07-29T00:24:19.596 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 132/172 2024-07-29T00:24:20.230 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-29T00:24:20.257 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 114/172 2024-07-29T00:24:20.265 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 140/172 2024-07-29T00:24:20.282 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 114/172 2024-07-29T00:24:20.282 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.282 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-29T00:24:20.282 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-29T00:24:20.282 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-29T00:24:20.282 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:20.289 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 140/172 2024-07-29T00:24:20.289 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.289 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-29T00:24:20.289 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-29T00:24:20.289 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-29T00:24:20.289 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:20.333 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1150.gba82a 141/172 2024-07-29T00:24:20.355 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1150.gba82a 141/172 2024-07-29T00:24:20.355 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.355 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-29T00:24:20.355 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:20.400 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 142/172 2024-07-29T00:24:20.423 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 142/172 2024-07-29T00:24:20.423 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.423 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-29T00:24:20.423 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-29T00:24:20.423 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-29T00:24:20.423 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:20.508 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 115/172 2024-07-29T00:24:20.530 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 115/172 2024-07-29T00:24:20.531 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.531 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-29T00:24:20.531 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-29T00:24:20.531 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-29T00:24:20.531 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:20.688 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 132/172 2024-07-29T00:24:20.721 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 133/172 2024-07-29T00:24:20.725 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 143/172 2024-07-29T00:24:20.748 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 143/172 2024-07-29T00:24:20.748 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.748 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-29T00:24:20.749 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-29T00:24:20.749 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-29T00:24:20.749 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:20.867 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 133/172 2024-07-29T00:24:20.967 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 116/172 2024-07-29T00:24:20.989 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 116/172 2024-07-29T00:24:20.989 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:20.990 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-29T00:24:20.990 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-29T00:24:20.990 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-29T00:24:20.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:21.054 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el 134/172 2024-07-29T00:24:21.081 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 135/172 2024-07-29T00:24:21.089 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-29T00:24:21.151 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 135/172 2024-07-29T00:24:21.151 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:21.151 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-29T00:24:21.151 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-29T00:24:21.151 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-29T00:24:21.151 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:21.154 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-29T00:24:21.207 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 136/172 2024-07-29T00:24:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-29T00:24:21.245 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 136/172 2024-07-29T00:24:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.1.0-1150.gba82afec.el9.x86_64 137/172 2024-07-29T00:24:21.402 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-29T00:24:21.422 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 138/172 2024-07-29T00:24:21.451 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-29T00:24:21.520 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-29T00:24:21.617 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-29T00:24:21.756 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-29T00:24:21.793 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-29T00:24:21.824 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-29T00:24:21.852 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-29T00:24:22.304 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.1.0-1150.gba82afec.el9.noarch 128/172 2024-07-29T00:24:22.334 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.1.0-1150.gba82afec.el9.noarch 128/172 2024-07-29T00:24:22.366 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.e 129/172 2024-07-29T00:24:22.401 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-29T00:24:22.428 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1150.gba82afec. 130/172 2024-07-29T00:24:22.451 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 140/172 2024-07-29T00:24:22.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 131/172 2024-07-29T00:24:22.520 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 140/172 2024-07-29T00:24:22.520 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:22.520 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-29T00:24:22.520 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-29T00:24:22.520 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-29T00:24:22.520 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:22.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1150.gba82a 141/172 2024-07-29T00:24:22.647 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1150.gba82a 141/172 2024-07-29T00:24:22.647 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:22.647 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-29T00:24:22.648 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:22.745 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 142/172 2024-07-29T00:24:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 142/172 2024-07-29T00:24:22.826 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:22.826 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-29T00:24:22.826 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-29T00:24:22.826 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-29T00:24:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:23.203 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 143/172 2024-07-29T00:24:23.277 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 143/172 2024-07-29T00:24:23.277 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-29T00:24:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-29T00:24:23.278 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-29T00:24:23.278 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:23.495 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 131/172 2024-07-29T00:24:23.534 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 132/172 2024-07-29T00:24:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 132/172 2024-07-29T00:24:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 133/172 2024-07-29T00:24:24.645 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 133/172 2024-07-29T00:24:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el 134/172 2024-07-29T00:24:24.837 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 135/172 2024-07-29T00:24:24.860 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 135/172 2024-07-29T00:24:24.860 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:24.860 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-29T00:24:24.861 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-29T00:24:24.861 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-29T00:24:24.861 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:24.904 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 136/172 2024-07-29T00:24:24.915 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 136/172 2024-07-29T00:24:25.014 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.1.0-1150.gba82afec.el9.x86_64 137/172 2024-07-29T00:24:25.061 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 138/172 2024-07-29T00:24:25.288 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 144/172 2024-07-29T00:24:25.330 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 145/172 2024-07-29T00:24:25.370 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 146/172 2024-07-29T00:24:25.402 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-29T00:24:25.429 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-29T00:24:25.618 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 149/172 2024-07-29T00:24:25.659 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-29T00:24:25.691 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_ 151/172 2024-07-29T00:24:25.732 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-29T00:24:25.762 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-29T00:24:25.808 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-29T00:24:25.972 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-29T00:24:25.998 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 140/172 2024-07-29T00:24:26.021 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 140/172 2024-07-29T00:24:26.021 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:26.021 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-29T00:24:26.021 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-29T00:24:26.021 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-29T00:24:26.021 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:26.074 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1150.gba82a 141/172 2024-07-29T00:24:26.096 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1150.gba82a 141/172 2024-07-29T00:24:26.096 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:26.096 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-29T00:24:26.096 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 142/172 2024-07-29T00:24:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 142/172 2024-07-29T00:24:26.164 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:26.164 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-29T00:24:26.164 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-29T00:24:26.164 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-29T00:24:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:26.423 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:26.461 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:26.467 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 143/172 2024-07-29T00:24:26.481 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 143/172 2024-07-29T00:24:26.489 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-29T00:24:26.489 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-29T00:24:26.489 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-29T00:24:26.489 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-29T00:24:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:27.335 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 156/172 2024-07-29T00:24:27.464 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-29T00:24:27.505 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-29T00:24:27.590 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-29T00:24:27.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-29T00:24:27.667 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-29T00:24:27.693 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-29T00:24:27.719 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-29T00:24:27.719 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-29T00:24:27.803 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-29T00:24:27.828 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-29T00:24:27.828 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-29T00:24:27.938 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-29T00:24:27.963 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-29T00:24:27.968 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:27.968 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:28.151 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:28.151 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-29T00:24:28.216 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 144/172 2024-07-29T00:24:28.235 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-29T00:24:28.259 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 145/172 2024-07-29T00:24:28.268 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-29T00:24:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-29T00:24:28.313 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 146/172 2024-07-29T00:24:28.348 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-29T00:24:28.358 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-29T00:24:28.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 149/172 2024-07-29T00:24:28.745 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-29T00:24:28.780 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_ 151/172 2024-07-29T00:24:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-29T00:24:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-29T00:24:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-29T00:24:29.893 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:29.941 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:29.997 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:31.020 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 144/172 2024-07-29T00:24:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 145/172 2024-07-29T00:24:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 146/172 2024-07-29T00:24:31.134 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-29T00:24:31.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-29T00:24:31.346 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 149/172 2024-07-29T00:24:31.372 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 156/172 2024-07-29T00:24:31.407 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-29T00:24:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_ 151/172 2024-07-29T00:24:31.474 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-29T00:24:31.484 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-29T00:24:31.515 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-29T00:24:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-29T00:24:31.570 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-29T00:24:31.669 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-29T00:24:31.708 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-29T00:24:31.741 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-29T00:24:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-29T00:24:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-29T00:24:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-29T00:24:31.975 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-29T00:24:32.021 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-29T00:24:32.022 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-29T00:24:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-29T00:24:32.180 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-29T00:24:32.182 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:32.214 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:32.214 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:32.235 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:32.254 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-29T00:24:32.559 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:32.560 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-29T00:24:32.704 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-29T00:24:32.757 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-29T00:24:32.795 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-29T00:24:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:33.087 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 156/172 2024-07-29T00:24:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-29T00:24:33.181 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:33.182 INFO:teuthology.orchestra.run.smithi175.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-07-29T00:24:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:33.209 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-29T00:24:33.298 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-29T00:24:33.355 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-29T00:24:33.381 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-29T00:24:33.406 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-29T00:24:33.433 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-29T00:24:33.433 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-29T00:24:33.525 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-29T00:24:33.550 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-29T00:24:33.550 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-29T00:24:33.635 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-29T00:24:33.659 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-29T00:24:33.664 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:33.664 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:33.853 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-29T00:24:33.853 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-29T00:24:33.937 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-29T00:24:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-29T00:24:33.996 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-29T00:24:34.060 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.1.0-1150.gba82afec.el9.x86_64 1/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 2/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 3/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 4/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1150.gba82a 5/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 6/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 7/172 2024-07-29T00:24:34.076 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 10/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 11/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 12/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 13/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_ 15/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 16/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 17/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.1.0-1150.gba82afec.el9.x86_6 18/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 19/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 20/172 2024-07-29T00:24:34.077 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1150.gba82afec.el 21/172 2024-07-29T00:24:34.078 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.1.0-1150.gba82afec.el9. 22/172 2024-07-29T00:24:34.078 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_6 23/172 2024-07-29T00:24:34.078 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 24/172 2024-07-29T00:24:34.078 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 25/172 2024-07-29T00:24:34.078 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 26/172 2024-07-29T00:24:34.078 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 27/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 28/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 29/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1150.gba82afec. 30/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 31/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 32/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 33/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el 34/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 35/172 2024-07-29T00:24:34.092 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.e 36/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 37/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.1.0-1150.gba82afec.el9.noarch 38/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 39/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-29T00:24:34.093 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-29T00:24:34.094 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-29T00:24:34.095 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-29T00:24:34.096 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-29T00:24:34.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-29T00:24:34.098 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-29T00:24:34.099 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-29T00:24:34.100 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-29T00:24:34.101 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.1.0-1150.gba82afec.el9.x86_64 149/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 151/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-29T00:24:34.102 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-29T00:24:34.103 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-29T00:24:34.104 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-29T00:24:34.104 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-29T00:24:37.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-29T00:24:37.044 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.045 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.046 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-29T00:24:37.047 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-29T00:24:37.048 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-29T00:24:37.049 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-29T00:24:37.050 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-29T00:24:37.051 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-29T00:24:37.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-29T00:24:37.053 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-29T00:24:37.054 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:37.055 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:24:37.056 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-07-29T00:24:37.477 DEBUG:teuthology.parallel:result is None 2024-07-29T00:24:39.039 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.1.0-1150.gba82afec.el9.x86_64 1/172 2024-07-29T00:24:39.039 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 2/172 2024-07-29T00:24:39.039 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 3/172 2024-07-29T00:24:39.039 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 4/172 2024-07-29T00:24:39.039 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1150.gba82a 5/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 6/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 7/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 10/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 11/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 12/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 13/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_ 15/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 16/172 2024-07-29T00:24:39.040 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 17/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.1.0-1150.gba82afec.el9.x86_6 18/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 19/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 20/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1150.gba82afec.el 21/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.1.0-1150.gba82afec.el9. 22/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_6 23/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 24/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 25/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 26/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 27/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 28/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 29/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1150.gba82afec. 30/172 2024-07-29T00:24:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 31/172 2024-07-29T00:24:39.042 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 32/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 33/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el 34/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 35/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.e 36/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 37/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.1.0-1150.gba82afec.el9.noarch 38/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 39/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-29T00:24:39.054 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-29T00:24:39.055 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-29T00:24:39.056 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-29T00:24:39.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-29T00:24:39.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-29T00:24:39.059 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-29T00:24:39.060 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-29T00:24:39.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.1.0-1150.gba82afec.el9.x86_64 149/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-29T00:24:39.062 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 151/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-29T00:24:39.063 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-29T00:24:39.064 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-29T00:24:39.064 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-29T00:24:39.064 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-29T00:24:39.064 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-29T00:24:39.064 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-29T00:24:39.637 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:39.637 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.1.0-1150.gba82afec.el9.x86_64 1/172 2024-07-29T00:24:39.637 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 2/172 2024-07-29T00:24:39.637 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 3/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 4/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1150.gba82a 5/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 6/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 7/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 8/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 9/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 10/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 11/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 12/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 13/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 14/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_ 15/172 2024-07-29T00:24:39.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 16/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 17/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.1.0-1150.gba82afec.el9.x86_6 18/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86 19/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 20/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1150.gba82afec.el 21/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.1.0-1150.gba82afec.el9. 22/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_6 23/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 24/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 25/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 26/172 2024-07-29T00:24:39.639 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 27/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 28/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 29/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1150.gba82afec. 30/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noa 31/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.n 32/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1150.gba8 33/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el 34/172 2024-07-29T00:24:39.652 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 35/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.e 36/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 37/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.1.0-1150.gba82afec.el9.noarch 38/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 39/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-29T00:24:39.653 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-29T00:24:39.654 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-29T00:24:39.654 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-29T00:24:39.654 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-29T00:24:39.654 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-29T00:24:39.654 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-29T00:24:39.655 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-29T00:24:39.656 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-29T00:24:39.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-29T00:24:39.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-29T00:24:39.659 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-29T00:24:39.660 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-29T00:24:39.661 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-29T00:24:39.662 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-29T00:24:39.663 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.1.0-1150.gba82afec.el9.x86_64 149/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 151/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-29T00:24:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-29T00:24:39.665 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-29T00:24:39.666 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-29T00:24:39.666 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-29T00:24:41.377 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-29T00:24:41.378 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.380 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-29T00:24:41.381 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-29T00:24:41.382 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-29T00:24:41.383 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-29T00:24:41.384 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-29T00:24:41.385 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-29T00:24:41.386 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-29T00:24:41.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.388 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-29T00:24:41.389 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-29T00:24:41.390 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-29T00:24:41.391 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:24:41.391 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-07-29T00:24:41.990 DEBUG:teuthology.parallel:result is None 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-29T00:24:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.434 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.1.0-1150.gba82afec.el9.noarch 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-29T00:24:42.435 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-29T00:24:42.436 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-29T00:24:42.437 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: lua-5.4.4-4.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.14-1.el9.x86_64 2024-07-29T00:24:42.438 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-29T00:24:42.439 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-29T00:24:42.440 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-29T00:24:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-29T00:24:42.442 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-29T00:24:42.443 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.1.0-1150.gba82afec.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-3.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-29T00:24:42.444 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-29T00:24:42.445 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-29T00:24:42.445 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-29T00:24:42.445 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-29T00:24:42.445 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-29T00:24:42.445 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:24:42.445 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-07-29T00:24:42.883 DEBUG:teuthology.parallel:result is None 2024-07-29T00:24:42.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:24:43.029 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-29T00:24:43.087 INFO:teuthology.orchestra.run.smithi175.stdout:19.1.0-1150.gba82afec.el9 2024-07-29T00:24:43.088 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1150.gba82afec.el9 2024-07-29T00:24:43.088 INFO:teuthology.task.install:The correct ceph version 19.1.0-1150.gba82afec is installed. 2024-07-29T00:24:43.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:24:43.233 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-29T00:24:43.256 INFO:teuthology.orchestra.run.smithi181.stdout:19.1.0-1150.gba82afec.el9 2024-07-29T00:24:43.256 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1150.gba82afec.el9 2024-07-29T00:24:43.256 INFO:teuthology.task.install:The correct ceph version 19.1.0-1150.gba82afec is installed. 2024-07-29T00:24:43.258 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:24:43.392 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-29T00:24:43.418 INFO:teuthology.orchestra.run.smithi204.stdout:19.1.0-1150.gba82afec.el9 2024-07-29T00:24:43.418 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1150.gba82afec.el9 2024-07-29T00:24:43.418 INFO:teuthology.task.install:The correct ceph version 19.1.0-1150.gba82afec is installed. 2024-07-29T00:24:43.420 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-29T00:24:43.420 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:24:43.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-29T00:24:43.492 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:24:43.492 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-29T00:24:43.521 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:24:43.521 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-29T00:24:43.552 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-29T00:24:43.553 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:24:43.553 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-07-29T00:24:43.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-29T00:24:43.723 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:24:43.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-07-29T00:24:43.752 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-29T00:24:43.823 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:24:43.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-07-29T00:24:43.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-29T00:24:43.918 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-29T00:24:43.919 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:24:43.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-29T00:24:43.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-29T00:24:44.084 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:24:44.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-29T00:24:44.112 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-29T00:24:44.179 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:24:44.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-29T00:24:44.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-29T00:24:44.275 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-29T00:24:44.275 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:24:44.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-07-29T00:24:44.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-29T00:24:44.453 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:24:44.454 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-07-29T00:24:44.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-29T00:24:44.550 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:24:44.550 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-07-29T00:24:44.578 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-29T00:24:44.646 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-29T00:24:44.737 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701', 'subvols': {'create': 2}} 2024-07-29T00:24:44.737 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:24:44.737 INFO:tasks.cephadm:Cluster fsid is f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:24:44.737 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-29T00:24:44.738 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.175', 'mon.b': '172.21.15.181', 'mon.c': '172.21.15.204'} 2024-07-29T00:24:44.738 INFO:tasks.cephadm:First mon is mon.a on smithi175 2024-07-29T00:24:44.738 INFO:tasks.cephadm:First mgr is x 2024-07-29T00:24:44.738 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-29T00:24:44.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-07-29T00:24:44.803 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-07-29T00:24:44.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-07-29T00:24:44.860 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-07-29T00:24:44.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:24:45.002 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ba82afec3628bf830c0830558bf3d3c3aea66701', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55715/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1150-gba82afec', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1150.gba82afec'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-28 16:18:35.150758', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-07-29T00:24:45.141 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 ba82afec3628bf830c0830558bf3d3c3aea66701 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:24:45.142 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/x86_64/flavors/default/cephadm 2024-07-29T00:24:45.142 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/x86_64/flavors/default/cephadm 2024-07-29T00:24:45.142 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:45.553 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 784412 Jul 29 00:24 /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:45.553 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:45.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r--. 1 ubuntu ubuntu 784412 Jul 29 00:24 /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:45.903 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/ba82afec3628bf830c0830558bf3d3c3aea66701/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:46.323 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 784412 Jul 29 00:24 /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:46.324 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:46.385 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:46.404 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-29T00:24:46.471 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 on all hosts... 2024-07-29T00:24:46.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 pull 2024-07-29T00:24:46.478 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 pull 2024-07-29T00:24:46.484 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 pull 2024-07-29T00:24:47.339 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701... 2024-07-29T00:24:47.350 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701... 2024-07-29T00:24:47.909 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701... 2024-07-29T00:25:23.216 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:25:23.217 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.1.0-1150-gba82afec (ba82afec3628bf830c0830558bf3d3c3aea66701) squid (rc)", 2024-07-29T00:25:23.217 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef", 2024-07-29T00:25:23.217 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-07-29T00:25:23.217 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6" 2024-07-29T00:25:23.217 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:25:23.217 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:25:29.682 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-07-29T00:25:29.683 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.1.0-1150-gba82afec (ba82afec3628bf830c0830558bf3d3c3aea66701) squid (rc)", 2024-07-29T00:25:29.683 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef", 2024-07-29T00:25:29.683 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-07-29T00:25:29.683 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6" 2024-07-29T00:25:29.683 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-07-29T00:25:29.683 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-07-29T00:25:41.860 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-07-29T00:25:41.861 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.1.0-1150-gba82afec (ba82afec3628bf830c0830558bf3d3c3aea66701) squid (rc)", 2024-07-29T00:25:41.861 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef", 2024-07-29T00:25:41.861 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-07-29T00:25:41.861 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6" 2024-07-29T00:25:41.861 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-07-29T00:25:41.861 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-07-29T00:25:41.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-07-29T00:25:41.979 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-07-29T00:25:42.006 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-07-29T00:25:42.039 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-07-29T00:25:42.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-07-29T00:25:42.150 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-07-29T00:25:42.178 INFO:tasks.cephadm:Writing seed config... 2024-07-29T00:25:42.179 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-07-29T00:25:42.179 INFO:tasks.cephadm: override: [client] debug client = 20 2024-07-29T00:25:42.179 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-07-29T00:25:42.179 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-07-29T00:25:42.179 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-07-29T00:25:42.180 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-29T00:25:42.181 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-29T00:25:42.182 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-07-29T00:25:42.182 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-07-29T00:25:42.182 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-29T00:25:42.182 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-07-29T00:25:42.182 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-29T00:25:42.183 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-29T00:25:42.184 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-29T00:25:42.184 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-29T00:25:42.184 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-29T00:25:42.184 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-07-29T00:25:42.184 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-07-29T00:25:42.184 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-07-29T00:25:42.185 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:25:42.185 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-29T00:25:42.244 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f484ff76-4d40-11ef-bcc8-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-07-29T00:25:42.244 DEBUG:teuthology.orchestra.run.smithi175:mon.a> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service 2024-07-29T00:25:42.251 DEBUG:teuthology.orchestra.run.smithi175:mgr.x> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x.service 2024-07-29T00:25:42.297 INFO:tasks.cephadm:Bootstrapping... 2024-07-29T00:25:42.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 -v bootstrap --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.175 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:25:42.584 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-07-29T00:25:42.585 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701', '-v', 'bootstrap', '--fsid', 'f484ff76-4d40-11ef-bcc8-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.175', '--skip-admin-label'] 2024-07-29T00:25:42.585 INFO:teuthology.orchestra.run.smithi175.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-29T00:25:42.585 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying podman|docker is present... 2024-07-29T00:25:42.639 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stdout 5.1.2 2024-07-29T00:25:42.640 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying lvm2 is present... 2024-07-29T00:25:42.640 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying time synchronization is in place... 2024-07-29T00:25:42.655 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-29T00:25:42.655 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-29T00:25:42.668 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-29T00:25:42.668 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-07-29T00:25:42.690 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout enabled 2024-07-29T00:25:42.710 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout active 2024-07-29T00:25:42.710 INFO:teuthology.orchestra.run.smithi175.stdout:Unit chronyd.service is enabled and running 2024-07-29T00:25:42.711 INFO:teuthology.orchestra.run.smithi175.stdout:Repeating the final host check... 2024-07-29T00:25:42.766 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stdout 5.1.2 2024-07-29T00:25:42.766 INFO:teuthology.orchestra.run.smithi175.stdout:podman (/bin/podman) version 5.1.2 is present 2024-07-29T00:25:42.766 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl is present 2024-07-29T00:25:42.766 INFO:teuthology.orchestra.run.smithi175.stdout:lvcreate is present 2024-07-29T00:25:42.783 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-29T00:25:42.784 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-29T00:25:42.800 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-29T00:25:42.800 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-07-29T00:25:42.824 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout enabled 2024-07-29T00:25:42.846 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout active 2024-07-29T00:25:42.846 INFO:teuthology.orchestra.run.smithi175.stdout:Unit chronyd.service is enabled and running 2024-07-29T00:25:42.846 INFO:teuthology.orchestra.run.smithi175.stdout:Host looks OK 2024-07-29T00:25:42.846 INFO:teuthology.orchestra.run.smithi175.stdout:Cluster fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:42.847 INFO:teuthology.orchestra.run.smithi175.stdout:Acquiring lock 140112059250816 on /run/cephadm/f484ff76-4d40-11ef-bcc8-c7b262605968.lock 2024-07-29T00:25:42.847 INFO:teuthology.orchestra.run.smithi175.stdout:Lock 140112059250816 acquired on /run/cephadm/f484ff76-4d40-11ef-bcc8-c7b262605968.lock 2024-07-29T00:25:42.848 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying IP 172.21.15.175 port 3300 ... 2024-07-29T00:25:42.850 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying IP 172.21.15.175 port 6789 ... 2024-07-29T00:25:42.850 INFO:teuthology.orchestra.run.smithi175.stdout:Base mon IP(s) is [172.21.15.175:3300, 172.21.15.175:6789], mon addrv is [v2:172.21.15.175:3300,v1:172.21.15.175:6789] 2024-07-29T00:25:42.864 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.175 metric 100 2024-07-29T00:25:42.865 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.175 metric 100 2024-07-29T00:25:42.877 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-29T00:25:42.878 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-07-29T00:25:42.878 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2024-07-29T00:25:42.889 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-29T00:25:42.889 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-29T00:25:42.889 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-29T00:25:42.889 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-07-29T00:25:42.889 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-07-29T00:25:42.890 INFO:teuthology.orchestra.run.smithi175.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-29T00:25:42.892 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.0/20` 2024-07-29T00:25:42.892 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.0/20` 2024-07-29T00:25:42.892 INFO:teuthology.orchestra.run.smithi175.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-29T00:25:42.895 INFO:teuthology.orchestra.run.smithi175.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-29T00:25:42.897 INFO:teuthology.orchestra.run.smithi175.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701... 2024-07-29T00:25:43.592 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stdout 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef 2024-07-29T00:25:43.592 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701... 2024-07-29T00:25:43.592 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stderr Getting image source signatures 2024-07-29T00:25:43.592 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stderr Copying blob sha256:43735fbce9214f061a6894341d7822563f5aafe16fc7532189f4415cff6d2d91 2024-07-29T00:25:43.592 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stderr Copying blob sha256:0e736ca6873bebb47a2a9c3b9b947ada1f11edfacfdfb320accc6f898ea05ebf 2024-07-29T00:25:43.592 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stderr Copying config sha256:07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef 2024-07-29T00:25:43.593 INFO:teuthology.orchestra.run.smithi175.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-29T00:25:44.783 INFO:teuthology.orchestra.run.smithi175.stdout:ceph: stdout ceph version 19.1.0-1150-gba82afec (ba82afec3628bf830c0830558bf3d3c3aea66701) squid (rc) 2024-07-29T00:25:44.783 INFO:teuthology.orchestra.run.smithi175.stdout:Ceph version: ceph version 19.1.0-1150-gba82afec (ba82afec3628bf830c0830558bf3d3c3aea66701) squid (rc) 2024-07-29T00:25:44.783 INFO:teuthology.orchestra.run.smithi175.stdout:Extracting ceph user uid/gid from container image... 2024-07-29T00:25:45.955 INFO:teuthology.orchestra.run.smithi175.stdout:stat: stdout 167 167 2024-07-29T00:25:45.955 INFO:teuthology.orchestra.run.smithi175.stdout:Creating initial keys... 2024-07-29T00:25:47.049 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-authtool: stdout AQCK4aZmvN7yFxAABcHlzwE72AKk4DOCYf3rXA== 2024-07-29T00:25:48.136 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-authtool: stdout AQCL4aZmIJbWIBAAwuCfdcCZMz+HDTL12dOidw== 2024-07-29T00:25:49.230 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-authtool: stdout AQCM4aZmyCcXIhAAXL8ew7593+ntkvgizZnyvQ== 2024-07-29T00:25:49.231 INFO:teuthology.orchestra.run.smithi175.stdout:Creating initial monmap... 2024-07-29T00:25:50.377 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-29T00:25:50.377 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-07-29T00:25:50.377 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:50.377 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-29T00:25:50.377 INFO:teuthology.orchestra.run.smithi175.stdout:monmaptool for a [v2:172.21.15.175:3300,v1:172.21.15.175:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-29T00:25:50.378 INFO:teuthology.orchestra.run.smithi175.stdout:setting min_mon_release = quincy 2024-07-29T00:25:50.378 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: set fsid to f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:50.378 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-29T00:25:50.378 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:25:50.378 INFO:teuthology.orchestra.run.smithi175.stdout:Creating mon... 2024-07-29T00:25:51.429 INFO:teuthology.orchestra.run.smithi175.stdout:create mon.a on 2024-07-29T00:25:51.761 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-29T00:25:51.999 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-29T00:25:52.326 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968.target → /etc/systemd/system/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968.target. 2024-07-29T00:25:52.326 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968.target → /etc/systemd/system/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968.target. 2024-07-29T00:25:52.716 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a 2024-07-29T00:25:52.716 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service: Unit ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service not loaded. 2024-07-29T00:25:53.006 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968.target.wants/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@.service. 2024-07-29T00:25:53.735 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:53 smithi175 podman[31749]: 2024-07-29 00:25:53.661663567 +0000 UTC m=+0.372383739 container start 910e25fe589413ebb70828cdbf1ce833a3bb62ca4bb62f22289b5b37e99f1367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-29T00:25:53.790 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-29T00:25:53.790 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout disabled 2024-07-29T00:25:53.805 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-29T00:25:53.806 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-07-29T00:25:53.806 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld.service is not enabled 2024-07-29T00:25:53.806 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to enable service . firewalld.service is not available 2024-07-29T00:25:53.807 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mon to start... 2024-07-29T00:25:53.808 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mon... 2024-07-29T00:25:54.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:53 smithi175 ceph-mon[31768]: mkfs f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:54.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:53 smithi175 ceph-mon[31768]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-29T00:25:54.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:53 smithi175 bash[31749]: 910e25fe589413ebb70828cdbf1ce833a3bb62ca4bb62f22289b5b37e99f1367 2024-07-29T00:25:54.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:53 smithi175 systemd[1]: Started Ceph mon.a for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: monmap epoch 1 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: last_changed 2024-07-29T00:25:49.691743+0000 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: min_mon_release 19 (squid) 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: election_strategy: 1 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: fsmap 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: osdmap e1: 0 total, 0 up, 0 in 2024-07-29T00:25:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: mgrmap e1: no daemons active 2024-07-29T00:25:55.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:54 smithi175 ceph-mon[31768]: from='client.? 172.21.15.175:0/3231437435' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-07-29T00:25:55.131 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout cluster: 2024-07-29T00:25:55.144 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout id: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:55.144 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-29T00:25:55.144 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:25:55.145 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout services: 2024-07-29T00:25:55.145 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.744107s) 2024-07-29T00:25:55.145 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-29T00:25:55.145 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-29T00:25:55.145 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:25:55.146 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout data: 2024-07-29T00:25:55.146 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-29T00:25:55.146 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-29T00:25:55.146 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-29T00:25:55.146 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout pgs: 2024-07-29T00:25:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:25:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.393+0000 7f5998a00640 1 Processor -- start 2024-07-29T00:25:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.394+0000 7f5998a00640 1 -- start start 2024-07-29T00:25:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.394+0000 7f5998a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59940814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:25:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.394+0000 7f5998a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5994081a70 con 0x7f59940810a0 2024-07-29T00:25:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.395+0000 7f5992800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59940814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:25:55.148 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.395+0000 7f5992800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59940814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38978/0 (socket says 172.21.15.175:38978) 2024-07-29T00:25:55.148 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.395+0000 7f5992800640 1 -- 172.21.15.175:0/1515146055 learned_addr learned my addr 172.21.15.175:0/1515146055 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:25:55.148 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.396+0000 7f5992800640 1 -- 172.21.15.175:0/1515146055 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5994081bf0 con 0x7f59940810a0 2024-07-29T00:25:55.148 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.399+0000 7f5992800640 1 --2- 172.21.15.175:0/1515146055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59940814a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5984009820 tx=0x7f598402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f99292f026e5de server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:25:55.149 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.399+0000 7f5991e00640 1 -- 172.21.15.175:0/1515146055 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5984037070 con 0x7f59940810a0 2024-07-29T00:25:55.149 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.399+0000 7f5991e00640 1 -- 172.21.15.175:0/1515146055 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5984032440 con 0x7f59940810a0 2024-07-29T00:25:55.149 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.399+0000 7f5991e00640 1 -- 172.21.15.175:0/1515146055 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5984032740 con 0x7f59940810a0 2024-07-29T00:25:55.149 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 -- 172.21.15.175:0/1515146055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 msgr2=0x7f59940814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:25:55.150 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 --2- 172.21.15.175:0/1515146055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59940814a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5984009820 tx=0x7f598402a270 comp rx=0 tx=0).stop 2024-07-29T00:25:55.150 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 -- 172.21.15.175:0/1515146055 shutdown_connections 2024-07-29T00:25:55.150 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 --2- 172.21.15.175:0/1515146055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59940814a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:25:55.150 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 -- 172.21.15.175:0/1515146055 >> 172.21.15.175:0/1515146055 conn(0x7f599407ca70 msgr2=0x7f599407ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:25:55.150 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 -- 172.21.15.175:0/1515146055 shutdown_connections 2024-07-29T00:25:55.150 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.400+0000 7f5998a00640 1 -- 172.21.15.175:0/1515146055 wait complete. 2024-07-29T00:25:55.151 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.401+0000 7f5998a00640 1 Processor -- start 2024-07-29T00:25:55.151 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.401+0000 7f5998a00640 1 -- start start 2024-07-29T00:25:55.151 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.402+0000 7f5998a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59941a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:25:55.151 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.402+0000 7f5998a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5994111100 con 0x7f59940810a0 2024-07-29T00:25:55.151 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.402+0000 7f5992800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59941a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:25:55.151 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.402+0000 7f5992800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59941a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38986/0 (socket says 172.21.15.175:38986) 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.402+0000 7f5992800640 1 -- 172.21.15.175:0/3231437435 learned_addr learned my addr 172.21.15.175:0/3231437435 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.402+0000 7f5992800640 1 -- 172.21.15.175:0/3231437435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59941a14f0 con 0x7f59940810a0 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f5992800640 1 --2- 172.21.15.175:0/3231437435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59941a0fb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59840090b0 tx=0x7f59840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5984041070 con 0x7f59940810a0 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598403c7a0 con 0x7f59940810a0 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5984037040 con 0x7f59940810a0 2024-07-29T00:25:55.152 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59941a1780 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59941a4470 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.403+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5984044070 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.404+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f598403d3f0 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.404+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5994081520 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.407+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f5984046450 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.473+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f599402d720 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.473+0000 7f598be00640 1 -- 172.21.15.175:0/3231437435 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f5984046630 con 0x7f59940810a0 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.474+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 msgr2=0x7f59941a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.474+0000 7f5998a00640 1 --2- 172.21.15.175:0/3231437435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59941a0fb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59840090b0 tx=0x7f59840046c0 comp rx=0 tx=0).stop 2024-07-29T00:25:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.475+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 shutdown_connections 2024-07-29T00:25:55.154 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.475+0000 7f5998a00640 1 --2- 172.21.15.175:0/3231437435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59940810a0 0x7f59941a0fb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:25:55.154 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.475+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 >> 172.21.15.175:0/3231437435 conn(0x7f599407ca70 msgr2=0x7f599407dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:25:55.154 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.475+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 shutdown_connections 2024-07-29T00:25:55.154 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:54.475+0000 7f5998a00640 1 -- 172.21.15.175:0/3231437435 wait complete. 2024-07-29T00:25:55.154 INFO:teuthology.orchestra.run.smithi175.stdout:mon is available 2024-07-29T00:25:55.154 INFO:teuthology.orchestra.run.smithi175.stdout:Assimilating anything we can from ceph.conf... 2024-07-29T00:25:56.169 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:55 smithi175 ceph-mon[31768]: from='client.? 172.21.15.175:0/1554200989' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-29T00:25:56.169 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:55 smithi175 ceph-mon[31768]: from='client.? 172.21.15.175:0/1554200989' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [global] 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout fsid = f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.175:3300,v1:172.21.15.175:6789] 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-29T00:25:56.476 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [osd] 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.725+0000 7f48a7200640 1 Processor -- start 2024-07-29T00:25:56.477 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.725+0000 7f48a7200640 1 -- start start 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.726+0000 7f48a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a81075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.726+0000 7f48a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8107ba0 con 0x7f48a81071d0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.726+0000 7f48a5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a81075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.726+0000 7f48a5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a81075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39002/0 (socket says 172.21.15.175:39002) 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.726+0000 7f48a5e00640 1 -- 172.21.15.175:0/591240149 learned_addr learned my addr 172.21.15.175:0/591240149 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.726+0000 7f48a5e00640 1 -- 172.21.15.175:0/591240149 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a81083d0 con 0x7f48a81071d0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.727+0000 7f48a5e00640 1 --2- 172.21.15.175:0/591240149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a81075d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4898009820 tx=0x7f489802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a466aaab0f058c41 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.728+0000 7f48a4a00640 1 -- 172.21.15.175:0/591240149 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4898037070 con 0x7f48a81071d0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.728+0000 7f48a4a00640 1 -- 172.21.15.175:0/591240149 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4898032440 con 0x7f48a81071d0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.728+0000 7f48a4a00640 1 -- 172.21.15.175:0/591240149 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4898032740 con 0x7f48a81071d0 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.728+0000 7f48a7200640 1 -- 172.21.15.175:0/591240149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 msgr2=0x7f48a81075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:25:56.478 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.728+0000 7f48a7200640 1 --2- 172.21.15.175:0/591240149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a81075d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4898009820 tx=0x7f489802a270 comp rx=0 tx=0).stop 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.729+0000 7f48a7200640 1 -- 172.21.15.175:0/591240149 shutdown_connections 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.729+0000 7f48a7200640 1 --2- 172.21.15.175:0/591240149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a81075d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.729+0000 7f48a7200640 1 -- 172.21.15.175:0/591240149 >> 172.21.15.175:0/591240149 conn(0x7f48a81029a0 msgr2=0x7f48a8104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.729+0000 7f48a7200640 1 -- 172.21.15.175:0/591240149 shutdown_connections 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.729+0000 7f48a7200640 1 -- 172.21.15.175:0/591240149 wait complete. 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.730+0000 7f48a7200640 1 Processor -- start 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.730+0000 7f48a7200640 1 -- start start 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.731+0000 7f48a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.731+0000 7f48a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8108970 con 0x7f48a81071d0 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.731+0000 7f48a5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.731+0000 7f48a5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39014/0 (socket says 172.21.15.175:39014) 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.731+0000 7f48a5e00640 1 -- 172.21.15.175:0/1554200989 learned_addr learned my addr 172.21.15.175:0/1554200989 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:25:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.731+0000 7f48a5e00640 1 -- 172.21.15.175:0/1554200989 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a819aed0 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f48a5e00640 1 --2- 172.21.15.175:0/1554200989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a819a990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f48980090b0 tx=0x7f48980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4898041070 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489803c7a0 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4898037040 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a819b160 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a819de50 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4898044070 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.732+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f489803d3f0 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.733+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a8108fc0 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.736+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f4898046450 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.800+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f48a802d720 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.807+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f4898046630 con 0x7f48a81071d0 2024-07-29T00:25:56.480 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.808+0000 7f489ea00640 1 -- 172.21.15.175:0/1554200989 <== mon.0 v2:172.21.15.175:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f48980584b0 con 0x7f48a81071d0 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.808+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 msgr2=0x7f48a819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.808+0000 7f48a7200640 1 --2- 172.21.15.175:0/1554200989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a819a990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f48980090b0 tx=0x7f48980046c0 comp rx=0 tx=0).stop 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.809+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 shutdown_connections 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.809+0000 7f48a7200640 1 --2- 172.21.15.175:0/1554200989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a81071d0 0x7f48a819a990 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.809+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 >> 172.21.15.175:0/1554200989 conn(0x7f48a81029a0 msgr2=0x7f48a818eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.809+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 shutdown_connections 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:55.809+0000 7f48a7200640 1 -- 172.21.15.175:0/1554200989 wait complete. 2024-07-29T00:25:56.481 INFO:teuthology.orchestra.run.smithi175.stdout:Generating new minimal ceph.conf... 2024-07-29T00:25:57.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:57 smithi175 ceph-mon[31768]: from='client.? 172.21.15.175:0/1368374209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:25:57.868 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.033+0000 7feeeb000640 1 Processor -- start 2024-07-29T00:25:57.868 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.033+0000 7feeeb000640 1 -- start start 2024-07-29T00:25:57.868 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.033+0000 7feeeb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:25:57.868 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.033+0000 7feeeb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feee407b7c0 con 0x7feee407cd90 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.034+0000 7feee9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.034+0000 7feee9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee407b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39018/0 (socket says 172.21.15.175:39018) 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.034+0000 7feee9c00640 1 -- 172.21.15.175:0/3395904885 learned_addr learned my addr 172.21.15.175:0/3395904885 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.034+0000 7feee9c00640 1 -- 172.21.15.175:0/3395904885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee407b940 con 0x7feee407cd90 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.035+0000 7feee9c00640 1 --2- 172.21.15.175:0/3395904885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee407b1f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feed8009820 tx=0x7feed802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c654f9ead2e05d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.035+0000 7feee3e00640 1 -- 172.21.15.175:0/3395904885 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feed8037070 con 0x7feee407cd90 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.035+0000 7feee3e00640 1 -- 172.21.15.175:0/3395904885 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feed8032900 con 0x7feee407cd90 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.035+0000 7feee3e00640 1 -- 172.21.15.175:0/3395904885 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feed8030850 con 0x7feee407cd90 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 -- 172.21.15.175:0/3395904885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 msgr2=0x7feee407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 --2- 172.21.15.175:0/3395904885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee407b1f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feed8009820 tx=0x7feed802a270 comp rx=0 tx=0).stop 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 -- 172.21.15.175:0/3395904885 shutdown_connections 2024-07-29T00:25:57.869 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 --2- 172.21.15.175:0/3395904885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee407b1f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 -- 172.21.15.175:0/3395904885 >> 172.21.15.175:0/3395904885 conn(0x7feee4102980 msgr2=0x7feee4104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 -- 172.21.15.175:0/3395904885 shutdown_connections 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.036+0000 7feeeb000640 1 -- 172.21.15.175:0/3395904885 wait complete. 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.037+0000 7feeeb000640 1 Processor -- start 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feeeb000640 1 -- start start 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feeeb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feeeb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feee410d820 con 0x7feee407cd90 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feee9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feee9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee419d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39032/0 (socket says 172.21.15.175:39032) 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feee9c00640 1 -- 172.21.15.175:0/1368374209 learned_addr learned my addr 172.21.15.175:0/1368374209 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.038+0000 7feee9c00640 1 -- 172.21.15.175:0/1368374209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee419d8c0 con 0x7feee407cd90 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.039+0000 7feee9c00640 1 --2- 172.21.15.175:0/1368374209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee419d380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feed80097f0 tx=0x7feed8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:25:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.039+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feed8037040 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.039+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feed8040070 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.039+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee419db50 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.039+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feed803a390 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.039+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41a0840 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.040+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feed8032900 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.040+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feed80454f0 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.041+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee407b270 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.043+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7feed803b430 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.108+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7feee419dfd0 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.108+0000 7feee2000640 1 -- 172.21.15.175:0/1368374209 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7feed803b7c0 con 0x7feee407cd90 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 msgr2=0x7feee419d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:25:57.871 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 --2- 172.21.15.175:0/1368374209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee419d380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feed80097f0 tx=0x7feed8030d80 comp rx=0 tx=0).stop 2024-07-29T00:25:57.872 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 shutdown_connections 2024-07-29T00:25:57.872 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 --2- 172.21.15.175:0/1368374209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feee407cd90 0x7feee419d380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:25:57.872 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 >> 172.21.15.175:0/1368374209 conn(0x7feee4102980 msgr2=0x7feee4103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:25:57.872 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 shutdown_connections 2024-07-29T00:25:57.872 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:25:57.110+0000 7feeeb000640 1 -- 172.21.15.175:0/1368374209 wait complete. 2024-07-29T00:25:57.872 INFO:teuthology.orchestra.run.smithi175.stdout:Restarting the monitor... 2024-07-29T00:25:58.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:57 smithi175 systemd[1]: Stopping Ceph mon.a for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:25:58.728 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:58 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a[31764]: 2024-07-29T00:25:58.341+0000 7f5847400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:25:58.728 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:58 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a[31764]: 2024-07-29T00:25:58.341+0000 7f5847400640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-07-29T00:25:58.728 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:58 smithi175 podman[32016]: 2024-07-29 00:25:58.393310108 +0000 UTC m=+0.177482495 container died 910e25fe589413ebb70828cdbf1ce833a3bb62ca4bb62f22289b5b37e99f1367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-29T00:25:59.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:58 smithi175 podman[32016]: 2024-07-29 00:25:58.829786413 +0000 UTC m=+0.613958807 container cleanup 910e25fe589413ebb70828cdbf1ce833a3bb62ca4bb62f22289b5b37e99f1367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-07-29T00:25:59.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:58 smithi175 bash[32016]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a 2024-07-29T00:25:59.522 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 podman[32030]: 2024-07-29 00:25:59.064903879 +0000 UTC m=+0.658236436 container remove 910e25fe589413ebb70828cdbf1ce833a3bb62ca4bb62f22289b5b37e99f1367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-07-29T00:25:59.914 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 systemd[1]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-29T00:25:59.915 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 systemd[1]: Stopped Ceph mon.a for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:25:59.915 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 systemd[1]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service: Consumed 1.827s CPU time. 2024-07-29T00:25:59.915 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 systemd[1]: Starting Ceph mon.a for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:26:00.218 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 podman[32120]: 2024-07-29 00:25:59.820148678 +0000 UTC m=+0.033381993 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:26:00.218 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:25:59 smithi175 podman[32120]: 2024-07-29 00:25:59.914617798 +0000 UTC m=+0.127851166 container create 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-07-29T00:26:00.218 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 podman[32120]: 2024-07-29 00:26:00.218049993 +0000 UTC m=+0.431283338 container init 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-29T00:26:00.319 INFO:teuthology.orchestra.run.smithi175.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-29T00:26:00.470 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 podman[32120]: 2024-07-29 00:26:00.228595732 +0000 UTC m=+0.441829073 container start 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724) 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: set uid:gid to 167:167 (ceph:ceph) 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: ceph version 19.1.0-1150-gba82afec (ba82afec3628bf830c0830558bf3d3c3aea66701) squid (rc), process ceph-mon, pid 2 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: pidfile_write: ignore empty --pid-file 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: load: jerasure load: lrc 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: RocksDB version: 7.9.2 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Git sha 0 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Compile date 2024-07-28 15:16:30 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: DB SUMMARY 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: DB Session ID: 13QOVQMWHXB8Z1TGAP3G 2024-07-29T00:26:00.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: CURRENT file: CURRENT 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: IDENTITY file: IDENTITY 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105828 ; 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.error_if_exists: 0 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.create_if_missing: 0 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.paranoid_checks: 1 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.env: 0x5582bc7b37a0 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.fs: PosixFileSystem 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.info_log: 0x5582eef7b440 2024-07-29T00:26:00.472 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_file_opening_threads: 16 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.statistics: (nil) 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.use_fsync: 0 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_log_file_size: 0 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.keep_log_file_num: 1000 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.recycle_log_file_num: 0 2024-07-29T00:26:00.473 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.allow_fallocate: 1 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.allow_mmap_reads: 0 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.allow_mmap_writes: 0 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.use_direct_reads: 0 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.create_missing_column_families: 0 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.db_log_dir: 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.wal_dir: 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-29T00:26:00.474 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.advise_random_on_open: 1 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.db_write_buffer_size: 0 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.write_buffer_manager: 0x5582eef8adc0 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.rate_limiter: (nil) 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-29T00:26:00.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.wal_recovery_mode: 2 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.enable_thread_tracking: 0 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.enable_pipelined_write: 0 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.unordered_write: 0 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.row_cache: None 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.wal_filter: None 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-29T00:26:00.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.allow_ingest_behind: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.two_write_queues: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.manual_wal_flush: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.wal_compression: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.atomic_flush: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.log_readahead_size: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.best_efforts_recovery: 0 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-29T00:26:00.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.allow_data_in_errors: 0 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.db_host_id: __hostname__ 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_background_jobs: 2 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_background_compactions: -1 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_subcompactions: 1 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_total_wal_size: 0 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-29T00:26:00.478 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_open_files: -1 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bytes_per_sync: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_readahead_size: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_background_flushes: -1 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Compression algorithms supported: 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kZSTD supported: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kXpressCompression supported: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kBZip2Compression supported: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-29T00:26:00.479 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kLZ4Compression supported: 1 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kZlibCompression supported: 1 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kLZ4HCCompression supported: 1 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: kSnappyCompression supported: 1 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.merge_operator: 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_filter: None 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_filter_factory: None 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.sst_partitioner_factory: None 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-29T00:26:00.480 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5582eef7b060) 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: index_type: 0 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: data_block_index_type: 0 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: index_shortening: 1 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: checksum: 4 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: no_block_cache: 0 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_cache: 0x5582eef6f1f0 2024-07-29T00:26:00.481 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_cache_options: 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: capacity : 536870912 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: num_shard_bits : 4 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: strict_capacity_limit : 0 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_cache_compressed: (nil) 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: persistent_cache: (nil) 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_size: 4096 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_size_deviation: 10 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_restart_interval: 16 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: index_block_restart_interval: 1 2024-07-29T00:26:00.482 INFO:journalctl@ceph.mon.a.smithi175.stdout: metadata_block_size: 4096 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: partition_filters: 0 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: use_delta_encoding: 1 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: filter_policy: bloomfilter 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: whole_key_filtering: 1 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: verify_compression: 0 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: format_version: 5 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: enable_index_compression: 1 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: block_align: 0 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: max_auto_readahead_size: 262144 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: prepopulate_block_cache: 0 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: initial_auto_readahead_size: 8192 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.write_buffer_size: 33554432 2024-07-29T00:26:00.483 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_write_buffer_number: 2 2024-07-29T00:26:00.484 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression: NoCompression 2024-07-29T00:26:00.484 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression: Disabled 2024-07-29T00:26:00.484 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.prefix_extractor: nullptr 2024-07-29T00:26:00.484 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-29T00:26:00.484 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.num_levels: 7 2024-07-29T00:26:00.484 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-29T00:26:00.485 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-29T00:26:00.486 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.level: 32767 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.strategy: 0 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.enabled: false 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-29T00:26:00.487 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.target_file_size_base: 67108864 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-29T00:26:00.488 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.arena_block_size: 1048576 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.disable_auto_compactions: 0 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-29T00:26:00.489 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.inplace_update_support: 0 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-29T00:26:00.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.bloom_locality: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.max_successive_merges: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.paranoid_file_checks: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.force_consistency_checks: 1 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.report_bg_io_stats: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.ttl: 2592000 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.enable_blob_files: false 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.min_blob_size: 0 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.blob_file_size: 268435456 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-29T00:26:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.blob_file_starting_level: 0 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1fa24f29-b4d0-4ba7-8316-0a7fe5c88af4 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722212760279529, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722212760281677, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100839, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98724, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14423, "raw_average_key_size": 49, "raw_value_size": 90930, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722212760, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1fa24f29-b4d0-4ba7-8316-0a7fe5c88af4", "db_session_id": "13QOVQMWHXB8Z1TGAP3G", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722212760281879, "job": 1, "event": "recovery_finished"} 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-07-29T00:26:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5582eef9ee00 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: DB pointer 0x5582ef098000 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: ** DB Stats ** 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-29T00:26:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout: 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: ** Compaction Stats [default] ** 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: L0 2/0 100.34 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: Sum 2/0 100.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: ** Compaction Stats [default] ** 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 65.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: 2024-07-29T00:26:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Cumulative compaction: 0.00 GB write, 13.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Interval compaction: 0.00 GB write, 13.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Block cache BinnedLRUCache@0x5582eef6f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: 2024-07-29T00:26:00.495 INFO:journalctl@ceph.mon.a.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: starting mon.a rank 0 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???) e1 preinit fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).mds e1 new map 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).mds e1 print_map 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: e1 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: btime 2024-07-29T00:25:53:731837+0000 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: legacy client fscid: -1 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout: No filesystems configured 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-29T00:26:00.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 bash[32120]: 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 systemd[1]: Started Ceph mon.a for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: monmap epoch 1 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: last_changed 2024-07-29T00:25:49.691743+0000 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: min_mon_release 19 (squid) 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: election_strategy: 1 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: fsmap 2024-07-29T00:26:00.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: osdmap e1: 0 total, 0 up, 0 in 2024-07-29T00:26:00.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:00 smithi175 ceph-mon[32138]: mgrmap e1: no daemons active 2024-07-29T00:26:01.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.958+0000 7f5979600640 1 Processor -- start 2024-07-29T00:26:01.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.959+0000 7f5979600640 1 -- start start 2024-07-29T00:26:01.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.959+0000 7f5979600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59740814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:01.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.959+0000 7f5979600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5974081a70 con 0x7f59740810a0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.960+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59740814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.960+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59740814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41894/0 (socket says 172.21.15.175:41894) 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.960+0000 7f5973e00640 1 -- 172.21.15.175:0/425468862 learned_addr learned my addr 172.21.15.175:0/425468862 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.960+0000 7f5973e00640 1 -- 172.21.15.175:0/425468862 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5974081bf0 con 0x7f59740810a0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.963+0000 7f5973e00640 1 --2- 172.21.15.175:0/425468862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59740814a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5964009820 tx=0x7f596402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e9bb660677b15d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.964+0000 7f5973400640 1 -- 172.21.15.175:0/425468862 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5964037070 con 0x7f59740810a0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.964+0000 7f5973400640 1 -- 172.21.15.175:0/425468862 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5964032900 con 0x7f59740810a0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.964+0000 7f5973400640 1 -- 172.21.15.175:0/425468862 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5964030850 con 0x7f59740810a0 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 -- 172.21.15.175:0/425468862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 msgr2=0x7f59740814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 --2- 172.21.15.175:0/425468862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59740814a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5964009820 tx=0x7f596402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:01.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 -- 172.21.15.175:0/425468862 shutdown_connections 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 --2- 172.21.15.175:0/425468862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59740814a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 -- 172.21.15.175:0/425468862 >> 172.21.15.175:0/425468862 conn(0x7f597407ca70 msgr2=0x7f597407ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 -- 172.21.15.175:0/425468862 shutdown_connections 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.965+0000 7f5979600640 1 -- 172.21.15.175:0/425468862 wait complete. 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.966+0000 7f5979600640 1 Processor -- start 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.966+0000 7f5979600640 1 -- start start 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.966+0000 7f5979600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59741a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.966+0000 7f5979600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5974111c60 con 0x7f59740810a0 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.967+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59741a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.967+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59741a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41906/0 (socket says 172.21.15.175:41906) 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.967+0000 7f5973e00640 1 -- 172.21.15.175:0/46884293 learned_addr learned my addr 172.21.15.175:0/46884293 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:01.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.967+0000 7f5973e00640 1 -- 172.21.15.175:0/46884293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59741a1d00 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.967+0000 7f5973e00640 1 --2- 172.21.15.175:0/46884293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59741a17c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5964009940 tx=0x7f5964032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5964042070 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59640308c0 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5964037040 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59741a1f90 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741a4c80 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5964045070 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.968+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5964047bc0 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.969+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5974081520 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:00.972+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f59640327b0 con 0x7f59740810a0 2024-07-29T00:26:01.690 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.036+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f59741a5270 con 0x7f59740810a0 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.040+0000 7f5971600640 1 -- 172.21.15.175:0/46884293 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f596403c870 con 0x7f59740810a0 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.041+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 msgr2=0x7f59741a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.041+0000 7f5979600640 1 --2- 172.21.15.175:0/46884293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59741a17c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5964009940 tx=0x7f5964032ce0 comp rx=0 tx=0).stop 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.042+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 shutdown_connections 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.042+0000 7f5979600640 1 --2- 172.21.15.175:0/46884293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59740810a0 0x7f59741a17c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.042+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 >> 172.21.15.175:0/46884293 conn(0x7f597407ca70 msgr2=0x7f597407d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.042+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 shutdown_connections 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:01.042+0000 7f5979600640 1 -- 172.21.15.175:0/46884293 wait complete. 2024-07-29T00:26:01.691 INFO:teuthology.orchestra.run.smithi175.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-29T00:26:01.713 INFO:teuthology.orchestra.run.smithi175.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:26:01.713 INFO:teuthology.orchestra.run.smithi175.stdout:Creating mgr... 2024-07-29T00:26:01.714 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying port 0.0.0.0:9283 ... 2024-07-29T00:26:01.715 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying port 0.0.0.0:8765 ... 2024-07-29T00:26:02.164 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x 2024-07-29T00:26:02.164 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x.service: Unit ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x.service not loaded. 2024-07-29T00:26:02.432 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968.target.wants/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@.service. 2024-07-29T00:26:02.475 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:02 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/46884293' entity='client.admin' 2024-07-29T00:26:02.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:02 smithi175 systemd[1]: Starting Ceph mgr.x for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:26:03.104 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:02 smithi175 podman[32352]: 2024-07-29 00:26:02.747359268 +0000 UTC m=+0.034934282 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:26:03.104 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:02 smithi175 podman[32352]: 2024-07-29 00:26:02.875358579 +0000 UTC m=+0.162933626 container create 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-07-29T00:26:03.104 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 podman[32352]: 2024-07-29 00:26:03.10369992 +0000 UTC m=+0.391274974 container init 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-29T00:26:03.235 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-29T00:26:03.235 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout disabled 2024-07-29T00:26:03.254 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-29T00:26:03.254 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-07-29T00:26:03.255 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld.service is not enabled 2024-07-29T00:26:03.255 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to enable service . firewalld.service is not available 2024-07-29T00:26:03.284 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-29T00:26:03.284 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout disabled 2024-07-29T00:26:03.305 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-29T00:26:03.305 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-07-29T00:26:03.305 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld.service is not enabled 2024-07-29T00:26:03.306 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-07-29T00:26:03.306 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr to start... 2024-07-29T00:26:03.306 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr... 2024-07-29T00:26:03.469 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 podman[32352]: 2024-07-29 00:26:03.113720232 +0000 UTC m=+0.401295286 container start 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-07-29T00:26:03.469 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 bash[32352]: 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf 2024-07-29T00:26:03.469 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 systemd[1]: Started Ceph mgr.x for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:26:03.469 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:03.309+0000 7f5ec5a4a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:26:03.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:03.506+0000 7f5ec5a4a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:26:03.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:03.585+0000 7f5ec5a4a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:26:04.064 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:03.892+0000 7f5ec5a4a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:26:04.064 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:03.967+0000 7f5ec5a4a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:26:04.521 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:04 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2056459152' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-29T00:26:04.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:04.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-29T00:26:04.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-29T00:26:04.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-29T00:26:04.857 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-29T00:26:04.857 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-29T00:26:04.857 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-29T00:26:04.857 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.857 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "btime": "2024-07-29T00:25:53:731837+0000", 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-29T00:26:04.858 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-29T00:26:04.859 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-07-29T00:25:53.732846+0000", 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.934+0000 7f286ca00640 1 Processor -- start 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.934+0000 7f286ca00640 1 -- start start 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.934+0000 7f286ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f28681075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.934+0000 7f286ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2868107bc0 con 0x7f28681071f0 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.935+0000 7f2867400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f28681075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.935+0000 7f2867400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f28681075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41936/0 (socket says 172.21.15.175:41936) 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.935+0000 7f2867400640 1 -- 172.21.15.175:0/3355722351 learned_addr learned my addr 172.21.15.175:0/3355722351 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:04.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.935+0000 7f2867400640 1 -- 172.21.15.175:0/3355722351 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28681083f0 con 0x7f28681071f0 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.936+0000 7f2867400640 1 --2- 172.21.15.175:0/3355722351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f28681075f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2858009820 tx=0x7f285802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364251949eeb4d61 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.937+0000 7f2866a00640 1 -- 172.21.15.175:0/3355722351 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858037070 con 0x7f28681071f0 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.937+0000 7f2866a00640 1 -- 172.21.15.175:0/3355722351 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2858032900 con 0x7f28681071f0 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.937+0000 7f2866a00640 1 -- 172.21.15.175:0/3355722351 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858030850 con 0x7f28681071f0 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.937+0000 7f286ca00640 1 -- 172.21.15.175:0/3355722351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 msgr2=0x7f28681075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.937+0000 7f286ca00640 1 --2- 172.21.15.175:0/3355722351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f28681075f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2858009820 tx=0x7f285802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.938+0000 7f286ca00640 1 -- 172.21.15.175:0/3355722351 shutdown_connections 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.938+0000 7f286ca00640 1 --2- 172.21.15.175:0/3355722351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f28681075f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.938+0000 7f286ca00640 1 -- 172.21.15.175:0/3355722351 >> 172.21.15.175:0/3355722351 conn(0x7f2868102980 msgr2=0x7f2868104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.938+0000 7f286ca00640 1 -- 172.21.15.175:0/3355722351 shutdown_connections 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.938+0000 7f286ca00640 1 -- 172.21.15.175:0/3355722351 wait complete. 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.939+0000 7f286ca00640 1 Processor -- start 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.939+0000 7f286ca00640 1 -- start start 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.939+0000 7f286ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f286807ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.939+0000 7f286ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f286810b7a0 con 0x7f28681071f0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.940+0000 7f2867400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f286807ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.940+0000 7f2867400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f286807ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41944/0 (socket says 172.21.15.175:41944) 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.940+0000 7f2867400640 1 -- 172.21.15.175:0/2056459152 learned_addr learned my addr 172.21.15.175:0/2056459152 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.940+0000 7f2867400640 1 -- 172.21.15.175:0/2056459152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286807cfc0 con 0x7f28681071f0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.940+0000 7f2867400640 1 --2- 172.21.15.175:0/2056459152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f286807ca80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f28580097f0 tx=0x7f2858030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858037040 con 0x7f28681071f0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2858040070 con 0x7f28681071f0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f285803a390 con 0x7f28681071f0 2024-07-29T00:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286807b1b0 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286807b690 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f285803a5e0 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.941+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2858045950 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.942+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286810b920 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:03.945+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f2858045460 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.010+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f286807b9d0 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.011+0000 7f2864c00640 1 -- 172.21.15.175:0/2056459152 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f285803b430 con 0x7f28681071f0 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.012+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 msgr2=0x7f286807ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.012+0000 7f286ca00640 1 --2- 172.21.15.175:0/2056459152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f286807ca80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f28580097f0 tx=0x7f2858030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.013+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 shutdown_connections 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.013+0000 7f286ca00640 1 --2- 172.21.15.175:0/2056459152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28681071f0 0x7f286807ca80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.013+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 >> 172.21.15.175:0/2056459152 conn(0x7f2868102980 msgr2=0x7f28681034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.013+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 shutdown_connections 2024-07-29T00:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:04.013+0000 7f286ca00640 1 -- 172.21.15.175:0/2056459152 wait complete. 2024-07-29T00:26:04.864 INFO:teuthology.orchestra.run.smithi175.stdout:mgr not available, waiting (1/15)... 2024-07-29T00:26:05.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.225+0000 7f5ec5a4a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:26:05.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.359+0000 7f5ec5a4a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:26:05.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.425+0000 7f5ec5a4a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:26:05.977 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.667+0000 7f5ec5a4a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:26:05.977 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.737+0000 7f5ec5a4a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:26:05.977 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.814+0000 7f5ec5a4a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:26:05.977 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:05.977+0000 7f5ec5a4a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:26:06.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:06 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:06.490+0000 7f5ec5a4a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:26:07.101 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:06 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:06.845+0000 7f5ec5a4a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:26:07.102 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:06 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:06.928+0000 7f5ec5a4a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:26:07.102 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:07.101+0000 7f5ec5a4a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:26:07.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:07 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/4183320267' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-29T00:26:07.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:07.480+0000 7f5ec5a4a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:26:07.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:07.562+0000 7f5ec5a4a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:26:08.058 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:07.784+0000 7f5ec5a4a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:26:08.058 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:07.982+0000 7f5ec5a4a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:26:08.058 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:08.058+0000 7f5ec5a4a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-29T00:26:08.302 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-29T00:26:08.303 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-29T00:26:08.304 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "btime": "2024-07-29T00:25:53:731837+0000", 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-29T00:26:08.305 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-07-29T00:25:53.732846+0000", 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:08.306 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-29T00:26:08.307 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:08.307 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.451+0000 7f5d57e00640 1 Processor -- start 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.451+0000 7f5d57e00640 1 -- start start 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.452+0000 7f5d57e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d581075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.452+0000 7f5d57e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58107bc0 con 0x7f5d581071f0 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.452+0000 7f5d56a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d581075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.452+0000 7f5d56a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d581075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41954/0 (socket says 172.21.15.175:41954) 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.452+0000 7f5d56a00640 1 -- 172.21.15.175:0/2274444054 learned_addr learned my addr 172.21.15.175:0/2274444054 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:08.311 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.453+0000 7f5d56a00640 1 -- 172.21.15.175:0/2274444054 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d581083f0 con 0x7f5d581071f0 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.454+0000 7f5d56a00640 1 --2- 172.21.15.175:0/2274444054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d581075f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009820 tx=0x7f5d4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94c89701a77cbd1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.454+0000 7f5d56000640 1 -- 172.21.15.175:0/2274444054 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d48037070 con 0x7f5d581071f0 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.454+0000 7f5d56000640 1 -- 172.21.15.175:0/2274444054 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d48032900 con 0x7f5d581071f0 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.454+0000 7f5d56000640 1 -- 172.21.15.175:0/2274444054 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d48030850 con 0x7f5d581071f0 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 -- 172.21.15.175:0/2274444054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 msgr2=0x7f5d581075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 --2- 172.21.15.175:0/2274444054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d581075f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009820 tx=0x7f5d4802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 -- 172.21.15.175:0/2274444054 shutdown_connections 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 --2- 172.21.15.175:0/2274444054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d581075f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 -- 172.21.15.175:0/2274444054 >> 172.21.15.175:0/2274444054 conn(0x7f5d58102980 msgr2=0x7f5d58104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 -- 172.21.15.175:0/2274444054 shutdown_connections 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.455+0000 7f5d57e00640 1 -- 172.21.15.175:0/2274444054 wait complete. 2024-07-29T00:26:08.312 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.456+0000 7f5d57e00640 1 Processor -- start 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.457+0000 7f5d57e00640 1 -- start start 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.457+0000 7f5d57e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d5819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.457+0000 7f5d57e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5810b7a0 con 0x7f5d581071f0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.457+0000 7f5d56a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d5819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.457+0000 7f5d56a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d5819d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41968/0 (socket says 172.21.15.175:41968) 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.457+0000 7f5d56a00640 1 -- 172.21.15.175:0/4183320267 learned_addr learned my addr 172.21.15.175:0/4183320267 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.458+0000 7f5d56a00640 1 -- 172.21.15.175:0/4183320267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5819d8c0 con 0x7f5d581071f0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.458+0000 7f5d56a00640 1 --2- 172.21.15.175:0/4183320267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d5819d380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5d480097f0 tx=0x7f5d48030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.458+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d48037040 con 0x7f5d581071f0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.458+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d48040070 con 0x7f5d581071f0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.458+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d4803a390 con 0x7f5d581071f0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.458+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5819db50 con 0x7f5d581071f0 2024-07-29T00:26:08.313 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.459+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581a0840 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.459+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5d4803a5e0 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.459+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5d48045950 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.460+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d58107670 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.462+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f5d48045460 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.529+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5d581a0a10 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.530+0000 7f5d4f400640 1 -- 172.21.15.175:0/4183320267 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f5d4803b430 con 0x7f5d581071f0 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.531+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 msgr2=0x7f5d5819d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.531+0000 7f5d57e00640 1 --2- 172.21.15.175:0/4183320267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d5819d380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5d480097f0 tx=0x7f5d48030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.531+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 shutdown_connections 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.531+0000 7f5d57e00640 1 --2- 172.21.15.175:0/4183320267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d581071f0 0x7f5d5819d380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.531+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 >> 172.21.15.175:0/4183320267 conn(0x7f5d58102980 msgr2=0x7f5d58103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.531+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 shutdown_connections 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:07.532+0000 7f5d57e00640 1 -- 172.21.15.175:0/4183320267 wait complete. 2024-07-29T00:26:08.314 INFO:teuthology.orchestra.run.smithi175.stdout:mgr not available, waiting (2/15)... 2024-07-29T00:26:08.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:08.154+0000 7f5ec5a4a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:26:08.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:08.225+0000 7f5ec5a4a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:26:09.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:08.844+0000 7f5ec5a4a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:09.023+0000 7f5ec5a4a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: Activating manager daemon x 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: mgrmap e2: x(active, starting, since 0.0120501s) 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:26:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: Manager daemon x is now available 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' 2024-07-29T00:26:09.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:09 smithi175 ceph-mon[32138]: from='mgr.14100 172.21.15.175:0/2360285398' entity='mgr.x' 2024-07-29T00:26:11.047 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:11 smithi175 ceph-mon[32138]: mgrmap e3: x(active, since 1.02091s) 2024-07-29T00:26:11.833 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:11.834 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-29T00:26:11.835 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-29T00:26:11.844 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "btime": "2024-07-29T00:25:53:731837+0000", 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-29T00:26:11.845 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-07-29T00:25:53.732846+0000", 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-07-29T00:26:11.846 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.916+0000 7ff799000640 1 Processor -- start 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.917+0000 7ff799000640 1 -- start start 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.917+0000 7ff799000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7940814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.917+0000 7ff799000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff794081a70 con 0x7ff7940810a0 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.917+0000 7ff793400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7940814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.917+0000 7ff793400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7940814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58852/0 (socket says 172.21.15.175:58852) 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.917+0000 7ff793400640 1 -- 172.21.15.175:0/2093925835 learned_addr learned my addr 172.21.15.175:0/2093925835 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.918+0000 7ff793400640 1 -- 172.21.15.175:0/2093925835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff794081bf0 con 0x7ff7940810a0 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.919+0000 7ff793400640 1 --2- 172.21.15.175:0/2093925835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7940814a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff784009820 tx=0x7ff78402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7210f5912691ec29 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.919+0000 7ff792a00640 1 -- 172.21.15.175:0/2093925835 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784037070 con 0x7ff7940810a0 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.919+0000 7ff792a00640 1 -- 172.21.15.175:0/2093925835 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff784032900 con 0x7ff7940810a0 2024-07-29T00:26:11.847 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.919+0000 7ff792a00640 1 -- 172.21.15.175:0/2093925835 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784030850 con 0x7ff7940810a0 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 -- 172.21.15.175:0/2093925835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 msgr2=0x7ff7940814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 --2- 172.21.15.175:0/2093925835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7940814a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff784009820 tx=0x7ff78402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 -- 172.21.15.175:0/2093925835 shutdown_connections 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 --2- 172.21.15.175:0/2093925835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7940814a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 -- 172.21.15.175:0/2093925835 >> 172.21.15.175:0/2093925835 conn(0x7ff79407ca70 msgr2=0x7ff79407ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 -- 172.21.15.175:0/2093925835 shutdown_connections 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.920+0000 7ff799000640 1 -- 172.21.15.175:0/2093925835 wait complete. 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.921+0000 7ff799000640 1 Processor -- start 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.922+0000 7ff799000640 1 -- start start 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.922+0000 7ff799000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7941a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.922+0000 7ff799000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff794111c60 con 0x7ff7940810a0 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.923+0000 7ff793400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7941a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.923+0000 7ff793400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7941a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58864/0 (socket says 172.21.15.175:58864) 2024-07-29T00:26:11.848 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.923+0000 7ff793400640 1 -- 172.21.15.175:0/1495654381 learned_addr learned my addr 172.21.15.175:0/1495654381 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.923+0000 7ff793400640 1 -- 172.21.15.175:0/1495654381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7941a1cb0 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.923+0000 7ff793400640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7941a1770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff7840097f0 tx=0x7ff784030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.924+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784037040 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.924+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff784040070 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.924+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7941a1f40 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.924+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff78403a390 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.924+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a4c30 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.925+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7ff78403b430 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.925+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff794081520 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.926+0000 7ff790c00640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7ff778038e60 0x7ff77803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.926+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff78406bec0 con 0x7ff7940810a0 2024-07-29T00:26:11.849 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.926+0000 7ff78aa00640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7ff778038e60 0x7ff77803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.927+0000 7ff78aa00640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7ff778038e60 0x7ff77803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098c0 tx=0x7ff77c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:10.932+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff7840371e0 con 0x7ff7940810a0 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.046+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff784070070 con 0x7ff7940810a0 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.188+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff7941a23c0 con 0x7ff7940810a0 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.190+0000 7ff790c00640 1 -- 172.21.15.175:0/1495654381 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff7840392f0 con 0x7ff7940810a0 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.193+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7ff778038e60 msgr2=0x7ff77803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.193+0000 7ff799000640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7ff778038e60 0x7ff77803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098c0 tx=0x7ff77c006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 msgr2=0x7ff7941a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7941a1770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff7840097f0 tx=0x7ff784030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 shutdown_connections 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7ff778038e60 0x7ff77803b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 --2- 172.21.15.175:0/1495654381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940810a0 0x7ff7941a1770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:11.850 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 >> 172.21.15.175:0/1495654381 conn(0x7ff79407ca70 msgr2=0x7ff79407d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:11.851 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 shutdown_connections 2024-07-29T00:26:11.851 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:11.194+0000 7ff799000640 1 -- 172.21.15.175:0/1495654381 wait complete. 2024-07-29T00:26:11.851 INFO:teuthology.orchestra.run.smithi175.stdout:mgr is available 2024-07-29T00:26:12.142 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:12 smithi175 ceph-mon[32138]: mgrmap e4: x(active, since 2s) 2024-07-29T00:26:12.142 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:12 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1495654381' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-29T00:26:13.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:13 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/556994162' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-29T00:26:13.319 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [global] 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout fsid = f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.175:3300,v1:172.21.15.175:6789] 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [osd] 2024-07-29T00:26:13.320 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.451+0000 7f0a8f200640 1 Processor -- start 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.451+0000 7f0a8f200640 1 -- start start 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.451+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a901075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.451+0000 7f0a8f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90107ba0 con 0x7f0a901071d0 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.452+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a901075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.452+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a901075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58872/0 (socket says 172.21.15.175:58872) 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.452+0000 7f0a8de00640 1 -- 172.21.15.175:0/2687968874 learned_addr learned my addr 172.21.15.175:0/2687968874 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.452+0000 7f0a8de00640 1 -- 172.21.15.175:0/2687968874 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a901083d0 con 0x7f0a901071d0 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.453+0000 7f0a8de00640 1 --2- 172.21.15.175:0/2687968874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a901075d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009820 tx=0x7f0a8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25afe525e71b174b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:13.321 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.453+0000 7f0a8d400640 1 -- 172.21.15.175:0/2687968874 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80037070 con 0x7f0a901071d0 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.453+0000 7f0a8d400640 1 -- 172.21.15.175:0/2687968874 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a80032900 con 0x7f0a901071d0 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.453+0000 7f0a8d400640 1 -- 172.21.15.175:0/2687968874 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80030850 con 0x7f0a901071d0 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.454+0000 7f0a8f200640 1 -- 172.21.15.175:0/2687968874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 msgr2=0x7f0a901075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.454+0000 7f0a8f200640 1 --2- 172.21.15.175:0/2687968874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a901075d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009820 tx=0x7f0a8002a270 comp rx=0 tx=0).stop 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.454+0000 7f0a8f200640 1 -- 172.21.15.175:0/2687968874 shutdown_connections 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.454+0000 7f0a8f200640 1 --2- 172.21.15.175:0/2687968874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a901075d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.454+0000 7f0a8f200640 1 -- 172.21.15.175:0/2687968874 >> 172.21.15.175:0/2687968874 conn(0x7f0a901029a0 msgr2=0x7f0a90104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.455+0000 7f0a8f200640 1 -- 172.21.15.175:0/2687968874 shutdown_connections 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.455+0000 7f0a8f200640 1 -- 172.21.15.175:0/2687968874 wait complete. 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.455+0000 7f0a8f200640 1 Processor -- start 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.456+0000 7f0a8f200640 1 -- start start 2024-07-29T00:26:13.322 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.456+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a9019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.456+0000 7f0a8f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010b780 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.456+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a9019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.456+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a9019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58888/0 (socket says 172.21.15.175:58888) 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.456+0000 7f0a8de00640 1 -- 172.21.15.175:0/556994162 learned_addr learned my addr 172.21.15.175:0/556994162 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a8de00640 1 -- 172.21.15.175:0/556994162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9019b6b0 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a8de00640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a9019b170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a800097f0 tx=0x7f0a80030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80037040 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a80040070 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a8003a390 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9019b940 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.457+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9019e630 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.458+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0a80032900 con 0x7f0a901071d0 2024-07-29T00:26:13.323 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.459+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90107650 con 0x7f0a901071d0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.459+0000 7f0a86a00640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f0a74038f70 0x7f0a7403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.459+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a8006c070 con 0x7f0a901071d0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.459+0000 7f0a87e00640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f0a74038f70 0x7f0a7403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.460+0000 7f0a87e00640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f0a74038f70 0x7f0a7403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a780098c0 tx=0x7f0a78006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.465+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0a8006bdd0 con 0x7f0a901071d0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.653+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0a9019bdc0 con 0x7f0a901071d0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.655+0000 7f0a86a00640 1 -- 172.21.15.175:0/556994162 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f0a80032bf0 con 0x7f0a901071d0 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f0a74038f70 msgr2=0x7f0a7403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f0a74038f70 0x7f0a7403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a780098c0 tx=0x7f0a78006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 msgr2=0x7f0a9019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a9019b170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a800097f0 tx=0x7f0a80030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 shutdown_connections 2024-07-29T00:26:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f0a74038f70 0x7f0a7403b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:13.325 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 --2- 172.21.15.175:0/556994162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a901071d0 0x7f0a9019b170 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:13.325 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.658+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 >> 172.21.15.175:0/556994162 conn(0x7f0a901029a0 msgr2=0x7f0a901034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:13.325 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.659+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 shutdown_connections 2024-07-29T00:26:13.325 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:12.659+0000 7f0a8f200640 1 -- 172.21.15.175:0/556994162 wait complete. 2024-07-29T00:26:13.325 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling cephadm module... 2024-07-29T00:26:14.470 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:14 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1050927580' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-29T00:26:14.471 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ignoring --setuser ceph since I am not root 2024-07-29T00:26:14.471 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ignoring --setgroup ceph since I am not root 2024-07-29T00:26:14.471 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:14.469+0000 7ff3c0409040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:26:14.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:14.642+0000 7ff3c0409040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:26:14.736 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:14.733+0000 7ff3c0409040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.914+0000 7f1587200640 1 Processor -- start 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.914+0000 7f1587200640 1 -- start start 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.915+0000 7f1587200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f15881075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.915+0000 7f1587200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1588107bc0 con 0x7f15881071f0 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.915+0000 7f1585e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f15881075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.915+0000 7f1585e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f15881075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58904/0 (socket says 172.21.15.175:58904) 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.915+0000 7f1585e00640 1 -- 172.21.15.175:0/1848511858 learned_addr learned my addr 172.21.15.175:0/1848511858 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.916+0000 7f1585e00640 1 -- 172.21.15.175:0/1848511858 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15881083f0 con 0x7f15881071f0 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.917+0000 7f1585e00640 1 --2- 172.21.15.175:0/1848511858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f15881075f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1578009820 tx=0x7f157802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348c8c839c88a1b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.917+0000 7f1584a00640 1 -- 172.21.15.175:0/1848511858 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1578037070 con 0x7f15881071f0 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.917+0000 7f1584a00640 1 -- 172.21.15.175:0/1848511858 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1578032900 con 0x7f15881071f0 2024-07-29T00:26:14.965 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.917+0000 7f1584a00640 1 -- 172.21.15.175:0/1848511858 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1578030850 con 0x7f15881071f0 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 -- 172.21.15.175:0/1848511858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 msgr2=0x7f15881075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 --2- 172.21.15.175:0/1848511858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f15881075f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1578009820 tx=0x7f157802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 -- 172.21.15.175:0/1848511858 shutdown_connections 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 --2- 172.21.15.175:0/1848511858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f15881075f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 -- 172.21.15.175:0/1848511858 >> 172.21.15.175:0/1848511858 conn(0x7f1588102980 msgr2=0x7f1588104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 -- 172.21.15.175:0/1848511858 shutdown_connections 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.918+0000 7f1587200640 1 -- 172.21.15.175:0/1848511858 wait complete. 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.919+0000 7f1587200640 1 Processor -- start 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.919+0000 7f1587200640 1 -- start start 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.920+0000 7f1587200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f158819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.920+0000 7f1587200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f158810b7a0 con 0x7f15881071f0 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.920+0000 7f1585e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f158819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:14.966 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.920+0000 7f1585e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f158819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58918/0 (socket says 172.21.15.175:58918) 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.920+0000 7f1585e00640 1 -- 172.21.15.175:0/1050927580 learned_addr learned my addr 172.21.15.175:0/1050927580 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.920+0000 7f1585e00640 1 -- 172.21.15.175:0/1050927580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158819d8f0 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.921+0000 7f1585e00640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f158819d3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15780097f0 tx=0x7f1578030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.921+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1578037040 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.921+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1578040070 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.921+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158819db80 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.921+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f157803a390 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.921+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15881a0870 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.922+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f157803b430 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.922+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1588107670 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.923+0000 7f157ea00640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f156c038f20 0x7f156c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.923+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f157806c110 con 0x7f15881071f0 2024-07-29T00:26:14.967 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.923+0000 7f1585400640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f156c038f20 0x7f156c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.924+0000 7f1585400640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f156c038f20 0x7f156c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15700098c0 tx=0x7f1570006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:13.928+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f15780371e0 con 0x7f15881071f0 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.166+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f158819e000 con 0x7f15881071f0 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.220+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1578032dd0 con 0x7f15881071f0 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f157ea00640 1 -- 172.21.15.175:0/1050927580 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f157804acf0 con 0x7f15881071f0 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f156c038f20 msgr2=0x7f156c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f156c038f20 0x7f156c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15700098c0 tx=0x7f1570006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 msgr2=0x7f158819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f158819d3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15780097f0 tx=0x7f1578030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 shutdown_connections 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f156c038f20 0x7f156c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 --2- 172.21.15.175:0/1050927580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15881071f0 0x7f158819d3b0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15780097f0 tx=0x7f1578030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:14.968 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.224+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 >> 172.21.15.175:0/1050927580 conn(0x7f1588102980 msgr2=0x7f15881034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:14.969 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.225+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 shutdown_connections 2024-07-29T00:26:14.969 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:14.225+0000 7f1587200640 1 -- 172.21.15.175:0/1050927580 wait complete. 2024-07-29T00:26:15.465 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:15 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1050927580' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-29T00:26:15.466 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:15 smithi175 ceph-mon[32138]: mgrmap e5: x(active, since 5s) 2024-07-29T00:26:15.466 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:15.053+0000 7ff3c0409040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:26:15.466 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:15.125+0000 7ff3c0409040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:26:16.502 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/790695144' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-29T00:26:16.502 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:16.364+0000 7ff3c0409040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:26:16.502 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:16.501+0000 7ff3c0409040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:26:16.680 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:16.680 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-29T00:26:16.680 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-29T00:26:16.680 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-29T00:26:16.680 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-29T00:26:16.680 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.713+0000 7f58bb400640 1 Processor -- start 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.713+0000 7f58bb400640 1 -- start start 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.713+0000 7f58bb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b410a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.713+0000 7f58bb400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4075600 con 0x7f58b4107f10 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.714+0000 7f58b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b410a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.714+0000 7f58b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b410a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58944/0 (socket says 172.21.15.175:58944) 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.714+0000 7f58b8a00640 1 -- 172.21.15.175:0/2846497917 learned_addr learned my addr 172.21.15.175:0/2846497917 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.714+0000 7f58b8a00640 1 -- 172.21.15.175:0/2846497917 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b410a840 con 0x7f58b4107f10 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.715+0000 7f58b8a00640 1 --2- 172.21.15.175:0/2846497917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b410a300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009820 tx=0x7f58a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c77c6b0054ce452 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.716+0000 7f58b3400640 1 -- 172.21.15.175:0/2846497917 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58a8037070 con 0x7f58b4107f10 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.716+0000 7f58b3400640 1 -- 172.21.15.175:0/2846497917 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58a8032900 con 0x7f58b4107f10 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.716+0000 7f58b3400640 1 -- 172.21.15.175:0/2846497917 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58a8030850 con 0x7f58b4107f10 2024-07-29T00:26:16.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 -- 172.21.15.175:0/2846497917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 msgr2=0x7f58b410a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 --2- 172.21.15.175:0/2846497917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b410a300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009820 tx=0x7f58a802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 -- 172.21.15.175:0/2846497917 shutdown_connections 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 --2- 172.21.15.175:0/2846497917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b410a300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 -- 172.21.15.175:0/2846497917 >> 172.21.15.175:0/2846497917 conn(0x7f58b4101ab0 msgr2=0x7f58b4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 -- 172.21.15.175:0/2846497917 shutdown_connections 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.717+0000 7f58bb400640 1 -- 172.21.15.175:0/2846497917 wait complete. 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.718+0000 7f58bb400640 1 Processor -- start 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.718+0000 7f58bb400640 1 -- start start 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.719+0000 7f58bb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.719+0000 7f58bb400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410b820 con 0x7f58b4107f10 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.719+0000 7f58b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.719+0000 7f58b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58956/0 (socket says 172.21.15.175:58956) 2024-07-29T00:26:16.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.719+0000 7f58b8a00640 1 -- 172.21.15.175:0/790695144 learned_addr learned my addr 172.21.15.175:0/790695144 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.719+0000 7f58b8a00640 1 -- 172.21.15.175:0/790695144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b419dab0 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.720+0000 7f58b8a00640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b419d570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58a80097f0 tx=0x7f58a8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.720+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58a8037040 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.720+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58a8040070 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.720+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b419dd40 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.720+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58a803a390 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.720+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41a0a30 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.721+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b4108370 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.721+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f58a803b430 con 0x7f58b4107f10 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.722+0000 7f58b1600640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 0x7f589c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:16.686 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.722+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f58a806c140 con 0x7f58b4107f10 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.722+0000 7f58b3e00640 1 -- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 msgr2=0x7f589c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2801743500 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.722+0000 7f58b3e00640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 0x7f589c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.727+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f58a806bea0 con 0x7f58b4107f10 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.923+0000 7f58b3e00640 1 -- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 msgr2=0x7f589c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2801743500 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.923+0000 7f58b3e00640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 0x7f589c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.965+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f58b41a1000 con 0x7f58b4107f10 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.966+0000 7f58b1600640 1 -- 172.21.15.175:0/790695144 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f58a8032900 con 0x7f58b4107f10 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.969+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 msgr2=0x7f589c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.969+0000 7f58bb400640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 0x7f589c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.969+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 msgr2=0x7f58b419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.969+0000 7f58bb400640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b419d570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58a80097f0 tx=0x7f58a8030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.970+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 shutdown_connections 2024-07-29T00:26:16.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.970+0000 7f58bb400640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7f589c038fc0 0x7f589c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:16.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.970+0000 7f58bb400640 1 --2- 172.21.15.175:0/790695144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b4107f10 0x7f58b419d570 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:16.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.970+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 >> 172.21.15.175:0/790695144 conn(0x7f58b4101ab0 msgr2=0x7f58b4102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:16.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.970+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 shutdown_connections 2024-07-29T00:26:16.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:15.970+0000 7f58bb400640 1 -- 172.21.15.175:0/790695144 wait complete. 2024-07-29T00:26:16.688 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for the mgr to restart... 2024-07-29T00:26:16.688 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr epoch 5... 2024-07-29T00:26:16.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:16.569+0000 7ff3c0409040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:26:17.162 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:16.834+0000 7ff3c0409040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:26:17.163 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:16.907+0000 7ff3c0409040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:26:17.163 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:16.987+0000 7ff3c0409040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:26:17.494 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:17.163+0000 7ff3c0409040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:26:17.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:17.688+0000 7ff3c0409040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:26:18.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:18.042+0000 7ff3c0409040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:26:18.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:18.120+0000 7ff3c0409040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:26:18.646 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:18.287+0000 7ff3c0409040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:26:18.935 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:18.645+0000 7ff3c0409040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:26:18.935 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:18.718+0000 7ff3c0409040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:26:18.935 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:18.934+0000 7ff3c0409040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:26:19.190 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:19.115+0000 7ff3c0409040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:26:19.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:19.189+0000 7ff3c0409040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:26:19.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:19.284+0000 7ff3c0409040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:26:19.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:19.354+0000 7ff3c0409040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:26:20.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:19.958+0000 7ff3c0409040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:26:20.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:20.136+0000 7ff3c0409040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:26:20.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: Active manager daemon x restarted 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: Activating manager daemon x 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: osdmap e2: 0 total, 0 up, 0 in 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: mgrmap e6: x(active, starting, since 0.00972253s) 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: Manager daemon x is now available 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:20.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:20 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:26:21.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: Found migration_current of "None". Setting to last migration. 2024-07-29T00:26:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:26:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:26:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:26:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:21 smithi175 ceph-mon[32138]: mgrmap e7: x(active, since 1.01692s) 2024-07-29T00:26:21.895 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.283+0000 7fe435400640 1 Processor -- start 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.283+0000 7fe435400640 1 -- start start 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.284+0000 7fe435400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43007b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.284+0000 7fe435400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe43007b710 con 0x7fe43007cd70 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.284+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43007b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.284+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43007b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58958/0 (socket says 172.21.15.175:58958) 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.284+0000 7fe42fe00640 1 -- 172.21.15.175:0/3894591400 learned_addr learned my addr 172.21.15.175:0/3894591400 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.285+0000 7fe42fe00640 1 -- 172.21.15.175:0/3894591400 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43007b890 con 0x7fe43007cd70 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.286+0000 7fe42fe00640 1 --2- 172.21.15.175:0/3894591400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43007b1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe420009820 tx=0x7fe42002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a5b4b92164562a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:21.896 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.286+0000 7fe42ea00640 1 -- 172.21.15.175:0/3894591400 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420037070 con 0x7fe43007cd70 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.286+0000 7fe42ea00640 1 -- 172.21.15.175:0/3894591400 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe420032900 con 0x7fe43007cd70 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.286+0000 7fe42ea00640 1 -- 172.21.15.175:0/3894591400 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420030850 con 0x7fe43007cd70 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 -- 172.21.15.175:0/3894591400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 msgr2=0x7fe43007b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 --2- 172.21.15.175:0/3894591400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43007b1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe420009820 tx=0x7fe42002a270 comp rx=0 tx=0).stop 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 -- 172.21.15.175:0/3894591400 shutdown_connections 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 --2- 172.21.15.175:0/3894591400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43007b1d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 -- 172.21.15.175:0/3894591400 >> 172.21.15.175:0/3894591400 conn(0x7fe430102980 msgr2=0x7fe430104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 -- 172.21.15.175:0/3894591400 shutdown_connections 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.287+0000 7fe435400640 1 -- 172.21.15.175:0/3894591400 wait complete. 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.288+0000 7fe435400640 1 Processor -- start 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.289+0000 7fe435400640 1 -- start start 2024-07-29T00:26:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.289+0000 7fe435400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.289+0000 7fe435400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010d840 con 0x7fe43007cd70 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.289+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.289+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43019d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58972/0 (socket says 172.21.15.175:58972) 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.289+0000 7fe42fe00640 1 -- 172.21.15.175:0/651432696 learned_addr learned my addr 172.21.15.175:0/651432696 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe42fe00640 1 -- 172.21.15.175:0/651432696 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43019d8c0 con 0x7fe43007cd70 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe42fe00640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43019d380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe420032ce0 tx=0x7fe420032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420042070 con 0x7fe43007cd70 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe4200308c0 con 0x7fe43007cd70 2024-07-29T00:26:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43019db50 con 0x7fe43007cd70 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420037040 con 0x7fe43007cd70 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.290+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301a0840 con 0x7fe43007cd70 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.292+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fe420045070 con 0x7fe43007cd70 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.292+0000 7fe42cc00640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.292+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 --> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe41403bb90 con 0x7fe414038fc0 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.292+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe42006c020 con 0x7fe43007cd70 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.292+0000 7fe42f400640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 msgr2=0x7fe41403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2801743500 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.292+0000 7fe42f400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.493+0000 7fe42f400640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 msgr2=0x7fe41403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2801743500 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.493+0000 7fe42f400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.894+0000 7fe42f400640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 msgr2=0x7fe41403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2801743500 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:17.894+0000 7fe42f400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-29T00:26:21.899 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:18.695+0000 7fe42f400640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 msgr2=0x7fe41403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2801743500 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:18.695+0000 7fe42f400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:20.146+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fe42006aae0 con 0x7fe43007cd70 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:20.146+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 msgr2=0x7fe41403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:20.146+0000 7fe42cc00640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.151+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fe42003c780 con 0x7fe43007cd70 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.151+0000 7fe42cc00640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fe41403c8f0 0x7fe41403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.151+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe42003aca0 con 0x7fe41403c8f0 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.153+0000 7fe42f400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fe41403c8f0 0x7fe41403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.158+0000 7fe42f400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fe41403c8f0 0x7fe41403ece0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe41c003980 tx=0x7fe41c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.160+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe42003aca0 con 0x7fe41403c8f0 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.168+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe43007b250 con 0x7fe41403c8f0 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.168+0000 7fe42cc00640 1 -- 172.21.15.175:0/651432696 <== mgr.14118 v2:172.21.15.175:6800/61876509 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe43007b250 con 0x7fe41403c8f0 2024-07-29T00:26:21.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.169+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fe41403c8f0 msgr2=0x7fe41403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.169+0000 7fe435400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fe41403c8f0 0x7fe41403ece0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe41c003980 tx=0x7fe41c0092b0 comp rx=0 tx=0).stop 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.169+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 msgr2=0x7fe43019d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.169+0000 7fe435400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43019d380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe420032ce0 tx=0x7fe420032d10 comp rx=0 tx=0).stop 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.169+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 shutdown_connections 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.170+0000 7fe435400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fe41403c8f0 0x7fe41403ece0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.170+0000 7fe435400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:6800/2801743500,v1:172.21.15.175:6801/2801743500] conn(0x7fe414038fc0 0x7fe41403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.170+0000 7fe435400640 1 --2- 172.21.15.175:0/651432696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43007cd70 0x7fe43019d380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.170+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 >> 172.21.15.175:0/651432696 conn(0x7fe430102980 msgr2=0x7fe430103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.170+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 shutdown_connections 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:21.170+0000 7fe435400640 1 -- 172.21.15.175:0/651432696 wait complete. 2024-07-29T00:26:21.901 INFO:teuthology.orchestra.run.smithi175.stdout:mgr epoch 5 is available 2024-07-29T00:26:21.902 INFO:teuthology.orchestra.run.smithi175.stdout:Setting orchestrator backend to cephadm... 2024-07-29T00:26:22.360 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-mon[32138]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-29T00:26:22.360 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-mon[32138]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-29T00:26:23.032 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:22.899+0000 7ff31ec00640 -1 log_channel(cephadm) log [ERR] : [29/Jul/2024:00:26:22] ENGINE Error in HTTPServer.serve 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: Traceback (most recent call last): 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._connections.run(self.expiration_interval) 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._run(expiration_interval) 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: s = self.context.wrap_socket( 2024-07-29T00:26:23.033 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: return self.sslsocket_class._create( 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self.do_handshake() 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._sslobj.do_handshake() 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:26:23.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:23.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:23 smithi175 ceph-mon[32138]: mgrmap e8: x(active, since 2s) 2024-07-29T00:26:23.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:23 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:23.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:23 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:26:23.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:23 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:26:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.481+0000 7f3be9c00640 1 Processor -- start 2024-07-29T00:26:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.481+0000 7f3be9c00640 1 -- start start 2024-07-29T00:26:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.481+0000 7f3be9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be410a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.481+0000 7f3be9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3be407be50 con 0x7f3be4107ea0 2024-07-29T00:26:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.482+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be410a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.482+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be410a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56148/0 (socket says 172.21.15.175:56148) 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.482+0000 7f3be2a00640 1 -- 172.21.15.175:0/3117620134 learned_addr learned my addr 172.21.15.175:0/3117620134 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.483+0000 7f3be2a00640 1 -- 172.21.15.175:0/3117620134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be410a7d0 con 0x7f3be4107ea0 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.484+0000 7f3be2a00640 1 --2- 172.21.15.175:0/3117620134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be410a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009820 tx=0x7f3bd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ba912f003ac597 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.484+0000 7f3be1600640 1 -- 172.21.15.175:0/3117620134 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4037070 con 0x7f3be4107ea0 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.484+0000 7f3be1600640 1 -- 172.21.15.175:0/3117620134 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3bd4032900 con 0x7f3be4107ea0 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.484+0000 7f3be1600640 1 -- 172.21.15.175:0/3117620134 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4030850 con 0x7f3be4107ea0 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 -- 172.21.15.175:0/3117620134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 msgr2=0x7f3be410a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 --2- 172.21.15.175:0/3117620134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be410a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009820 tx=0x7f3bd402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 -- 172.21.15.175:0/3117620134 shutdown_connections 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 --2- 172.21.15.175:0/3117620134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be410a290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 -- 172.21.15.175:0/3117620134 >> 172.21.15.175:0/3117620134 conn(0x7f3be4101ac0 msgr2=0x7f3be4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 -- 172.21.15.175:0/3117620134 shutdown_connections 2024-07-29T00:26:23.391 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.485+0000 7f3be9c00640 1 -- 172.21.15.175:0/3117620134 wait complete. 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.486+0000 7f3be9c00640 1 Processor -- start 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.486+0000 7f3be9c00640 1 -- start start 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.486+0000 7f3be9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.486+0000 7f3be9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3be410b7b0 con 0x7f3be4107ea0 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.487+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.487+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be419d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56158/0 (socket says 172.21.15.175:56158) 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.487+0000 7f3be2a00640 1 -- 172.21.15.175:0/3362909355 learned_addr learned my addr 172.21.15.175:0/3362909355 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.487+0000 7f3be2a00640 1 -- 172.21.15.175:0/3362909355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be419da80 con 0x7f3be4107ea0 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.488+0000 7f3be2a00640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be419d540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40097f0 tx=0x7f3bd4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.488+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4037040 con 0x7f3be4107ea0 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.488+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3bd4040070 con 0x7f3be4107ea0 2024-07-29T00:26:23.392 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.488+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd403a390 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.488+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be419dd10 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.488+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41a0a00 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.489+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be4108300 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.490+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3bd403b430 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.490+0000 7f3bdb400640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f3bc8038fc0 0x7f3bc803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.490+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3bd406c510 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.491+0000 7f3be2000640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f3bc8038fc0 0x7f3bc803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.491+0000 7f3be2000640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f3bc8038fc0 0x7f3bc803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098c0 tx=0x7f3bd0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.496+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3bd406be60 con 0x7f3be4107ea0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.697+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3be4102c40 con 0x7f3bc8038fc0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.705+0000 7f3bdb400640 1 -- 172.21.15.175:0/3362909355 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3be4102c40 con 0x7f3bc8038fc0 2024-07-29T00:26:23.393 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.707+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f3bc8038fc0 msgr2=0x7f3bc803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.707+0000 7f3be9c00640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f3bc8038fc0 0x7f3bc803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098c0 tx=0x7f3bd0006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 msgr2=0x7f3be419d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be419d540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40097f0 tx=0x7f3bd4030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 shutdown_connections 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f3bc8038fc0 0x7f3bc803b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 --2- 172.21.15.175:0/3362909355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4107ea0 0x7f3be419d540 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 >> 172.21.15.175:0/3362909355 conn(0x7f3be4101ac0 msgr2=0x7f3be4102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 shutdown_connections 2024-07-29T00:26:23.394 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:22.708+0000 7f3be9c00640 1 -- 172.21.15.175:0/3362909355 wait complete. 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:24 smithi175 ceph-mon[32138]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:24 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:22] ENGINE Bus STARTING 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:24 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:22] ENGINE Serving on https://172.21.15.175:7150 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:24 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:22] ENGINE Error in HTTPServer.serve 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout: Traceback (most recent call last): 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._connections.run(self.expiration_interval) 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._run(expiration_interval) 2024-07-29T00:26:24.285 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: s = self.context.wrap_socket( 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: return self.sslsocket_class._create( 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: self.do_handshake() 2024-07-29T00:26:24.286 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:26:24.287 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._sslobj.do_handshake() 2024-07-29T00:26:24.287 INFO:journalctl@ceph.mon.a.smithi175.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:26:24.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:24 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:22] ENGINE Serving on http://172.21.15.175:8765 2024-07-29T00:26:24.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:24 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:22] ENGINE Bus STARTED 2024-07-29T00:26:24.875 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-29T00:26:24.877 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.960+0000 7ff16d600640 1 Processor -- start 2024-07-29T00:26:24.877 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.960+0000 7ff16d600640 1 -- start start 2024-07-29T00:26:24.877 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.960+0000 7ff16d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1680814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:24.877 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.960+0000 7ff16d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff168081a70 con 0x7ff1680810a0 2024-07-29T00:26:24.877 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.961+0000 7ff167e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1680814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:24.877 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.961+0000 7ff167e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1680814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56162/0 (socket says 172.21.15.175:56162) 2024-07-29T00:26:24.879 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.961+0000 7ff167e00640 1 -- 172.21.15.175:0/3824937084 learned_addr learned my addr 172.21.15.175:0/3824937084 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.961+0000 7ff167e00640 1 -- 172.21.15.175:0/3824937084 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff168081bf0 con 0x7ff1680810a0 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.962+0000 7ff167e00640 1 --2- 172.21.15.175:0/3824937084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1680814a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff158009820 tx=0x7ff15802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597fefd8d6b8c221 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.962+0000 7ff166a00640 1 -- 172.21.15.175:0/3824937084 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158037070 con 0x7ff1680810a0 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff166a00640 1 -- 172.21.15.175:0/3824937084 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff158032900 con 0x7ff1680810a0 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff166a00640 1 -- 172.21.15.175:0/3824937084 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158030850 con 0x7ff1680810a0 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 -- 172.21.15.175:0/3824937084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 msgr2=0x7ff1680814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 --2- 172.21.15.175:0/3824937084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1680814a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff158009820 tx=0x7ff15802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 -- 172.21.15.175:0/3824937084 shutdown_connections 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 --2- 172.21.15.175:0/3824937084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1680814a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 -- 172.21.15.175:0/3824937084 >> 172.21.15.175:0/3824937084 conn(0x7ff16807ca70 msgr2=0x7ff16807ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 -- 172.21.15.175:0/3824937084 shutdown_connections 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.963+0000 7ff16d600640 1 -- 172.21.15.175:0/3824937084 wait complete. 2024-07-29T00:26:24.880 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.964+0000 7ff16d600640 1 Processor -- start 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff16d600640 1 -- start start 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff16d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1681a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff16d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff168111c60 con 0x7ff1680810a0 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff167e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1681a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff167e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1681a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56164/0 (socket says 172.21.15.175:56164) 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff167e00640 1 -- 172.21.15.175:0/3001046216 learned_addr learned my addr 172.21.15.175:0/3001046216 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.965+0000 7ff167e00640 1 -- 172.21.15.175:0/3001046216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1681a1ce0 con 0x7ff1680810a0 2024-07-29T00:26:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.966+0000 7ff167e00640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1681a17a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff158009940 tx=0x7ff158032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:24.884 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.966+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158042070 con 0x7ff1680810a0 2024-07-29T00:26:24.884 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.966+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1580308c0 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.966+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158037040 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.966+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1681a1f70 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.966+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a4c60 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.967+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff168081520 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.967+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff158045070 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.968+0000 7ff164c00640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff14c038fc0 0x7ff14c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.968+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff15806c090 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.968+0000 7ff167400640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff14c038fc0 0x7ff14c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.969+0000 7ff167400640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff14c038fc0 0x7ff14c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1500098c0 tx=0x7ff150006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:23.973+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff15804b170 con 0x7ff1680810a0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.166+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff16807dcd0 con 0x7ff14c038fc0 2024-07-29T00:26:24.885 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.167+0000 7ff164c00640 1 -- 172.21.15.175:0/3001046216 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff16807dcd0 con 0x7ff14c038fc0 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.170+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff14c038fc0 msgr2=0x7ff14c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.170+0000 7ff16d600640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff14c038fc0 0x7ff14c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1500098c0 tx=0x7ff150006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.170+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 msgr2=0x7ff1681a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.170+0000 7ff16d600640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1681a17a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff158009940 tx=0x7ff158032ce0 comp rx=0 tx=0).stop 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.171+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 shutdown_connections 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.171+0000 7ff16d600640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff14c038fc0 0x7ff14c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.171+0000 7ff16d600640 1 --2- 172.21.15.175:0/3001046216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1680810a0 0x7ff1681a17a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.171+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 >> 172.21.15.175:0/3001046216 conn(0x7ff16807ca70 msgr2=0x7ff16807d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.171+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 shutdown_connections 2024-07-29T00:26:24.886 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:24.171+0000 7ff16d600640 1 -- 172.21.15.175:0/3001046216 wait complete. 2024-07-29T00:26:24.887 INFO:teuthology.orchestra.run.smithi175.stdout:Generating ssh key... 2024-07-29T00:26:25.438 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-mon[32138]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: Generating public/private rsa key pair. 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: Your identification has been saved in /tmp/tmpfciuzmr5/key 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: Your public key has been saved in /tmp/tmpfciuzmr5/key.pub 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: The key fingerprint is: 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: SHA256:f5+yZ0coLquKMeTVvD79Q7m8aPPfPjpBjOsrfeD4C44 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: The key's randomart image is: 2024-07-29T00:26:26.264 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: +---[RSA 3072]----+ 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | | 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | | 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | o | 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | o . o | 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | . . S o. . | 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | o . o o+.. .| 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | + .o*+oo.. | 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | + .+o***+o+.| 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: | . .Eo==XOX*o+| 2024-07-29T00:26:26.265 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: +----[SHA256]-----+ 2024-07-29T00:26:26.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.438+0000 7f1962800640 1 Processor -- start 2024-07-29T00:26:26.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.438+0000 7f1962800640 1 -- start start 2024-07-29T00:26:26.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.438+0000 7f1962800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c0814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:26.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.438+0000 7f1962800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f195c081a70 con 0x7f195c0810a0 2024-07-29T00:26:26.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.439+0000 7f1961400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c0814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.439+0000 7f1961400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c0814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56178/0 (socket says 172.21.15.175:56178) 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.439+0000 7f1961400640 1 -- 172.21.15.175:0/487318967 learned_addr learned my addr 172.21.15.175:0/487318967 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.439+0000 7f1961400640 1 -- 172.21.15.175:0/487318967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c081bf0 con 0x7f195c0810a0 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.440+0000 7f1961400640 1 --2- 172.21.15.175:0/487318967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c0814a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1950009820 tx=0x7f195002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc66f14fa36acf9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f195be00640 1 -- 172.21.15.175:0/487318967 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1950037070 con 0x7f195c0810a0 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f195be00640 1 -- 172.21.15.175:0/487318967 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1950032900 con 0x7f195c0810a0 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f195be00640 1 -- 172.21.15.175:0/487318967 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1950030850 con 0x7f195c0810a0 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f1962800640 1 -- 172.21.15.175:0/487318967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 msgr2=0x7f195c0814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f1962800640 1 --2- 172.21.15.175:0/487318967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c0814a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1950009820 tx=0x7f195002a270 comp rx=0 tx=0).stop 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f1962800640 1 -- 172.21.15.175:0/487318967 shutdown_connections 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f1962800640 1 --2- 172.21.15.175:0/487318967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c0814a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.441+0000 7f1962800640 1 -- 172.21.15.175:0/487318967 >> 172.21.15.175:0/487318967 conn(0x7f195c07ca70 msgr2=0x7f195c07ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:26.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.442+0000 7f1962800640 1 -- 172.21.15.175:0/487318967 shutdown_connections 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.442+0000 7f1962800640 1 -- 172.21.15.175:0/487318967 wait complete. 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.442+0000 7f1962800640 1 Processor -- start 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.443+0000 7f1962800640 1 -- start start 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.443+0000 7f1962800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.443+0000 7f1962800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f195c111c60 con 0x7f195c0810a0 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.443+0000 7f1961400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.443+0000 7f1961400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56184/0 (socket says 172.21.15.175:56184) 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.443+0000 7f1961400640 1 -- 172.21.15.175:0/4278296220 learned_addr learned my addr 172.21.15.175:0/4278296220 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f1961400640 1 -- 172.21.15.175:0/4278296220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c1a1ce0 con 0x7f195c0810a0 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f1961400640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c1a17a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19500097f0 tx=0x7f1950030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1950037040 con 0x7f195c0810a0 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1950040070 con 0x7f195c0810a0 2024-07-29T00:26:26.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f1962800640 1 -- 172.21.15.175:0/4278296220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c1a1f70 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f195003a390 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.444+0000 7f1962800640 1 -- 172.21.15.175:0/4278296220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c1a4c60 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.445+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1950032ab0 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.446+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c081520 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.446+0000 7f195a000640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1944038fc0 0x7f194403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.446+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f195006c900 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.446+0000 7f1960a00640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1944038fc0 0x7f194403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.447+0000 7f1960a00640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1944038fc0 0x7f194403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f19480098c0 tx=0x7f1948006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.451+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f195003b4e0 con 0x7f195c0810a0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.643+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f195c062b30 con 0x7f1944038fc0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.889+0000 7f195a000640 1 -- 172.21.15.175:0/4278296220 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f195c062b30 con 0x7f1944038fc0 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1944038fc0 msgr2=0x7f194403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1944038fc0 0x7f194403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f19480098c0 tx=0x7f1948006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:26.544 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 msgr2=0x7f195c1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c1a17a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19500097f0 tx=0x7f1950030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 shutdown_connections 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1944038fc0 0x7f194403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 --2- 172.21.15.175:0/4278296220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f195c0810a0 0x7f195c1a17a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 >> 172.21.15.175:0/4278296220 conn(0x7f195c07ca70 msgr2=0x7f195c07d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 shutdown_connections 2024-07-29T00:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:25.892+0000 7f194f400640 1 -- 172.21.15.175:0/4278296220 wait complete. 2024-07-29T00:26:27.113 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:26 smithi175 ceph-mon[32138]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:27.113 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:26 smithi175 ceph-mon[32138]: Generating ssh key... 2024-07-29T00:26:27.113 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:26 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:27.113 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:26 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:27.924 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:27 smithi175 ceph-mon[32138]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:27.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:27.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.110+0000 7fb3fcc00640 1 Processor -- start 2024-07-29T00:26:27.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.110+0000 7fb3fcc00640 1 -- start start 2024-07-29T00:26:27.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.110+0000 7fb3fcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f80814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:27.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.110+0000 7fb3fcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8081a70 con 0x7fb3f80810a0 2024-07-29T00:26:27.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.111+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f80814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:27.938 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.111+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f80814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56200/0 (socket says 172.21.15.175:56200) 2024-07-29T00:26:27.938 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.111+0000 7fb3f7400640 1 -- 172.21.15.175:0/3346670559 learned_addr learned my addr 172.21.15.175:0/3346670559 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:27.938 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.111+0000 7fb3f7400640 1 -- 172.21.15.175:0/3346670559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f8081bf0 con 0x7fb3f80810a0 2024-07-29T00:26:27.939 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.112+0000 7fb3f7400640 1 --2- 172.21.15.175:0/3346670559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f80814a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009820 tx=0x7fb3e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576629068a878fa0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:27.939 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.113+0000 7fb3f6a00640 1 -- 172.21.15.175:0/3346670559 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e8037070 con 0x7fb3f80810a0 2024-07-29T00:26:27.939 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.113+0000 7fb3f6a00640 1 -- 172.21.15.175:0/3346670559 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3e8032900 con 0x7fb3f80810a0 2024-07-29T00:26:27.939 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.113+0000 7fb3f6a00640 1 -- 172.21.15.175:0/3346670559 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e8030850 con 0x7fb3f80810a0 2024-07-29T00:26:27.939 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 -- 172.21.15.175:0/3346670559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 msgr2=0x7fb3f80814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:27.939 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 --2- 172.21.15.175:0/3346670559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f80814a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009820 tx=0x7fb3e802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 -- 172.21.15.175:0/3346670559 shutdown_connections 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 --2- 172.21.15.175:0/3346670559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f80814a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 -- 172.21.15.175:0/3346670559 >> 172.21.15.175:0/3346670559 conn(0x7fb3f807ca70 msgr2=0x7fb3f807ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 -- 172.21.15.175:0/3346670559 shutdown_connections 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.114+0000 7fb3fcc00640 1 -- 172.21.15.175:0/3346670559 wait complete. 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.115+0000 7fb3fcc00640 1 Processor -- start 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.115+0000 7fb3fcc00640 1 -- start start 2024-07-29T00:26:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.116+0000 7fb3fcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.116+0000 7fb3fcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8111c60 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.116+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.116+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f81a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56216/0 (socket says 172.21.15.175:56216) 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.116+0000 7fb3f7400640 1 -- 172.21.15.175:0/108376320 learned_addr learned my addr 172.21.15.175:0/108376320 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.116+0000 7fb3f7400640 1 -- 172.21.15.175:0/108376320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f81a1ce0 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.117+0000 7fb3f7400640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f81a17a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097f0 tx=0x7fb3e8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.117+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e8037040 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.117+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3e8040070 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.117+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f81a1f70 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.117+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3e803a390 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.117+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81a4c60 con 0x7fb3f80810a0 2024-07-29T00:26:27.944 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.118+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb3e803b430 con 0x7fb3f80810a0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.118+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f8081520 con 0x7fb3f80810a0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.119+0000 7fb3f4c00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fb3dc038fc0 0x7fb3dc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.119+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb3e806c4e0 con 0x7fb3f80810a0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.119+0000 7fb3eea00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fb3dc038fc0 0x7fb3dc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.120+0000 7fb3eea00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fb3dc038fc0 0x7fb3dc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00098c0 tx=0x7fb3e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.124+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb3e806be30 con 0x7fb3f80810a0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.314+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb3f807dcd0 con 0x7fb3dc038fc0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.316+0000 7fb3f4c00640 1 -- 172.21.15.175:0/108376320 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb3f807dcd0 con 0x7fb3dc038fc0 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.318+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fb3dc038fc0 msgr2=0x7fb3dc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.318+0000 7fb3fcc00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fb3dc038fc0 0x7fb3dc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00098c0 tx=0x7fb3e0006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.318+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 msgr2=0x7fb3f81a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.318+0000 7fb3fcc00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f81a17a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097f0 tx=0x7fb3e8030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.319+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 shutdown_connections 2024-07-29T00:26:27.946 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.319+0000 7fb3fcc00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7fb3dc038fc0 0x7fb3dc03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:27.946 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.319+0000 7fb3fcc00640 1 --2- 172.21.15.175:0/108376320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f80810a0 0x7fb3f81a17a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:27.946 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.319+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 >> 172.21.15.175:0/108376320 conn(0x7fb3f807ca70 msgr2=0x7fb3f807d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:27.946 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.319+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 shutdown_connections 2024-07-29T00:26:27.946 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:27.319+0000 7fb3fcc00640 1 -- 172.21.15.175:0/108376320 wait complete. 2024-07-29T00:26:27.946 INFO:teuthology.orchestra.run.smithi175.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-29T00:26:27.947 INFO:teuthology.orchestra.run.smithi175.stdout:Adding key to root@localhost authorized_keys... 2024-07-29T00:26:27.947 INFO:teuthology.orchestra.run.smithi175.stdout:Adding host smithi175... 2024-07-29T00:26:29.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:29 smithi175 ceph-mon[32138]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "addr": "172.21.15.175", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:30.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:30 smithi175 ceph-mon[32138]: Deploying cephadm binary to smithi175 2024-07-29T00:26:34.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Added host 'smithi175' with addr '172.21.15.175' 2024-07-29T00:26:34.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.520+0000 7f1cdd600640 1 Processor -- start 2024-07-29T00:26:34.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.520+0000 7f1cdd600640 1 -- start start 2024-07-29T00:26:34.248 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.521+0000 7f1cdd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd8109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:34.248 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.521+0000 7f1cdd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd810a260 con 0x7f1cd8109890 2024-07-29T00:26:34.248 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.521+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd8109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:34.248 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.521+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd8109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56222/0 (socket says 172.21.15.175:56222) 2024-07-29T00:26:34.248 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.521+0000 7f1cd7e00640 1 -- 172.21.15.175:0/4281639835 learned_addr learned my addr 172.21.15.175:0/4281639835 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:34.248 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.522+0000 7f1cd7e00640 1 -- 172.21.15.175:0/4281639835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd810a610 con 0x7f1cd8109890 2024-07-29T00:26:34.251 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.523+0000 7f1cd7e00640 1 --2- 172.21.15.175:0/4281639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd8109c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009820 tx=0x7f1cc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd33f0f0c590188 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:34.251 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.523+0000 7f1cd7400640 1 -- 172.21.15.175:0/4281639835 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc8037070 con 0x7f1cd8109890 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.523+0000 7f1cd7400640 1 -- 172.21.15.175:0/4281639835 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1cc8032900 con 0x7f1cd8109890 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.523+0000 7f1cd7400640 1 -- 172.21.15.175:0/4281639835 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc8030850 con 0x7f1cd8109890 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 -- 172.21.15.175:0/4281639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 msgr2=0x7f1cd8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 --2- 172.21.15.175:0/4281639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd8109c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009820 tx=0x7f1cc802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 -- 172.21.15.175:0/4281639835 shutdown_connections 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 --2- 172.21.15.175:0/4281639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd8109c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 -- 172.21.15.175:0/4281639835 >> 172.21.15.175:0/4281639835 conn(0x7f1cd807ca70 msgr2=0x7f1cd807cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 -- 172.21.15.175:0/4281639835 shutdown_connections 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.524+0000 7f1cdd600640 1 -- 172.21.15.175:0/4281639835 wait complete. 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.526+0000 7f1cdd600640 1 Processor -- start 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.526+0000 7f1cdd600640 1 -- start start 2024-07-29T00:26:34.252 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.526+0000 7f1cdd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.526+0000 7f1cdd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd810d9c0 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.527+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.527+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56224/0 (socket says 172.21.15.175:56224) 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.527+0000 7f1cd7e00640 1 -- 172.21.15.175:0/3777480968 learned_addr learned my addr 172.21.15.175:0/3777480968 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.527+0000 7f1cd7e00640 1 -- 172.21.15.175:0/3777480968 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd819d8f0 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.527+0000 7f1cd7e00640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd819d3b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8032ce0 tx=0x7f1cc8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.528+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc8042070 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.528+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd819db80 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.528+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1cc80308c0 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.528+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc8037040 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.528+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81a0870 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.529+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8109c90 con 0x7f1cd8109890 2024-07-29T00:26:34.253 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.529+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1cc8045070 con 0x7f1cd8109890 2024-07-29T00:26:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.530+0000 7f1cd5600640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1cbc038f70 0x7f1cbc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.530+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1cc806c040 con 0x7f1cd8109890 2024-07-29T00:26:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.530+0000 7f1ccf400640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1cbc038f70 0x7f1cbc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.531+0000 7f1ccf400640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1cbc038f70 0x7f1cbc03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00098c0 tx=0x7f1cc0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.535+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1cc806b990 con 0x7f1cd8109890 2024-07-29T00:26:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:28.731+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "addr": "172.21.15.175", "target": ["mon-mgr", ""]}) -- 0x7f1cd8107b90 con 0x7f1cbc038f70 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.564+0000 7f1cd5600640 1 -- 172.21.15.175:0/3777480968 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1cd8107b90 con 0x7f1cbc038f70 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1cbc038f70 msgr2=0x7f1cbc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1cbc038f70 0x7f1cbc03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00098c0 tx=0x7f1cc0006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 msgr2=0x7f1cd819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd819d3b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8032ce0 tx=0x7f1cc8032d10 comp rx=0 tx=0).stop 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 shutdown_connections 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f1cbc038f70 0x7f1cbc03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 --2- 172.21.15.175:0/3777480968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd8109890 0x7f1cd819d3b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.568+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 >> 172.21.15.175:0/3777480968 conn(0x7f1cd807ca70 msgr2=0x7f1cd8107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.569+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 shutdown_connections 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:33.569+0000 7f1cdd600640 1 -- 172.21.15.175:0/3777480968 wait complete. 2024-07-29T00:26:34.256 INFO:teuthology.orchestra.run.smithi175.stdout:Deploying unmanaged mon service... 2024-07-29T00:26:34.819 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:34 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:34.820 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:34 smithi175 ceph-mon[32138]: Added host smithi175 2024-07-29T00:26:34.820 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:34 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:26:35.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.850+0000 7f4d1f200640 1 Processor -- start 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.851+0000 7f4d1f200640 1 -- start start 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.851+0000 7f4d1f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d180814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.851+0000 7f4d1f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d18081a70 con 0x7f4d180810a0 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.852+0000 7f4d1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d180814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.852+0000 7f4d1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d180814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45224/0 (socket says 172.21.15.175:45224) 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.852+0000 7f4d1de00640 1 -- 172.21.15.175:0/409240455 learned_addr learned my addr 172.21.15.175:0/409240455 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.852+0000 7f4d1de00640 1 -- 172.21.15.175:0/409240455 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d18081bf0 con 0x7f4d180810a0 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.853+0000 7f4d1de00640 1 --2- 172.21.15.175:0/409240455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d180814a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009820 tx=0x7f4d0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f61e0a80877978 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.853+0000 7f4d1ca00640 1 -- 172.21.15.175:0/409240455 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d0c037070 con 0x7f4d180810a0 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1ca00640 1 -- 172.21.15.175:0/409240455 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d0c032900 con 0x7f4d180810a0 2024-07-29T00:26:35.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1ca00640 1 -- 172.21.15.175:0/409240455 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d0c030850 con 0x7f4d180810a0 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1f200640 1 -- 172.21.15.175:0/409240455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 msgr2=0x7f4d180814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1f200640 1 --2- 172.21.15.175:0/409240455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d180814a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009820 tx=0x7f4d0c02a270 comp rx=0 tx=0).stop 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1f200640 1 -- 172.21.15.175:0/409240455 shutdown_connections 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1f200640 1 --2- 172.21.15.175:0/409240455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d180814a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.854+0000 7f4d1f200640 1 -- 172.21.15.175:0/409240455 >> 172.21.15.175:0/409240455 conn(0x7f4d1807ca70 msgr2=0x7f4d1807ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.855+0000 7f4d1f200640 1 -- 172.21.15.175:0/409240455 shutdown_connections 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.855+0000 7f4d1f200640 1 -- 172.21.15.175:0/409240455 wait complete. 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.856+0000 7f4d1f200640 1 Processor -- start 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.856+0000 7f4d1f200640 1 -- start start 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.856+0000 7f4d1f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d181a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.856+0000 7f4d1f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d18111c60 con 0x7f4d180810a0 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.856+0000 7f4d1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d181a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:35.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.857+0000 7f4d1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d181a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45238/0 (socket says 172.21.15.175:45238) 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.857+0000 7f4d1de00640 1 -- 172.21.15.175:0/3315410784 learned_addr learned my addr 172.21.15.175:0/3315410784 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.857+0000 7f4d1de00640 1 -- 172.21.15.175:0/3315410784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d181a1ce0 con 0x7f4d180810a0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.857+0000 7f4d1de00640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d181a17a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0097f0 tx=0x7f4d0c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.858+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d0c037040 con 0x7f4d180810a0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.858+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d0c040070 con 0x7f4d180810a0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.858+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d181a1f70 con 0x7f4d180810a0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.858+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d0c03a390 con 0x7f4d180810a0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.858+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d181a4c60 con 0x7f4d180810a0 2024-07-29T00:26:35.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.859+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4d0c032900 con 0x7f4d180810a0 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.859+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d18081520 con 0x7f4d180810a0 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.859+0000 7f4d12a00640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f4d00038f70 0x7f4d0003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.859+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d0c06c540 con 0x7f4d180810a0 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.860+0000 7f4d1d400640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f4d00038f70 0x7f4d0003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.860+0000 7f4d1d400640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f4d00038f70 0x7f4d0003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d080098c0 tx=0x7f4d08006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:34.865+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4d0c06be90 con 0x7f4d180810a0 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.071+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4d1807dcd0 con 0x7f4d00038f70 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.081+0000 7f4d12a00640 1 -- 172.21.15.175:0/3315410784 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4d1807dcd0 con 0x7f4d00038f70 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f4d00038f70 msgr2=0x7f4d0003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f4d00038f70 0x7f4d0003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d080098c0 tx=0x7f4d08006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 msgr2=0x7f4d181a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:35.714 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d181a17a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0097f0 tx=0x7f4d0c030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 shutdown_connections 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f4d00038f70 0x7f4d0003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 --2- 172.21.15.175:0/3315410784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d180810a0 0x7f4d181a17a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 >> 172.21.15.175:0/3315410784 conn(0x7f4d1807ca70 msgr2=0x7f4d1807d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.084+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 shutdown_connections 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:35.085+0000 7f4d1f200640 1 -- 172.21.15.175:0/3315410784 wait complete. 2024-07-29T00:26:35.715 INFO:teuthology.orchestra.run.smithi175.stdout:Deploying unmanaged mgr service... 2024-07-29T00:26:36.317 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:36 smithi175 ceph-mon[32138]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:36.317 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:36 smithi175 ceph-mon[32138]: Saving service mon spec with placement count:5 2024-07-29T00:26:36.317 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:36 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:37.677 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:37 smithi175 ceph-mon[32138]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:37.677 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:37 smithi175 ceph-mon[32138]: Saving service mgr spec with placement count:2 2024-07-29T00:26:37.678 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:37 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.316+0000 7f88a3e00640 1 Processor -- start 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.316+0000 7f88a3e00640 1 -- start start 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.316+0000 7f88a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a4109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.316+0000 7f88a3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410a2b0 con 0x7f88a41098e0 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.317+0000 7f88a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a4109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.317+0000 7f88a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a4109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45248/0 (socket says 172.21.15.175:45248) 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.317+0000 7f88a2a00640 1 -- 172.21.15.175:0/3227240353 learned_addr learned my addr 172.21.15.175:0/3227240353 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:38.158 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.317+0000 7f88a2a00640 1 -- 172.21.15.175:0/3227240353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a410a5b0 con 0x7f88a41098e0 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.319+0000 7f88a2a00640 1 --2- 172.21.15.175:0/3227240353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a4109ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8894009820 tx=0x7f889402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482948ebfc4128d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.319+0000 7f88a2000640 1 -- 172.21.15.175:0/3227240353 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8894037070 con 0x7f88a41098e0 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.319+0000 7f88a2000640 1 -- 172.21.15.175:0/3227240353 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8894032900 con 0x7f88a41098e0 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.319+0000 7f88a2000640 1 -- 172.21.15.175:0/3227240353 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8894030850 con 0x7f88a41098e0 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 -- 172.21.15.175:0/3227240353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 msgr2=0x7f88a4109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 --2- 172.21.15.175:0/3227240353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a4109ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8894009820 tx=0x7f889402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 -- 172.21.15.175:0/3227240353 shutdown_connections 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 --2- 172.21.15.175:0/3227240353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a4109ce0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 -- 172.21.15.175:0/3227240353 >> 172.21.15.175:0/3227240353 conn(0x7f88a407cb00 msgr2=0x7f88a407cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 -- 172.21.15.175:0/3227240353 shutdown_connections 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.320+0000 7f88a3e00640 1 -- 172.21.15.175:0/3227240353 wait complete. 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.321+0000 7f88a3e00640 1 Processor -- start 2024-07-29T00:26:38.159 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.321+0000 7f88a3e00640 1 -- start start 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.321+0000 7f88a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.321+0000 7f88a3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410d960 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.322+0000 7f88a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.322+0000 7f88a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a419d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45250/0 (socket says 172.21.15.175:45250) 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.322+0000 7f88a2a00640 1 -- 172.21.15.175:0/2017940899 learned_addr learned my addr 172.21.15.175:0/2017940899 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.322+0000 7f88a2a00640 1 -- 172.21.15.175:0/2017940899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a419d8b0 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.322+0000 7f88a2a00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a419d370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88940097f0 tx=0x7f8894030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.323+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8894037040 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.323+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8894040070 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.323+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f889403a390 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.323+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a419db40 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.323+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41a0830 con 0x7f88a41098e0 2024-07-29T00:26:38.160 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.324+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f889403b430 con 0x7f88a41098e0 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.324+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a410dae0 con 0x7f88a41098e0 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.325+0000 7f889b400640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f8888038f70 0x7f888803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.325+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f889406c4b0 con 0x7f88a41098e0 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.325+0000 7f889be00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f8888038f70 0x7f888803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.326+0000 7f889be00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f8888038f70 0x7f888803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f888c0098c0 tx=0x7f888c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.330+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8894032900 con 0x7f88a41098e0 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.538+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f88a4107c60 con 0x7f8888038f70 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.548+0000 7f889b400640 1 -- 172.21.15.175:0/2017940899 <== mgr.14118 v2:172.21.15.175:6800/61876509 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f88a4107c60 con 0x7f8888038f70 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f8888038f70 msgr2=0x7f888803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:38.161 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f8888038f70 0x7f888803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f888c0098c0 tx=0x7f888c006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 msgr2=0x7f88a419d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a419d370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88940097f0 tx=0x7f8894030d80 comp rx=0 tx=0).stop 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 shutdown_connections 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f8888038f70 0x7f888803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 --2- 172.21.15.175:0/2017940899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a41098e0 0x7f88a419d370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 >> 172.21.15.175:0/2017940899 conn(0x7f88a407cb00 msgr2=0x7f88a4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 shutdown_connections 2024-07-29T00:26:38.162 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:36.551+0000 7f88a3e00640 1 -- 172.21.15.175:0/2017940899 wait complete. 2024-07-29T00:26:38.965 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:38 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:38.965 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:38 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:39.588 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.712+0000 7f59c8000640 1 Processor -- start 2024-07-29T00:26:39.589 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.712+0000 7f59c8000640 1 -- start start 2024-07-29T00:26:39.589 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.713+0000 7f59c8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c01099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:39.589 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.713+0000 7f59c8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0109fb0 con 0x7f59c01095e0 2024-07-29T00:26:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.713+0000 7f59c5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c01099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.713+0000 7f59c5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c01099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45254/0 (socket says 172.21.15.175:45254) 2024-07-29T00:26:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.713+0000 7f59c5600640 1 -- 172.21.15.175:0/3901267942 learned_addr learned my addr 172.21.15.175:0/3901267942 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.714+0000 7f59c5600640 1 -- 172.21.15.175:0/3901267942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c010a7e0 con 0x7f59c01095e0 2024-07-29T00:26:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.715+0000 7f59c5600640 1 --2- 172.21.15.175:0/3901267942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c01099e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009820 tx=0x7f59b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13542f04aa653a47 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.715+0000 7f59bfe00640 1 -- 172.21.15.175:0/3901267942 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b4037070 con 0x7f59c01095e0 2024-07-29T00:26:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.715+0000 7f59bfe00640 1 -- 172.21.15.175:0/3901267942 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b4032920 con 0x7f59c01095e0 2024-07-29T00:26:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.715+0000 7f59bfe00640 1 -- 172.21.15.175:0/3901267942 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b4030810 con 0x7f59c01095e0 2024-07-29T00:26:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 -- 172.21.15.175:0/3901267942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 msgr2=0x7f59c01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 --2- 172.21.15.175:0/3901267942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c01099e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009820 tx=0x7f59b402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 -- 172.21.15.175:0/3901267942 shutdown_connections 2024-07-29T00:26:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 --2- 172.21.15.175:0/3901267942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c01099e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 -- 172.21.15.175:0/3901267942 >> 172.21.15.175:0/3901267942 conn(0x7f59c007cc40 msgr2=0x7f59c01073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 -- 172.21.15.175:0/3901267942 shutdown_connections 2024-07-29T00:26:39.593 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.716+0000 7f59c8000640 1 -- 172.21.15.175:0/3901267942 wait complete. 2024-07-29T00:26:39.593 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.717+0000 7f59c8000640 1 Processor -- start 2024-07-29T00:26:39.593 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.717+0000 7f59c8000640 1 -- start start 2024-07-29T00:26:39.593 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.718+0000 7f59c8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:39.593 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.718+0000 7f59c8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010db90 con 0x7f59c01095e0 2024-07-29T00:26:39.594 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.718+0000 7f59c5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:39.595 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.718+0000 7f59c5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45266/0 (socket says 172.21.15.175:45266) 2024-07-29T00:26:39.595 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.718+0000 7f59c5600640 1 -- 172.21.15.175:0/37040306 learned_addr learned my addr 172.21.15.175:0/37040306 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:39.595 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.718+0000 7f59c5600640 1 -- 172.21.15.175:0/37040306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c019db10 con 0x7f59c01095e0 2024-07-29T00:26:39.595 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.719+0000 7f59c5600640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c019d5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59b40097f0 tx=0x7f59b40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:39.595 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.719+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b4037040 con 0x7f59c01095e0 2024-07-29T00:26:39.595 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.719+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b4030cc0 con 0x7f59c01095e0 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.719+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c019dda0 con 0x7f59c01095e0 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.719+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b403a640 con 0x7f59c01095e0 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.719+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a0a90 con 0x7f59c01095e0 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.720+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f59b403b3f0 con 0x7f59c01095e0 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.720+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c010dd10 con 0x7f59c01095e0 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.721+0000 7f59be000640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f59a8038f70 0x7f59a803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:39.597 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.721+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59b406c4e0 con 0x7f59c01095e0 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.721+0000 7f59c4c00640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f59a8038f70 0x7f59a803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.722+0000 7f59c4c00640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f59a8038f70 0x7f59a803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098c0 tx=0x7f59ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.726+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f59b4042e10 con 0x7f59c01095e0 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.940+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f59c0109a60 con 0x7f59c01095e0 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.943+0000 7f59be000640 1 -- 172.21.15.175:0/37040306 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f59b403ba40 con 0x7f59c01095e0 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.946+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f59a8038f70 msgr2=0x7f59a803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.946+0000 7f59c8000640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f59a8038f70 0x7f59a803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098c0 tx=0x7f59ac006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.946+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 msgr2=0x7f59c019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.946+0000 7f59c8000640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c019d5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59b40097f0 tx=0x7f59b40327b0 comp rx=0 tx=0).stop 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.947+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 shutdown_connections 2024-07-29T00:26:39.598 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.947+0000 7f59c8000640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f59a8038f70 0x7f59a803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:39.599 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.947+0000 7f59c8000640 1 --2- 172.21.15.175:0/37040306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c01095e0 0x7f59c019d5d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:39.599 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.947+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 >> 172.21.15.175:0/37040306 conn(0x7f59c007cc40 msgr2=0x7f59c0106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:39.599 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.947+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 shutdown_connections 2024-07-29T00:26:39.599 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:38.947+0000 7f59c8000640 1 -- 172.21.15.175:0/37040306 wait complete. 2024-07-29T00:26:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:39 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/37040306' entity='client.admin' 2024-07-29T00:26:41.346 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:41 smithi175 ceph-mon[32138]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:26:41.346 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:41 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:41.346 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:41 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1251942497' entity='client.admin' 2024-07-29T00:26:41.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.856+0000 7f243e800640 1 Processor -- start 2024-07-29T00:26:41.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.856+0000 7f243e800640 1 -- start start 2024-07-29T00:26:41.860 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.857+0000 7f243e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f2438109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.857+0000 7f243e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f243810a260 con 0x7f2438109890 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.857+0000 7f2437400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f2438109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.857+0000 7f2437400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f2438109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45244/0 (socket says 172.21.15.175:45244) 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.857+0000 7f2437400640 1 -- 172.21.15.175:0/1183147891 learned_addr learned my addr 172.21.15.175:0/1183147891 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.857+0000 7f2437400640 1 -- 172.21.15.175:0/1183147891 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243810a760 con 0x7f2438109890 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.859+0000 7f2437400640 1 --2- 172.21.15.175:0/1183147891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f2438109c90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2428009820 tx=0x7f242802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dccd2abcd04401f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.859+0000 7f2436000640 1 -- 172.21.15.175:0/1183147891 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2428037070 con 0x7f2438109890 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.859+0000 7f2436000640 1 -- 172.21.15.175:0/1183147891 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2428032920 con 0x7f2438109890 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.859+0000 7f2436000640 1 -- 172.21.15.175:0/1183147891 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2428030810 con 0x7f2438109890 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.860+0000 7f243e800640 1 -- 172.21.15.175:0/1183147891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 msgr2=0x7f2438109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.860+0000 7f243e800640 1 --2- 172.21.15.175:0/1183147891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f2438109c90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2428009820 tx=0x7f242802a270 comp rx=0 tx=0).stop 2024-07-29T00:26:41.861 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.860+0000 7f243e800640 1 -- 172.21.15.175:0/1183147891 shutdown_connections 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.860+0000 7f243e800640 1 --2- 172.21.15.175:0/1183147891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f2438109c90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.860+0000 7f243e800640 1 -- 172.21.15.175:0/1183147891 >> 172.21.15.175:0/1183147891 conn(0x7f243807cc30 msgr2=0x7f243807d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.861+0000 7f243e800640 1 -- 172.21.15.175:0/1183147891 shutdown_connections 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.861+0000 7f243e800640 1 -- 172.21.15.175:0/1183147891 wait complete. 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.863+0000 7f243e800640 1 Processor -- start 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.863+0000 7f243e800640 1 -- start start 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.863+0000 7f243e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f243819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.863+0000 7f243e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f243810dba0 con 0x7f2438109890 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.864+0000 7f2437400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f243819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.864+0000 7f2437400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f243819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45252/0 (socket says 172.21.15.175:45252) 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.864+0000 7f2437400640 1 -- 172.21.15.175:0/1251942497 learned_addr learned my addr 172.21.15.175:0/1251942497 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:41.862 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.864+0000 7f2437400640 1 -- 172.21.15.175:0/1251942497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243819db30 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.864+0000 7f2437400640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f243819d5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24280097f0 tx=0x7f2428030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.865+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2428037070 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.865+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2428040070 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.865+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243819ddc0 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.865+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f242803b900 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.865+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a0ab0 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.866+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243810dd20 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.866+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f242803bb30 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.867+0000 7f242fe00640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f241c038fc0 0x7f241c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.867+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f242806c950 con 0x7f2438109890 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.867+0000 7f2436a00640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f241c038fc0 0x7f241c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.868+0000 7f2436a00640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f241c038fc0 0x7f241c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f24240098c0 tx=0x7f2424006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:41.863 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:40.873+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f242806c2a0 con 0x7f2438109890 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.072+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f2438109c90 con 0x7f2438109890 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.076+0000 7f242fe00640 1 -- 172.21.15.175:0/1251942497 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f242803be20 con 0x7f2438109890 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f241c038fc0 msgr2=0x7f241c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f241c038fc0 0x7f241c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f24240098c0 tx=0x7f2424006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 msgr2=0x7f243819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f243819d5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24280097f0 tx=0x7f2428030d40 comp rx=0 tx=0).stop 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 shutdown_connections 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f241c038fc0 0x7f241c03b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 --2- 172.21.15.175:0/1251942497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2438109890 0x7f243819d5f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.079+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 >> 172.21.15.175:0/1251942497 conn(0x7f243807cc30 msgr2=0x7f2438107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:41.864 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.080+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 shutdown_connections 2024-07-29T00:26:41.865 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:41.080+0000 7f243e800640 1 -- 172.21.15.175:0/1251942497 wait complete. 2024-07-29T00:26:41.865 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling the dashboard module... 2024-07-29T00:26:43.444 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-mon[32138]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:26:43.445 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-mon[32138]: from='mgr.14118 172.21.15.175:0/3047644825' entity='mgr.x' 2024-07-29T00:26:43.445 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3523406204' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-29T00:26:43.745 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ignoring --setuser ceph since I am not root 2024-07-29T00:26:43.745 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ignoring --setgroup ceph since I am not root 2024-07-29T00:26:43.745 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:43.743+0000 7f1dd5813040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:26:44.009 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:43.927+0000 7f1dd5813040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:26:44.010 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:44.008+0000 7f1dd5813040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:26:44.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:44 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3523406204' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-29T00:26:44.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:44 smithi175 ceph-mon[32138]: mgrmap e9: x(active, since 23s) 2024-07-29T00:26:44.535 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:44.310+0000 7f1dd5813040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:26:44.535 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:44.380+0000 7f1dd5813040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.410+0000 7f7bb4200640 1 Processor -- start 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.411+0000 7f7bb4200640 1 -- start start 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.411+0000 7f7bb4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.411+0000 7f7bb4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac10a340 con 0x7f7bac109970 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.411+0000 7f7bb1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.412+0000 7f7bb1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45260/0 (socket says 172.21.15.175:45260) 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.412+0000 7f7bb1800640 1 -- 172.21.15.175:0/3531254376 learned_addr learned my addr 172.21.15.175:0/3531254376 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:45.011 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.412+0000 7f7bb1800640 1 -- 172.21.15.175:0/3531254376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac10a7e0 con 0x7f7bac109970 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.413+0000 7f7bb1800640 1 --2- 172.21.15.175:0/3531254376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac109d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009820 tx=0x7f7ba002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33535e58ff6188ac server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.413+0000 7f7babe00640 1 -- 172.21.15.175:0/3531254376 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba0037070 con 0x7f7bac109970 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.413+0000 7f7babe00640 1 -- 172.21.15.175:0/3531254376 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba0032920 con 0x7f7bac109970 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.413+0000 7f7babe00640 1 -- 172.21.15.175:0/3531254376 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba0030810 con 0x7f7bac109970 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.414+0000 7f7bb4200640 1 -- 172.21.15.175:0/3531254376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 msgr2=0x7f7bac109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.414+0000 7f7bb4200640 1 --2- 172.21.15.175:0/3531254376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac109d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009820 tx=0x7f7ba002a270 comp rx=0 tx=0).stop 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.414+0000 7f7bb4200640 1 -- 172.21.15.175:0/3531254376 shutdown_connections 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.414+0000 7f7bb4200640 1 --2- 172.21.15.175:0/3531254376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac109d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.414+0000 7f7bb4200640 1 -- 172.21.15.175:0/3531254376 >> 172.21.15.175:0/3531254376 conn(0x7f7bac07cc40 msgr2=0x7f7bac07d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.414+0000 7f7bb4200640 1 -- 172.21.15.175:0/3531254376 shutdown_connections 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.415+0000 7f7bb4200640 1 -- 172.21.15.175:0/3531254376 wait complete. 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb4200640 1 Processor -- start 2024-07-29T00:26:45.012 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb4200640 1 -- start start 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac10db90 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45264/0 (socket says 172.21.15.175:45264) 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.416+0000 7f7bb1800640 1 -- 172.21.15.175:0/3523406204 learned_addr learned my addr 172.21.15.175:0/3523406204 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.417+0000 7f7bb1800640 1 -- 172.21.15.175:0/3523406204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac19dac0 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.417+0000 7f7bb1800640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac19d580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00097f0 tx=0x7f7ba0030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.417+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba0037040 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.417+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba0040070 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.417+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac19dd50 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.417+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba003a6d0 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.418+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac1a0a40 con 0x7f7bac109970 2024-07-29T00:26:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.419+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7ba003b420 con 0x7f7bac109970 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.419+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac109df0 con 0x7f7bac109970 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.419+0000 7f7baa000640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f7b94038f70 0x7f7b9403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.419+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ba006c630 con 0x7f7bac109970 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.419+0000 7f7bb0e00640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f7b94038f70 0x7f7b9403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.420+0000 7f7bb0e00640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f7b94038f70 0x7f7b9403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7b980098c0 tx=0x7f7b98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.425+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ba004a200 con 0x7f7bac109970 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:42.672+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7bac1a1030 con 0x7f7bac109970 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.477+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7ba0031e80 con 0x7f7bac109970 2024-07-29T00:26:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7baa000640 1 -- 172.21.15.175:0/3523406204 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f7ba003bb20 con 0x7f7bac109970 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f7b94038f70 msgr2=0x7f7b9403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f7b94038f70 0x7f7b9403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7b980098c0 tx=0x7f7b98006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 msgr2=0x7f7bac19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac19d580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00097f0 tx=0x7f7ba0030d40 comp rx=0 tx=0).stop 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 shutdown_connections 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f7b94038f70 0x7f7b9403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 --2- 172.21.15.175:0/3523406204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bac109970 0x7f7bac19d580 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.482+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 >> 172.21.15.175:0/3523406204 conn(0x7f7bac07cc40 msgr2=0x7f7bac107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.483+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 shutdown_connections 2024-07-29T00:26:45.015 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:43.483+0000 7f7bb4200640 1 -- 172.21.15.175:0/3523406204 wait complete. 2024-07-29T00:26:45.826 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:45.609+0000 7f1dd5813040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:26:45.826 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:45.749+0000 7f1dd5813040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:26:45.826 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:45.818+0000 7f1dd5813040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:26:46.149 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:45 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2557150432' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-29T00:26:46.150 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:46.072+0000 7f1dd5813040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:26:46.150 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:46.148+0000 7f1dd5813040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:26:46.523 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:46.228+0000 7f1dd5813040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:26:46.524 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:46.396+0000 7f1dd5813040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:26:46.532 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:46.538 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-29T00:26:46.538 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-29T00:26:46.538 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-29T00:26:46.538 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.575+0000 7f9331200640 1 Processor -- start 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.575+0000 7f9331200640 1 -- start start 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.575+0000 7f9331200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c1075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.575+0000 7f9331200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f932c107bc0 con 0x7f932c1071f0 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.576+0000 7f932b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c1075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.576+0000 7f932b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c1075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45298/0 (socket says 172.21.15.175:45298) 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.576+0000 7f932b400640 1 -- 172.21.15.175:0/401596753 learned_addr learned my addr 172.21.15.175:0/401596753 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.576+0000 7f932b400640 1 -- 172.21.15.175:0/401596753 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c1083f0 con 0x7f932c1071f0 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.577+0000 7f932b400640 1 --2- 172.21.15.175:0/401596753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c1075f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f931c009820 tx=0x7f931c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ec0ccca8cbb839 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.578+0000 7f932aa00640 1 -- 172.21.15.175:0/401596753 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f931c037070 con 0x7f932c1071f0 2024-07-29T00:26:46.539 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.578+0000 7f932aa00640 1 -- 172.21.15.175:0/401596753 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c032920 con 0x7f932c1071f0 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.578+0000 7f932aa00640 1 -- 172.21.15.175:0/401596753 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f931c030810 con 0x7f932c1071f0 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.578+0000 7f9331200640 1 -- 172.21.15.175:0/401596753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 msgr2=0x7f932c1075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.578+0000 7f9331200640 1 --2- 172.21.15.175:0/401596753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c1075f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f931c009820 tx=0x7f931c02a270 comp rx=0 tx=0).stop 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.579+0000 7f9331200640 1 -- 172.21.15.175:0/401596753 shutdown_connections 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.579+0000 7f9331200640 1 --2- 172.21.15.175:0/401596753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c1075f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.579+0000 7f9331200640 1 -- 172.21.15.175:0/401596753 >> 172.21.15.175:0/401596753 conn(0x7f932c102980 msgr2=0x7f932c104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.579+0000 7f9331200640 1 -- 172.21.15.175:0/401596753 shutdown_connections 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.579+0000 7f9331200640 1 -- 172.21.15.175:0/401596753 wait complete. 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f9331200640 1 Processor -- start 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f9331200640 1 -- start start 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f9331200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f9331200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f932c10b7a0 con 0x7f932c1071f0 2024-07-29T00:26:46.540 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f932b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f932b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c19d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45302/0 (socket says 172.21.15.175:45302) 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.580+0000 7f932b400640 1 -- 172.21.15.175:0/2557150432 learned_addr learned my addr 172.21.15.175:0/2557150432 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.581+0000 7f932b400640 1 -- 172.21.15.175:0/2557150432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c19d900 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.581+0000 7f932b400640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c19d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097f0 tx=0x7f931c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.581+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f931c037040 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.581+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c030cc0 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.581+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f931c03a640 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.581+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c19db90 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.582+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1a0880 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.583+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f931c03b3f0 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.583+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c10b920 con 0x7f932c1071f0 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.583+0000 7f9328c00640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 0x7f931003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:46.541 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.583+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f931c06c530 con 0x7f932c1071f0 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.583+0000 7f9323e00640 1 -- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 msgr2=0x7f931003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/61876509 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.583+0000 7f9323e00640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 0x7f931003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.588+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f931c0328c0 con 0x7f932c1071f0 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.784+0000 7f9323e00640 1 -- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 msgr2=0x7f931003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/61876509 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.784+0000 7f9323e00640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 0x7f931003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.825+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f932c1a0a50 con 0x7f932c1071f0 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.826+0000 7f9328c00640 1 -- 172.21.15.175:0/2557150432 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f931c042dc0 con 0x7f932c1071f0 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 msgr2=0x7f931003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 0x7f931003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 msgr2=0x7f932c19d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c19d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097f0 tx=0x7f931c0327b0 comp rx=0 tx=0).stop 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 shutdown_connections 2024-07-29T00:26:46.542 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7f9310038fc0 0x7f931003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:46.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 --2- 172.21.15.175:0/2557150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c1071f0 0x7f932c19d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:46.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 >> 172.21.15.175:0/2557150432 conn(0x7f932c102980 msgr2=0x7f932c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:46.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.830+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 shutdown_connections 2024-07-29T00:26:46.543 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:45.831+0000 7f9331200640 1 -- 172.21.15.175:0/2557150432 wait complete. 2024-07-29T00:26:46.543 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for the mgr to restart... 2024-07-29T00:26:46.543 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr epoch 9... 2024-07-29T00:26:47.151 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:46.949+0000 7f1dd5813040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:26:47.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:47.316+0000 7f1dd5813040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:26:47.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:47.393+0000 7f1dd5813040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:26:47.919 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:47.562+0000 7f1dd5813040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:26:47.919 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:47.918+0000 7f1dd5813040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:26:48.205 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:47.988+0000 7f1dd5813040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:26:48.205 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:48.204+0000 7f1dd5813040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:26:48.460 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:48.384+0000 7f1dd5813040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:26:48.461 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:48.459+0000 7f1dd5813040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:26:48.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:48.557+0000 7f1dd5813040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:26:48.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:48.627+0000 7f1dd5813040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:26:49.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:49.231+0000 7f1dd5813040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:26:49.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:49.408+0000 7f1dd5813040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:26:49.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: Active manager daemon x restarted 2024-07-29T00:26:49.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: Activating manager daemon x 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: osdmap e3: 0 total, 0 up, 0 in 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: mgrmap e10: x(active, starting, since 0.00524275s) 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: Manager daemon x is now available 2024-07-29T00:26:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:26:50.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:26:50.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:26:50.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-mon[32138]: mgrmap e11: x(active, since 1.0088s) 2024-07-29T00:26:51.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:50.861+0000 7f1d2c400640 -1 log_channel(cephadm) log [ERR] : [29/Jul/2024:00:26:50] ENGINE Error in HTTPServer.serve 2024-07-29T00:26:51.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: Traceback (most recent call last): 2024-07-29T00:26:51.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:26:51.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._connections.run(self.expiration_interval) 2024-07-29T00:26:51.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._run(expiration_interval) 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: s = self.context.wrap_socket( 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: return self.sslsocket_class._create( 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self.do_handshake() 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:26:51.037 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._sslobj.do_handshake() 2024-07-29T00:26:51.038 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:26:51.038 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:26:50 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:26:51.668 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:51 smithi175 ceph-mon[32138]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-29T00:26:51.669 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:51 smithi175 ceph-mon[32138]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-29T00:26:51.676 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.151+0000 7ff1d6e00640 1 Processor -- start 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.151+0000 7ff1d6e00640 1 -- start start 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.151+0000 7ff1d6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d01075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:51.677 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.151+0000 7ff1d6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0107bc0 con 0x7ff1d01071f0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.152+0000 7ff1d5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d01075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.152+0000 7ff1d5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d01075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45312/0 (socket says 172.21.15.175:45312) 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.152+0000 7ff1d5a00640 1 -- 172.21.15.175:0/2996015576 learned_addr learned my addr 172.21.15.175:0/2996015576 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.152+0000 7ff1d5a00640 1 -- 172.21.15.175:0/2996015576 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d01083f0 con 0x7ff1d01071f0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.153+0000 7ff1d5a00640 1 --2- 172.21.15.175:0/2996015576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d01075f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009820 tx=0x7ff1c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b0ecdee45ba0df server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.153+0000 7ff1cfe00640 1 -- 172.21.15.175:0/2996015576 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c4037070 con 0x7ff1d01071f0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.153+0000 7ff1cfe00640 1 -- 172.21.15.175:0/2996015576 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c4032920 con 0x7ff1d01071f0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.153+0000 7ff1cfe00640 1 -- 172.21.15.175:0/2996015576 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c4030810 con 0x7ff1d01071f0 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.154+0000 7ff1d6e00640 1 -- 172.21.15.175:0/2996015576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 msgr2=0x7ff1d01075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.154+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/2996015576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d01075f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009820 tx=0x7ff1c402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:51.681 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.154+0000 7ff1d6e00640 1 -- 172.21.15.175:0/2996015576 shutdown_connections 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.154+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/2996015576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d01075f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.154+0000 7ff1d6e00640 1 -- 172.21.15.175:0/2996015576 >> 172.21.15.175:0/2996015576 conn(0x7ff1d0102980 msgr2=0x7ff1d0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.154+0000 7ff1d6e00640 1 -- 172.21.15.175:0/2996015576 shutdown_connections 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.155+0000 7ff1d6e00640 1 -- 172.21.15.175:0/2996015576 wait complete. 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.155+0000 7ff1d6e00640 1 Processor -- start 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d6e00640 1 -- start start 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d010b7a0 con 0x7ff1d01071f0 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45324/0 (socket says 172.21.15.175:45324) 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d5a00640 1 -- 172.21.15.175:0/56572344 learned_addr learned my addr 172.21.15.175:0/56572344 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:51.682 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.156+0000 7ff1d5a00640 1 -- 172.21.15.175:0/56572344 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d019d920 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.157+0000 7ff1d5a00640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d019d3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40097f0 tx=0x7ff1c40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.157+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c4037040 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.157+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c4030cc0 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.157+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d019dbb0 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.157+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c403a640 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.157+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d01a08a0 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.158+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff1c403b3f0 con 0x7ff1d01071f0 2024-07-29T00:26:51.683 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.159+0000 7ff1ce000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.159+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 --> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1b803bb90 con 0x7ff1b8038fc0 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.159+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff1c406c550 con 0x7ff1d01071f0 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.159+0000 7ff1d5000640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 msgr2=0x7ff1b803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/61876509 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.159+0000 7ff1d5000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.360+0000 7ff1d5000640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 msgr2=0x7ff1b803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/61876509 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.360+0000 7ff1d5000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.761+0000 7ff1d5000640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 msgr2=0x7ff1b803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/61876509 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:47.761+0000 7ff1d5000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:48.562+0000 7ff1d5000640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 msgr2=0x7ff1b803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/61876509 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:48.562+0000 7ff1d5000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:49.416+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7ff1c406b000 con 0x7ff1d01071f0 2024-07-29T00:26:51.687 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:49.417+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 msgr2=0x7ff1b803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:49.417+0000 7ff1ce000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.417+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7ff1c402e8a0 con 0x7ff1d01071f0 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.417+0000 7ff1ce000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1b803ca80 0x7ff1b803ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.417+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1c4047e50 con 0x7ff1b803ca80 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.419+0000 7ff1d5000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1b803ca80 0x7ff1b803ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.420+0000 7ff1d5000640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1b803ca80 0x7ff1b803ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc003980 tx=0x7ff1bc006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.421+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff1c4047e50 con 0x7ff1b803ca80 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.434+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff1d010b920 con 0x7ff1b803ca80 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1ce000640 1 -- 172.21.15.175:0/56572344 <== mgr.14150 v2:172.21.15.175:6800/3246788131 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff1d010b920 con 0x7ff1b803ca80 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1b803ca80 msgr2=0x7ff1b803ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1b803ca80 0x7ff1b803ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc003980 tx=0x7ff1bc006e10 comp rx=0 tx=0).stop 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 msgr2=0x7ff1d019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:51.688 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d019d3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40097f0 tx=0x7ff1c40327b0 comp rx=0 tx=0).stop 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 shutdown_connections 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1b803ca80 0x7ff1b803ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:6800/61876509,v1:172.21.15.175:6801/61876509] conn(0x7ff1b8038fc0 0x7ff1b803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 --2- 172.21.15.175:0/56572344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1d01071f0 0x7ff1d019d3e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.435+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 >> 172.21.15.175:0/56572344 conn(0x7ff1d0102980 msgr2=0x7ff1d0103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.436+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 shutdown_connections 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:50.436+0000 7ff1d6e00640 1 -- 172.21.15.175:0/56572344 wait complete. 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:mgr epoch 9 is available 2024-07-29T00:26:51.689 INFO:teuthology.orchestra.run.smithi175.stdout:Generating a dashboard self-signed certificate... 2024-07-29T00:26:52.531 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:50] ENGINE Bus STARTING 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:50] ENGINE Serving on http://172.21.15.175:8765 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:50] ENGINE Serving on https://172.21.15.175:7150 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:50] ENGINE Bus STARTED 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: [29/Jul/2024:00:26:50] ENGINE Error in HTTPServer.serve 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: Traceback (most recent call last): 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._connections.run(self.expiration_interval) 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._run(expiration_interval) 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:26:52.532 INFO:journalctl@ceph.mon.a.smithi175.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: s = self.context.wrap_socket( 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: return self.sslsocket_class._create( 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: self.do_handshake() 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._sslobj.do_handshake() 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: mgrmap e12: x(active, since 2s) 2024-07-29T00:26:52.533 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:26:53.264 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-29T00:26:53.264 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.243+0000 7fc4bf600640 1 Processor -- start 2024-07-29T00:26:53.264 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.243+0000 7fc4bf600640 1 -- start start 2024-07-29T00:26:53.264 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.243+0000 7fc4bf600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b807c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:53.264 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.243+0000 7fc4bf600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b807c8a0 con 0x7fc4b807de60 2024-07-29T00:26:53.264 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.244+0000 7fc4bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b807c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.244+0000 7fc4bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b807c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43834/0 (socket says 172.21.15.175:43834) 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.244+0000 7fc4bcc00640 1 -- 172.21.15.175:0/1425481477 learned_addr learned my addr 172.21.15.175:0/1425481477 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.244+0000 7fc4bcc00640 1 -- 172.21.15.175:0/1425481477 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b807ca20 con 0x7fc4b807de60 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.245+0000 7fc4bcc00640 1 --2- 172.21.15.175:0/1425481477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b807c360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009820 tx=0x7fc4ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92aa4d7d8b0f990 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.246+0000 7fc4b7400640 1 -- 172.21.15.175:0/1425481477 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ac037070 con 0x7fc4b807de60 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.246+0000 7fc4b7400640 1 -- 172.21.15.175:0/1425481477 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac032920 con 0x7fc4b807de60 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.246+0000 7fc4b7400640 1 -- 172.21.15.175:0/1425481477 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ac030810 con 0x7fc4b807de60 2024-07-29T00:26:53.265 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.246+0000 7fc4bf600640 1 -- 172.21.15.175:0/1425481477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 msgr2=0x7fc4b807c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.246+0000 7fc4bf600640 1 --2- 172.21.15.175:0/1425481477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b807c360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009820 tx=0x7fc4ac02a270 comp rx=0 tx=0).stop 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.247+0000 7fc4bf600640 1 -- 172.21.15.175:0/1425481477 shutdown_connections 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.247+0000 7fc4bf600640 1 --2- 172.21.15.175:0/1425481477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b807c360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.247+0000 7fc4bf600640 1 -- 172.21.15.175:0/1425481477 >> 172.21.15.175:0/1425481477 conn(0x7fc4b8102bd0 msgr2=0x7fc4b8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.247+0000 7fc4bf600640 1 -- 172.21.15.175:0/1425481477 shutdown_connections 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.247+0000 7fc4bf600640 1 -- 172.21.15.175:0/1425481477 wait complete. 2024-07-29T00:26:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.248+0000 7fc4bf600640 1 Processor -- start 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.248+0000 7fc4bf600640 1 -- start start 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.249+0000 7fc4bf600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.249+0000 7fc4bf600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b810d9f0 con 0x7fc4b807de60 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.249+0000 7fc4bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.249+0000 7fc4bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b819d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43838/0 (socket says 172.21.15.175:43838) 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.249+0000 7fc4bcc00640 1 -- 172.21.15.175:0/2577895649 learned_addr learned my addr 172.21.15.175:0/2577895649 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.249+0000 7fc4bcc00640 1 -- 172.21.15.175:0/2577895649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b819da70 con 0x7fc4b807de60 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.250+0000 7fc4bcc00640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b819d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0097f0 tx=0x7fc4ac0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:53.267 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.250+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ac037040 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.250+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac030cc0 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.250+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b819dd00 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.250+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ac03a6b0 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.250+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b81a09f0 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.252+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc4ac03baf0 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.252+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b807c3e0 con 0x7fc4b807de60 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.252+0000 7fc4b5600640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4a0039020 0x7fc4a003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.252+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc4ac06c640 con 0x7fc4b807de60 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.253+0000 7fc4b7e00640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4a0039020 0x7fc4a003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.253+0000 7fc4b7e00640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4a0039020 0x7fc4a003b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40098c0 tx=0x7fc4a4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.258+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc4ac042e10 con 0x7fc4b807de60 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.471+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fc4b8103e40 con 0x7fc4a0039020 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.530+0000 7fc4b5600640 1 -- 172.21.15.175:0/2577895649 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fc4b8103e40 con 0x7fc4a0039020 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4a0039020 msgr2=0x7fc4a003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4a0039020 0x7fc4a003b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40098c0 tx=0x7fc4a4006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:53.269 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 msgr2=0x7fc4b819d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b819d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0097f0 tx=0x7fc4ac0327b0 comp rx=0 tx=0).stop 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 shutdown_connections 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4a0039020 0x7fc4a003b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 --2- 172.21.15.175:0/2577895649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b807de60 0x7fc4b819d530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 >> 172.21.15.175:0/2577895649 conn(0x7fc4b8102bd0 msgr2=0x7fc4b8103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 shutdown_connections 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:52.533+0000 7fc4bf600640 1 -- 172.21.15.175:0/2577895649 wait complete. 2024-07-29T00:26:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:Creating initial admin user... 2024-07-29T00:26:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:53 smithi175 ceph-mon[32138]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:26:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:26:55.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:26:55.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:26:55.445 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$.fyEK.oaBZfLw.ypyLsu4OYV8nsPulu0G.J2a.XdQCOwFPOcx2iEK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1722212814, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-29T00:26:55.445 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.837+0000 7fd15ac00640 1 Processor -- start 2024-07-29T00:26:55.445 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.837+0000 7fd15ac00640 1 -- start start 2024-07-29T00:26:55.445 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.837+0000 7fd15ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd154109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.837+0000 7fd15ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd15410a260 con 0x7fd154109890 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.838+0000 7fd153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd154109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.838+0000 7fd153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd154109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43854/0 (socket says 172.21.15.175:43854) 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.838+0000 7fd153e00640 1 -- 172.21.15.175:0/2034750013 learned_addr learned my addr 172.21.15.175:0/2034750013 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.838+0000 7fd153e00640 1 -- 172.21.15.175:0/2034750013 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15410a760 con 0x7fd154109890 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.839+0000 7fd153e00640 1 --2- 172.21.15.175:0/2034750013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd154109c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd144009820 tx=0x7fd14402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417c32b2fbd98a83 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd152a00640 1 -- 172.21.15.175:0/2034750013 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd144037070 con 0x7fd154109890 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd152a00640 1 -- 172.21.15.175:0/2034750013 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd144032920 con 0x7fd154109890 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd152a00640 1 -- 172.21.15.175:0/2034750013 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd144030810 con 0x7fd154109890 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd15ac00640 1 -- 172.21.15.175:0/2034750013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 msgr2=0x7fd154109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:55.446 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd15ac00640 1 --2- 172.21.15.175:0/2034750013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd154109c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd144009820 tx=0x7fd14402a270 comp rx=0 tx=0).stop 2024-07-29T00:26:55.447 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd15ac00640 1 -- 172.21.15.175:0/2034750013 shutdown_connections 2024-07-29T00:26:55.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd15ac00640 1 --2- 172.21.15.175:0/2034750013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd154109c90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:55.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.840+0000 7fd15ac00640 1 -- 172.21.15.175:0/2034750013 >> 172.21.15.175:0/2034750013 conn(0x7fd15407cc30 msgr2=0x7fd15407d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:55.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.841+0000 7fd15ac00640 1 -- 172.21.15.175:0/2034750013 shutdown_connections 2024-07-29T00:26:55.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.841+0000 7fd15ac00640 1 -- 172.21.15.175:0/2034750013 wait complete. 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd15ac00640 1 Processor -- start 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd15ac00640 1 -- start start 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd15ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd15419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd15ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd15410dba0 con 0x7fd154109890 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd15419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd15419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43860/0 (socket says 172.21.15.175:43860) 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.842+0000 7fd153e00640 1 -- 172.21.15.175:0/3490908875 learned_addr learned my addr 172.21.15.175:0/3490908875 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.843+0000 7fd153e00640 1 -- 172.21.15.175:0/3490908875 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15419daf0 con 0x7fd154109890 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.843+0000 7fd153e00640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd15419d5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1440097f0 tx=0x7fd144030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.844+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd144037070 con 0x7fd154109890 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.844+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd144040070 con 0x7fd154109890 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.844+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd14403b900 con 0x7fd154109890 2024-07-29T00:26:55.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.844+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15419dd80 con 0x7fd154109890 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.844+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1541a0a70 con 0x7fd154109890 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.845+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fd14403bb30 con 0x7fd154109890 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.845+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15410dd20 con 0x7fd154109890 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.845+0000 7fd150c00640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd138039020 0x7fd13803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.845+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd14406ca90 con 0x7fd154109890 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.846+0000 7fd153400640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd138039020 0x7fd13803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.846+0000 7fd153400640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd138039020 0x7fd13803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1400098c0 tx=0x7fd140006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:53.855+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd14402e200 con 0x7fd154109890 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.073+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fd1541a0d00 con 0x7fd138039020 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.308+0000 7fd150c00640 1 -- 172.21.15.175:0/3490908875 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fd1541a0d00 con 0x7fd138039020 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.311+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd138039020 msgr2=0x7fd13803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.311+0000 7fd15ac00640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd138039020 0x7fd13803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1400098c0 tx=0x7fd140006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.311+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 msgr2=0x7fd15419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.311+0000 7fd15ac00640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd15419d5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1440097f0 tx=0x7fd144030d40 comp rx=0 tx=0).stop 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.312+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 shutdown_connections 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.312+0000 7fd15ac00640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd138039020 0x7fd13803b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.312+0000 7fd15ac00640 1 --2- 172.21.15.175:0/3490908875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd154109890 0x7fd15419d5b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.312+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 >> 172.21.15.175:0/3490908875 conn(0x7fd15407cc30 msgr2=0x7fd154107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.312+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 shutdown_connections 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:54.312+0000 7fd15ac00640 1 -- 172.21.15.175:0/3490908875 wait complete. 2024-07-29T00:26:55.453 INFO:teuthology.orchestra.run.smithi175.stdout:Fetching dashboard port number... 2024-07-29T00:26:55.687 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:55 smithi175 ceph-mon[32138]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:26:56.707 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:56 smithi175 ceph-mon[32138]: mgrmap e13: x(active, since 6s) 2024-07-29T00:26:57.351 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 8443 2024-07-29T00:26:57.352 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.500+0000 7fa8e6600640 1 Processor -- start 2024-07-29T00:26:57.352 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.500+0000 7fa8e6600640 1 -- start start 2024-07-29T00:26:57.352 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.501+0000 7fa8e6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e0107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:57.352 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.501+0000 7fa8e6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0107df0 con 0x7fa8e0107420 2024-07-29T00:26:57.353 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.501+0000 7fa8df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e0107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:57.353 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.501+0000 7fa8df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e0107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43874/0 (socket says 172.21.15.175:43874) 2024-07-29T00:26:57.353 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.501+0000 7fa8df400640 1 -- 172.21.15.175:0/2511011898 learned_addr learned my addr 172.21.15.175:0/2511011898 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:57.353 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.502+0000 7fa8df400640 1 -- 172.21.15.175:0/2511011898 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e0108580 con 0x7fa8e0107420 2024-07-29T00:26:57.354 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.503+0000 7fa8df400640 1 --2- 172.21.15.175:0/2511011898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e0107820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009820 tx=0x7fa8d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785215f5c8f8fec6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:57.354 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.504+0000 7fa8de000640 1 -- 172.21.15.175:0/2511011898 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0037070 con 0x7fa8e0107420 2024-07-29T00:26:57.354 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.504+0000 7fa8de000640 1 -- 172.21.15.175:0/2511011898 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d0032920 con 0x7fa8e0107420 2024-07-29T00:26:57.354 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.504+0000 7fa8de000640 1 -- 172.21.15.175:0/2511011898 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0030810 con 0x7fa8e0107420 2024-07-29T00:26:57.354 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.504+0000 7fa8e6600640 1 -- 172.21.15.175:0/2511011898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 msgr2=0x7fa8e0107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:57.355 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.504+0000 7fa8e6600640 1 --2- 172.21.15.175:0/2511011898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e0107820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009820 tx=0x7fa8d002a270 comp rx=0 tx=0).stop 2024-07-29T00:26:57.355 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.505+0000 7fa8e6600640 1 -- 172.21.15.175:0/2511011898 shutdown_connections 2024-07-29T00:26:57.355 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.505+0000 7fa8e6600640 1 --2- 172.21.15.175:0/2511011898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e0107820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:57.355 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.505+0000 7fa8e6600640 1 -- 172.21.15.175:0/2511011898 >> 172.21.15.175:0/2511011898 conn(0x7fa8e0102bd0 msgr2=0x7fa8e0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:57.356 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.505+0000 7fa8e6600640 1 -- 172.21.15.175:0/2511011898 shutdown_connections 2024-07-29T00:26:57.356 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.505+0000 7fa8e6600640 1 -- 172.21.15.175:0/2511011898 wait complete. 2024-07-29T00:26:57.356 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.506+0000 7fa8e6600640 1 Processor -- start 2024-07-29T00:26:57.356 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.506+0000 7fa8e6600640 1 -- start start 2024-07-29T00:26:57.356 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.506+0000 7fa8e6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:57.356 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.506+0000 7fa8e6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e010b930 con 0x7fa8e0107420 2024-07-29T00:26:57.357 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.507+0000 7fa8df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:57.357 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.507+0000 7fa8df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43878/0 (socket says 172.21.15.175:43878) 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.507+0000 7fa8df400640 1 -- 172.21.15.175:0/3677869122 learned_addr learned my addr 172.21.15.175:0/3677869122 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.507+0000 7fa8df400640 1 -- 172.21.15.175:0/3677869122 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e007e0d0 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.507+0000 7fa8df400640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e007db90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00097f0 tx=0x7fa8d00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.507+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0037040 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.508+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d0030cc0 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.508+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d003a640 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.508+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e007c2c0 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.508+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e007c800 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.509+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa8d003b3f0 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.509+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e01078a0 con 0x7fa8e0107420 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.509+0000 7fa8d7e00640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa8c40390c0 0x7fa8c403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:57.359 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.509+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa8d006c790 con 0x7fa8e0107420 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.510+0000 7fa8dea00640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa8c40390c0 0x7fa8c403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.510+0000 7fa8dea00640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa8c40390c0 0x7fa8c403b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0098c0 tx=0x7fa8cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.515+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa8d00313a0 con 0x7fa8e0107420 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.706+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa8e007ca80 con 0x7fa8e0107420 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.708+0000 7fa8d7e00640 1 -- 172.21.15.175:0/3677869122 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa8d0038250 con 0x7fa8e0107420 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.710+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa8c40390c0 msgr2=0x7fa8c403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.710+0000 7fa8e6600640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa8c40390c0 0x7fa8c403b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0098c0 tx=0x7fa8cc006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.710+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 msgr2=0x7fa8e007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.710+0000 7fa8e6600640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e007db90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00097f0 tx=0x7fa8d00327b0 comp rx=0 tx=0).stop 2024-07-29T00:26:57.361 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.711+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 shutdown_connections 2024-07-29T00:26:57.362 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.711+0000 7fa8e6600640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa8c40390c0 0x7fa8c403b580 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:57.362 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.711+0000 7fa8e6600640 1 --2- 172.21.15.175:0/3677869122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e0107420 0x7fa8e007db90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:57.362 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.711+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 >> 172.21.15.175:0/3677869122 conn(0x7fa8e0102bd0 msgr2=0x7fa8e0103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:57.362 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.711+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 shutdown_connections 2024-07-29T00:26:57.362 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:56.711+0000 7fa8e6600640 1 -- 172.21.15.175:0/3677869122 wait complete. 2024-07-29T00:26:57.399 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-29T00:26:57.399 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout disabled 2024-07-29T00:26:57.425 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-29T00:26:57.425 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-07-29T00:26:57.426 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld.service is not enabled 2024-07-29T00:26:57.426 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-29T00:26:57.433 INFO:teuthology.orchestra.run.smithi175.stdout:Ceph Dashboard is now available at: 2024-07-29T00:26:57.433 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:57.433 INFO:teuthology.orchestra.run.smithi175.stdout: URL: https://smithi175.front.sepia.ceph.com:8443/ 2024-07-29T00:26:57.433 INFO:teuthology.orchestra.run.smithi175.stdout: User: admin 2024-07-29T00:26:57.433 INFO:teuthology.orchestra.run.smithi175.stdout: Password: p30x8p23oa 2024-07-29T00:26:57.433 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:57.434 INFO:teuthology.orchestra.run.smithi175.stdout:Saving cluster configuration to /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config directory 2024-07-29T00:26:57.887 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:57 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3677869122' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-29T00:26:59.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.021+0000 7ff6f7e00640 1 Processor -- start 2024-07-29T00:26:59.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.021+0000 7ff6f7e00640 1 -- start start 2024-07-29T00:26:59.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.021+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f007c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:59.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.021+0000 7ff6f7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f007c930 con 0x7ff6f007deb0 2024-07-29T00:26:59.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.022+0000 7ff6f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f007c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:59.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.022+0000 7ff6f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f007c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43890/0 (socket says 172.21.15.175:43890) 2024-07-29T00:26:59.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.022+0000 7ff6f5400640 1 -- 172.21.15.175:0/3743548619 learned_addr learned my addr 172.21.15.175:0/3743548619 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:59.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.022+0000 7ff6f5400640 1 -- 172.21.15.175:0/3743548619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f007cab0 con 0x7ff6f007deb0 2024-07-29T00:26:59.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.023+0000 7ff6f5400640 1 --2- 172.21.15.175:0/3743548619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f007c360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009e80 tx=0x7ff6e402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40c16ad3b1d231f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.024+0000 7ff6efe00640 1 -- 172.21.15.175:0/3743548619 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e4036070 con 0x7ff6f007deb0 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.024+0000 7ff6efe00640 1 -- 172.21.15.175:0/3743548619 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e40318e0 con 0x7ff6f007deb0 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.024+0000 7ff6efe00640 1 -- 172.21.15.175:0/3743548619 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e4007a00 con 0x7ff6f007deb0 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.024+0000 7ff6f7e00640 1 -- 172.21.15.175:0/3743548619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 msgr2=0x7ff6f007c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.024+0000 7ff6f7e00640 1 --2- 172.21.15.175:0/3743548619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f007c360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009e80 tx=0x7ff6e402a7a0 comp rx=0 tx=0).stop 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.025+0000 7ff6f7e00640 1 -- 172.21.15.175:0/3743548619 shutdown_connections 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.025+0000 7ff6f7e00640 1 --2- 172.21.15.175:0/3743548619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f007c360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.025+0000 7ff6f7e00640 1 -- 172.21.15.175:0/3743548619 >> 172.21.15.175:0/3743548619 conn(0x7ff6f0102c10 msgr2=0x7ff6f0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.025+0000 7ff6f7e00640 1 -- 172.21.15.175:0/3743548619 shutdown_connections 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.025+0000 7ff6f7e00640 1 -- 172.21.15.175:0/3743548619 wait complete. 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f7e00640 1 Processor -- start 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f7e00640 1 -- start start 2024-07-29T00:26:59.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:59.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f010da10 con 0x7ff6f007deb0 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43892/0 (socket says 172.21.15.175:43892) 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.026+0000 7ff6f5400640 1 -- 172.21.15.175:0/837186619 learned_addr learned my addr 172.21.15.175:0/837186619 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.027+0000 7ff6f5400640 1 -- 172.21.15.175:0/837186619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f019da90 con 0x7ff6f007deb0 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.027+0000 7ff6f5400640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f019d550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4031cf0 tx=0x7ff6e4031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.027+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e4036050 con 0x7ff6f007deb0 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.027+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4042070 con 0x7ff6f007deb0 2024-07-29T00:26:59.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.027+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e403d9b0 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.027+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f019dd20 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.028+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f01a0a10 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.029+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff6e403bcf0 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.029+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f007c3e0 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.029+0000 7ff6ee000640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6d80390c0 0x7ff6d803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.029+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff6e406d620 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.029+0000 7ff6f4a00640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6d80390c0 0x7ff6d803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.030+0000 7ff6f4a00640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6d80390c0 0x7ff6d803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0098c0 tx=0x7ff6dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.034+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff6e4044550 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.315+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff6f01a0be0 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.321+0000 7ff6ee000640 1 -- 172.21.15.175:0/837186619 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7ff6e4042220 con 0x7ff6f007deb0 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-29T00:26:59.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6d80390c0 msgr2=0x7ff6d803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6d80390c0 0x7ff6d803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0098c0 tx=0x7ff6dc006db0 comp rx=0 tx=0).stop 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 msgr2=0x7ff6f019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f019d550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4031cf0 tx=0x7ff6e4031d20 comp rx=0 tx=0).stop 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 shutdown_connections 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6d80390c0 0x7ff6d803b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 --2- 172.21.15.175:0/837186619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f007deb0 0x7ff6f019d550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:26:59.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 >> 172.21.15.175:0/837186619 conn(0x7ff6f0102c10 msgr2=0x7ff6f0103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 shutdown_connections 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-07-29T00:26:58.324+0000 7ff6f7e00640 1 -- 172.21.15.175:0/837186619 wait complete. 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:Or, if you are only running a single cluster on this host: 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: ceph telemetry on 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout:For more information see: 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:26:59.073 INFO:teuthology.orchestra.run.smithi175.stdout:Bootstrap complete. 2024-07-29T00:26:59.191 INFO:tasks.cephadm:Fetching config... 2024-07-29T00:26:59.191 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:26:59.191 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-29T00:26:59.240 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-29T00:26:59.240 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:26:59.240 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-29T00:26:59.320 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-29T00:26:59.321 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:26:59.321 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/keyring of=/dev/stdout 2024-07-29T00:26:59.442 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-29T00:26:59.442 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:26:59.442 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-29T00:26:59.492 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-29T00:26:59.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-29T00:26:59.595 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:26:59 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/837186619' entity='client.admin' 2024-07-29T00:26:59.642 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:59.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-29T00:26:59.711 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:59.723 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-29T00:26:59.764 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCioJF4vdwHm8odbhgP8fXSrvwKYa1UMmmF0AxetDacae+aEpB5Vw7WxUP+Z/kR11yGam1524PfBJ2kJKmVnXGbVOgk7s6zDpb8/Xkmor+GMZLElv0HfU5ZFjGGYuUhX9fYpfns1U6ONQcD+sb4k3cjV+uiqteIQP61gYx9EjQ/1AcIG0HoXOcRTvndDcGBKfx+RKPQtIuAc3NFVCz/oZt87W0n/jQSdnYL6iETwAje0/uZii31LzIIh1BdfIi6mJUwMGYdZLBjnMyy5fZTV8Myymq7qJ2GR3WL3501BO9U3LN6KjSm49cjdf18WAC6LxUVAUYmDowfV1Dbc48toZZeyYfmnK8idVVhtjz1oVTorl8WExdQq7N46VtyRBrqq4/vtcS11hbPSOx2zrnCGCt6+MuHsKJ9dniDZ/CDFoUhcDlM9VEPmpTN7FncEmFsW2tKBREFrhOGyuWoFAU76TO8eeP7SvjfF3nF2wMIldtnZpXk29JIk2xmXbFEC1S5R8s= ceph-f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:26:59.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-29T00:27:00.207 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:00.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 -- 172.21.15.175:0/466887367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 msgr2=0x7fcb48104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:00.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 --2- 172.21.15.175:0/466887367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb48104190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009880 tx=0x7fcb3c02a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:00.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 -- 172.21.15.175:0/466887367 shutdown_connections 2024-07-29T00:27:00.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 --2- 172.21.15.175:0/466887367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb48104190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:00.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 -- 172.21.15.175:0/466887367 >> 172.21.15.175:0/466887367 conn(0x7fcb480fb9a0 msgr2=0x7fcb480fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:00.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 -- 172.21.15.175:0/466887367 shutdown_connections 2024-07-29T00:27:00.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.841+0000 7fcb50200640 1 -- 172.21.15.175:0/466887367 wait complete. 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.842+0000 7fcb50200640 1 Processor -- start 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.842+0000 7fcb50200640 1 -- start start 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.843+0000 7fcb50200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb481969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.843+0000 7fcb50200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481073b0 con 0x7fcb48101da0 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.843+0000 7fcb4d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb481969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.843+0000 7fcb4d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb481969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38586/0 (socket says 172.21.15.175:38586) 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.843+0000 7fcb4d800640 1 -- 172.21.15.175:0/1749710634 learned_addr learned my addr 172.21.15.175:0/1749710634 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb4d800640 1 -- 172.21.15.175:0/1749710634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb48196f20 con 0x7fcb48101da0 2024-07-29T00:27:00.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb4d800640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb481969e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c004f60 tx=0x7fcb3c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:00.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c03f070 con 0x7fcb48101da0 2024-07-29T00:27:00.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb3c003ad0 con 0x7fcb48101da0 2024-07-29T00:27:00.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c035030 con 0x7fcb48101da0 2024-07-29T00:27:00.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb48197210 con 0x7fcb48101da0 2024-07-29T00:27:00.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.844+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb48199ea0 con 0x7fcb48101da0 2024-07-29T00:27:00.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.846+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb48107b30 con 0x7fcb48101da0 2024-07-29T00:27:00.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.846+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcb3c042070 con 0x7fcb48101da0 2024-07-29T00:27:00.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.846+0000 7fcb46000640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fcb300390c0 0x7fcb3003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:00.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.846+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb3c06ce40 con 0x7fcb48101da0 2024-07-29T00:27:00.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.847+0000 7fcb4ce00640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fcb300390c0 0x7fcb3003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:00.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.848+0000 7fcb4ce00640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fcb300390c0 0x7fcb3003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb340098c0 tx=0x7fcb34006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:00.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:00.852+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcb3c03a3c0 con 0x7fcb48101da0 2024-07-29T00:27:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.047+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fcb4802d6b0 con 0x7fcb48101da0 2024-07-29T00:27:01.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.052+0000 7fcb46000640 1 -- 172.21.15.175:0/1749710634 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fcb3c030d90 con 0x7fcb48101da0 2024-07-29T00:27:01.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fcb300390c0 msgr2=0x7fcb3003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fcb300390c0 0x7fcb3003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb340098c0 tx=0x7fcb34006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 msgr2=0x7fcb481969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb481969e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c004f60 tx=0x7fcb3c0048f0 comp rx=0 tx=0).stop 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 shutdown_connections 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fcb300390c0 0x7fcb3003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 --2- 172.21.15.175:0/1749710634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb48101da0 0x7fcb481969e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.055+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 >> 172.21.15.175:0/1749710634 conn(0x7fcb480fb9a0 msgr2=0x7fcb480fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.056+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 shutdown_connections 2024-07-29T00:27:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:01.056+0000 7fcb50200640 1 -- 172.21.15.175:0/1749710634 wait complete. 2024-07-29T00:27:01.992 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-29T00:27:01.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-29T00:27:02.231 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:02 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1749710634' entity='client.admin' 2024-07-29T00:27:02.451 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:03.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 -- 172.21.15.175:0/817273670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 msgr2=0x7f26e4103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:03.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 --2- 172.21.15.175:0/817273670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4103910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009880 tx=0x7f26d402a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:03.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 -- 172.21.15.175:0/817273670 shutdown_connections 2024-07-29T00:27:03.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 --2- 172.21.15.175:0/817273670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4103910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:03.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 -- 172.21.15.175:0/817273670 >> 172.21.15.175:0/817273670 conn(0x7f26e40fece0 msgr2=0x7f26e4101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 -- 172.21.15.175:0/817273670 shutdown_connections 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.210+0000 7f26e9400640 1 -- 172.21.15.175:0/817273670 wait complete. 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.211+0000 7f26e9400640 1 Processor -- start 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.211+0000 7f26e9400640 1 -- start start 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.212+0000 7f26e9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.212+0000 7f26e9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4109620 con 0x7f26e4103510 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.212+0000 7f26e2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.212+0000 7f26e2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38612/0 (socket says 172.21.15.175:38612) 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.212+0000 7f26e2800640 1 -- 172.21.15.175:0/4217315685 learned_addr learned my addr 172.21.15.175:0/4217315685 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26e2800640 1 -- 172.21.15.175:0/4217315685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e4199190 con 0x7f26e4103510 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26e2800640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4198c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004f60 tx=0x7f26d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d403f070 con 0x7f26e4103510 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4004cc0 con 0x7f26e4103510 2024-07-29T00:27:03.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4035030 con 0x7f26e4103510 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e4199420 con 0x7f26e4103510 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.213+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e419c110 con 0x7f26e4103510 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.215+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f26d4042070 con 0x7f26e4103510 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.215+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e4109da0 con 0x7f26e4103510 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.215+0000 7f26db400640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26c80390c0 0x7f26c803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.215+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f26d406cd10 con 0x7f26e4103510 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.216+0000 7f26e1e00640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26c80390c0 0x7f26c803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.216+0000 7f26e1e00640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26c80390c0 0x7f26c803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26d00098c0 tx=0x7f26d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.220+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f26d4040260 con 0x7f26e4103510 2024-07-29T00:27:03.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.415+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f26e402d6a0 con 0x7f26c80390c0 2024-07-29T00:27:03.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.421+0000 7f26db400640 1 -- 172.21.15.175:0/4217315685 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26e402d6a0 con 0x7f26c80390c0 2024-07-29T00:27:03.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26c80390c0 msgr2=0x7f26c803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26c80390c0 0x7f26c803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26d00098c0 tx=0x7f26d0006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 msgr2=0x7f26e4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4198c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004f60 tx=0x7f26d40048f0 comp rx=0 tx=0).stop 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 shutdown_connections 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26c80390c0 0x7f26c803b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 --2- 172.21.15.175:0/4217315685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4103510 0x7f26e4198c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.424+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 >> 172.21.15.175:0/4217315685 conn(0x7f26e40fece0 msgr2=0x7f26e40ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:03.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.425+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 shutdown_connections 2024-07-29T00:27:03.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:03.425+0000 7f26e9400640 1 -- 172.21.15.175:0/4217315685 wait complete. 2024-07-29T00:27:04.104 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-07-29T00:27:04.104 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:27:04.105 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-07-29T00:27:04.121 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:27:04.121 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:04.177 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-07-29T00:27:04.178 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch host add smithi181 2024-07-29T00:27:04.704 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:04 smithi175 ceph-mon[32138]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:04.705 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:05.095 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:05.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 -- 172.21.15.175:0/1473126071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 msgr2=0x7f45681014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:05.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 --2- 172.21.15.175:0/1473126071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f45681014f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f455c009880 tx=0x7f455c02a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:05.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 -- 172.21.15.175:0/1473126071 shutdown_connections 2024-07-29T00:27:05.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 --2- 172.21.15.175:0/1473126071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f45681014f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:05.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 -- 172.21.15.175:0/1473126071 >> 172.21.15.175:0/1473126071 conn(0x7f45680fc880 msgr2=0x7f45680fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:05.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 -- 172.21.15.175:0/1473126071 shutdown_connections 2024-07-29T00:27:05.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.713+0000 7f456f000640 1 -- 172.21.15.175:0/1473126071 wait complete. 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.714+0000 7f456f000640 1 Processor -- start 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.714+0000 7f456f000640 1 -- start start 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.715+0000 7f456f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f4568198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.715+0000 7f456f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4568107200 con 0x7f45681010f0 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.715+0000 7f456dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f4568198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.715+0000 7f456dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f4568198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38636/0 (socket says 172.21.15.175:38636) 2024-07-29T00:27:05.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.715+0000 7f456dc00640 1 -- 172.21.15.175:0/2100725334 learned_addr learned my addr 172.21.15.175:0/2100725334 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:05.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.715+0000 7f456dc00640 1 -- 172.21.15.175:0/2100725334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4568199040 con 0x7f45681010f0 2024-07-29T00:27:05.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.716+0000 7f456dc00640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f4568198b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f455c004f60 tx=0x7f455c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:05.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.716+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c03f070 con 0x7f45681010f0 2024-07-29T00:27:05.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.716+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455c004cc0 con 0x7f45681010f0 2024-07-29T00:27:05.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.716+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c035030 con 0x7f45681010f0 2024-07-29T00:27:05.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.716+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681992d0 con 0x7f45681010f0 2024-07-29T00:27:05.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.716+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456819bfc0 con 0x7f45681010f0 2024-07-29T00:27:05.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.717+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f455c042070 con 0x7f45681010f0 2024-07-29T00:27:05.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.718+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4568107980 con 0x7f45681010f0 2024-07-29T00:27:05.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.718+0000 7f4566000640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4550039060 0x7f455003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:05.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.718+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f455c06cd40 con 0x7f45681010f0 2024-07-29T00:27:05.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.718+0000 7f456d200640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4550039060 0x7f455003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:05.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.719+0000 7f456d200640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4550039060 0x7f455003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f45540098c0 tx=0x7f4554006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:05.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.724+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f455c0035f0 con 0x7f45681010f0 2024-07-29T00:27:05.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:05.922+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) -- 0x7f456802d610 con 0x7f4550039060 2024-07-29T00:27:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:27:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:06 smithi175 ceph-mon[32138]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:27:07.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:07 smithi175 ceph-mon[32138]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:07.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:07 smithi175 ceph-mon[32138]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:07.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:07 smithi175 ceph-mon[32138]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:08.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:27:08.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: Deploying cephadm binary to smithi181 2024-07-29T00:27:08.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:08.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:08.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:08.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:08.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:08.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:08.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:09.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.437+0000 7f4566000640 1 -- 172.21.15.175:0/2100725334 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f456802d610 con 0x7f4550039060 2024-07-29T00:27:09.438 INFO:teuthology.orchestra.run.smithi175.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.441+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4550039060 msgr2=0x7f455003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.441+0000 7f456f000640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4550039060 0x7f455003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f45540098c0 tx=0x7f4554006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.441+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 msgr2=0x7f4568198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.441+0000 7f456f000640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f4568198b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f455c004f60 tx=0x7f455c0048f0 comp rx=0 tx=0).stop 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.442+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 shutdown_connections 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.442+0000 7f456f000640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4550039060 0x7f455003b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.442+0000 7f456f000640 1 --2- 172.21.15.175:0/2100725334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45681010f0 0x7f4568198b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.442+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 >> 172.21.15.175:0/2100725334 conn(0x7f45680fc880 msgr2=0x7f45680fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.442+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 shutdown_connections 2024-07-29T00:27:09.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:09.442+0000 7f456f000640 1 -- 172.21.15.175:0/2100725334 wait complete. 2024-07-29T00:27:10.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch host ls --format=json 2024-07-29T00:27:10.603 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:10.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:10 smithi175 ceph-mon[32138]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:10.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:10.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:10 smithi175 ceph-mon[32138]: Added host smithi181 2024-07-29T00:27:10.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:10.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:10.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:11.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 -- 172.21.15.175:0/4183930338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 msgr2=0x7fabf41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:11.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 --2- 172.21.15.175:0/4183930338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf41038f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009c20 tx=0x7fabe402aa90 comp rx=0 tx=0).stop 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 -- 172.21.15.175:0/4183930338 shutdown_connections 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 --2- 172.21.15.175:0/4183930338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf41038f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 -- 172.21.15.175:0/4183930338 >> 172.21.15.175:0/4183930338 conn(0x7fabf40fefb0 msgr2=0x7fabf41013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 -- 172.21.15.175:0/4183930338 shutdown_connections 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.219+0000 7fabf8e00640 1 -- 172.21.15.175:0/4183930338 wait complete. 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.220+0000 7fabf8e00640 1 Processor -- start 2024-07-29T00:27:11.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.220+0000 7fabf8e00640 1 -- start start 2024-07-29T00:27:11.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.221+0000 7fabf8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:11.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.221+0000 7fabf8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabf410d960 con 0x7fabf4103510 2024-07-29T00:27:11.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.221+0000 7fabf3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:11.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.221+0000 7fabf3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf4199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44574/0 (socket says 172.21.15.175:44574) 2024-07-29T00:27:11.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.221+0000 7fabf3400640 1 -- 172.21.15.175:0/3858656649 learned_addr learned my addr 172.21.15.175:0/3858656649 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.221+0000 7fabf3400640 1 -- 172.21.15.175:0/3858656649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41996c0 con 0x7fabf4103510 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.222+0000 7fabf3400640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf4199180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabe4005e30 tx=0x7fabe402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.222+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4047070 con 0x7fabf4103510 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.222+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41999b0 con 0x7fabf4103510 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.222+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe4003df0 con 0x7fabf4103510 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.222+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4033050 con 0x7fabf4103510 2024-07-29T00:27:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.222+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf419da50 con 0x7fabf4103510 2024-07-29T00:27:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.223+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fabe404a070 con 0x7fabf4103510 2024-07-29T00:27:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.223+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf41049f0 con 0x7fabf4103510 2024-07-29T00:27:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.224+0000 7fabebe00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fabd8039060 0x7fabd803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.224+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabe403e030 con 0x7fabf4103510 2024-07-29T00:27:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.224+0000 7fabf2a00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fabd8039060 0x7fabd803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.225+0000 7fabf2a00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fabd8039060 0x7fabd803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fabe00098c0 tx=0x7fabe0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:11.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.229+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fabe4040c90 con 0x7fabf4103510 2024-07-29T00:27:11.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.421+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabf419a4e0 con 0x7fabd8039060 2024-07-29T00:27:11.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.423+0000 7fabebe00640 1 -- 172.21.15.175:0/3858656649 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fabf419a4e0 con 0x7fabd8039060 2024-07-29T00:27:11.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:27:11.423 INFO:teuthology.orchestra.run.smithi175.stdout:[{"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-07-29T00:27:11.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fabd8039060 msgr2=0x7fabd803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:11.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fabd8039060 0x7fabd803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fabe00098c0 tx=0x7fabe0006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:11.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 msgr2=0x7fabf4199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:11.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf4199180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabe4005e30 tx=0x7fabe402fc10 comp rx=0 tx=0).stop 2024-07-29T00:27:11.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 shutdown_connections 2024-07-29T00:27:11.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fabd8039060 0x7fabd803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:11.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 --2- 172.21.15.175:0/3858656649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf4103510 0x7fabf4199180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:11.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 >> 172.21.15.175:0/3858656649 conn(0x7fabf40fefb0 msgr2=0x7fabf410aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:11.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 shutdown_connections 2024-07-29T00:27:11.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:11.426+0000 7fabf8e00640 1 -- 172.21.15.175:0/3858656649 wait complete. 2024-07-29T00:27:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:11 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:11 smithi175 ceph-mon[32138]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:27:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:11 smithi175 ceph-mon[32138]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:12.161 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-07-29T00:27:12.162 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:27:12.162 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-07-29T00:27:12.180 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:27:12.180 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:12.238 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-07-29T00:27:12.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch host add smithi204 2024-07-29T00:27:12.685 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:13.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 -- 172.21.15.175:0/2852153773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 msgr2=0x7f6800101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:13.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 --2- 172.21.15.175:0/2852153773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f6800101900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009880 tx=0x7f67f002a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:13.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 -- 172.21.15.175:0/2852153773 shutdown_connections 2024-07-29T00:27:13.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 --2- 172.21.15.175:0/2852153773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f6800101900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:13.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 -- 172.21.15.175:0/2852153773 >> 172.21.15.175:0/2852153773 conn(0x7f68000fcfa0 msgr2=0x7f68000ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:13.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 -- 172.21.15.175:0/2852153773 shutdown_connections 2024-07-29T00:27:13.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.244+0000 7f6805800640 1 -- 172.21.15.175:0/2852153773 wait complete. 2024-07-29T00:27:13.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.245+0000 7f6805800640 1 Processor -- start 2024-07-29T00:27:13.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.246+0000 7f6805800640 1 -- start start 2024-07-29T00:27:13.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.246+0000 7f6805800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f68000741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:13.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.246+0000 7f6805800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f680010b970 con 0x7f6800101520 2024-07-29T00:27:13.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.246+0000 7f67fea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f68000741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:13.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.246+0000 7f67fea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f68000741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44594/0 (socket says 172.21.15.175:44594) 2024-07-29T00:27:13.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.246+0000 7f67fea00640 1 -- 172.21.15.175:0/978104907 learned_addr learned my addr 172.21.15.175:0/978104907 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.247+0000 7f67fea00640 1 -- 172.21.15.175:0/978104907 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68000760d0 con 0x7f6800101520 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.247+0000 7f67fea00640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f68000741e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009230 tx=0x7f67f0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.247+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f003f070 con 0x7f6800101520 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.247+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f003bd40 con 0x7f6800101520 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.247+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f0035030 con 0x7f6800101520 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.247+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6800074780 con 0x7f6800101520 2024-07-29T00:27:13.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.248+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6800074c40 con 0x7f6800101520 2024-07-29T00:27:13.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.249+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f67f0042070 con 0x7f6800101520 2024-07-29T00:27:13.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.249+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6800102a00 con 0x7f6800101520 2024-07-29T00:27:13.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.249+0000 7f67f7e00640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67e40390c0 0x7f67e403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:13.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.249+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f67f006cc30 con 0x7f6800101520 2024-07-29T00:27:13.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.249+0000 7f67f6000640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67e40390c0 0x7f67e403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:13.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.250+0000 7f67f6000640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67e40390c0 0x7f67e403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098c0 tx=0x7f67e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:13.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.254+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f67f0030d90 con 0x7f6800101520 2024-07-29T00:27:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:13.450+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7f6800075780 con 0x7f67e40390c0 2024-07-29T00:27:14.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:14.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:27:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: Deploying cephadm binary to smithi204 2024-07-29T00:27:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:27:15.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:15.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:15.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:15 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:16 smithi175 ceph-mon[32138]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:17.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.107+0000 7f67f7e00640 1 -- 172.21.15.175:0/978104907 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6800075780 con 0x7f67e40390c0 2024-07-29T00:27:17.108 INFO:teuthology.orchestra.run.smithi175.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-07-29T00:27:17.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.112+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67e40390c0 msgr2=0x7f67e403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:17.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.112+0000 7f6805800640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67e40390c0 0x7f67e403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098c0 tx=0x7f67e8006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:17.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.112+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 msgr2=0x7f68000741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:17.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.112+0000 7f6805800640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f68000741e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009230 tx=0x7f67f0003d80 comp rx=0 tx=0).stop 2024-07-29T00:27:17.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.113+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 shutdown_connections 2024-07-29T00:27:17.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.113+0000 7f6805800640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67e40390c0 0x7f67e403b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:17.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.113+0000 7f6805800640 1 --2- 172.21.15.175:0/978104907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6800101520 0x7f68000741e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:17.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.113+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 >> 172.21.15.175:0/978104907 conn(0x7f68000fcfa0 msgr2=0x7f6800108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:17.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.113+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 shutdown_connections 2024-07-29T00:27:17.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:17.113+0000 7f6805800640 1 -- 172.21.15.175:0/978104907 wait complete. 2024-07-29T00:27:17.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch host ls --format=json 2024-07-29T00:27:18.241 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:18.383 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:18.383 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:18 smithi175 ceph-mon[32138]: Added host smithi204 2024-07-29T00:27:18.383 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:18.383 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:18.383 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:18 smithi175 ceph-mon[32138]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:18.384 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:18.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.830+0000 7fbc5e600640 1 -- 172.21.15.175:0/2075518211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 msgr2=0x7fbc581038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.830+0000 7fbc5e600640 1 --2- 172.21.15.175:0/2075518211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc581038f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009880 tx=0x7fbc4002a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.830+0000 7fbc5e600640 1 -- 172.21.15.175:0/2075518211 shutdown_connections 2024-07-29T00:27:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.830+0000 7fbc5e600640 1 --2- 172.21.15.175:0/2075518211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc581038f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.830+0000 7fbc5e600640 1 -- 172.21.15.175:0/2075518211 >> 172.21.15.175:0/2075518211 conn(0x7fbc580fefb0 msgr2=0x7fbc581013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.831+0000 7fbc5e600640 1 -- 172.21.15.175:0/2075518211 shutdown_connections 2024-07-29T00:27:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.831+0000 7fbc5e600640 1 -- 172.21.15.175:0/2075518211 wait complete. 2024-07-29T00:27:18.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.831+0000 7fbc5e600640 1 Processor -- start 2024-07-29T00:27:18.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.832+0000 7fbc5e600640 1 -- start start 2024-07-29T00:27:18.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.832+0000 7fbc5e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc58199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:18.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.832+0000 7fbc5e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5810d960 con 0x7fbc58103510 2024-07-29T00:27:18.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.832+0000 7fbc5d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc58199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:18.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.832+0000 7fbc5d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc58199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:36060/0 (socket says 172.21.15.175:36060) 2024-07-29T00:27:18.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.832+0000 7fbc5d200640 1 -- 172.21.15.175:0/4026136755 learned_addr learned my addr 172.21.15.175:0/4026136755 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:18.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.833+0000 7fbc5d200640 1 -- 172.21.15.175:0/4026136755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc581996c0 con 0x7fbc58103510 2024-07-29T00:27:18.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.833+0000 7fbc5d200640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc58199180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009230 tx=0x7fbc40003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.833+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc4003f070 con 0x7fbc58103510 2024-07-29T00:27:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.833+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4003bd40 con 0x7fbc58103510 2024-07-29T00:27:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.833+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc40035030 con 0x7fbc58103510 2024-07-29T00:27:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.833+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc581999b0 con 0x7fbc58103510 2024-07-29T00:27:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.834+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5819da50 con 0x7fbc58103510 2024-07-29T00:27:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.835+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbc40042070 con 0x7fbc58103510 2024-07-29T00:27:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.835+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc581049f0 con 0x7fbc58103510 2024-07-29T00:27:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.835+0000 7fbc56000640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbc380390c0 0x7fbc3803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.835+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc4006cbb0 con 0x7fbc58103510 2024-07-29T00:27:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.836+0000 7fbc4fe00640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbc380390c0 0x7fbc3803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.836+0000 7fbc4fe00640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbc380390c0 0x7fbc3803b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc480098c0 tx=0x7fbc48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:18.841+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbc40030d60 con 0x7fbc58103510 2024-07-29T00:27:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.036+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc5819a4e0 con 0x7fbc380390c0 2024-07-29T00:27:19.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.038+0000 7fbc56000640 1 -- 172.21.15.175:0/4026136755 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fbc5819a4e0 con 0x7fbc380390c0 2024-07-29T00:27:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:27:19.038 INFO:teuthology.orchestra.run.smithi175.stdout:[{"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-07-29T00:27:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbc380390c0 msgr2=0x7fbc3803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbc380390c0 0x7fbc3803b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc480098c0 tx=0x7fbc48006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 msgr2=0x7fbc58199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc58199180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009230 tx=0x7fbc40003d80 comp rx=0 tx=0).stop 2024-07-29T00:27:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 shutdown_connections 2024-07-29T00:27:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbc380390c0 0x7fbc3803b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 --2- 172.21.15.175:0/4026136755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc58103510 0x7fbc58199180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 >> 172.21.15.175:0/4026136755 conn(0x7fbc580fefb0 msgr2=0x7fbc5810aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 shutdown_connections 2024-07-29T00:27:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:19.041+0000 7fbc5e600640 1 -- 172.21.15.175:0/4026136755 wait complete. 2024-07-29T00:27:19.726 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:19.809 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-29T00:27:19.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd crush tunables default 2024-07-29T00:27:20.262 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:20.696 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:20 smithi175 ceph-mon[32138]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:27:20.696 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:20 smithi175 ceph-mon[32138]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 -- 172.21.15.175:0/768681087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 msgr2=0x7f1934106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 --2- 172.21.15.175:0/768681087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f1934106460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1924009c20 tx=0x7f192402aa90 comp rx=0 tx=0).stop 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 -- 172.21.15.175:0/768681087 shutdown_connections 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 --2- 172.21.15.175:0/768681087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f1934106460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 -- 172.21.15.175:0/768681087 >> 172.21.15.175:0/768681087 conn(0x7f1934075d00 msgr2=0x7f1934076110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 -- 172.21.15.175:0/768681087 shutdown_connections 2024-07-29T00:27:20.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.833+0000 7f1939e00640 1 -- 172.21.15.175:0/768681087 wait complete. 2024-07-29T00:27:20.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.834+0000 7f1939e00640 1 Processor -- start 2024-07-29T00:27:20.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.834+0000 7f1939e00640 1 -- start start 2024-07-29T00:27:20.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.835+0000 7f1939e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f19341004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:20.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.835+0000 7f1939e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f193410b890 con 0x7f1934106080 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.835+0000 7f1938a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f19341004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.835+0000 7f1938a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f19341004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:36070/0 (socket says 172.21.15.175:36070) 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.835+0000 7f1938a00640 1 -- 172.21.15.175:0/3928582669 learned_addr learned my addr 172.21.15.175:0/3928582669 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.835+0000 7f1938a00640 1 -- 172.21.15.175:0/3928582669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19341023e0 con 0x7f1934106080 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.836+0000 7f1938a00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f19341004f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1924009d40 tx=0x7f192402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.836+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1924047070 con 0x7f1934106080 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.836+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924003e40 con 0x7f1934106080 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.836+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1934100a30 con 0x7f1934106080 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.836+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1924033050 con 0x7f1934106080 2024-07-29T00:27:20.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.836+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1934100e90 con 0x7f1934106080 2024-07-29T00:27:20.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.837+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f192404a070 con 0x7f1934106080 2024-07-29T00:27:20.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.837+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1934102840 con 0x7f1934106080 2024-07-29T00:27:20.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.838+0000 7f1931600640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f19180390c0 0x7f191803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:20.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.838+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f192403e030 con 0x7f1934106080 2024-07-29T00:27:20.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.838+0000 7f1933e00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f19180390c0 0x7f191803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:20.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.838+0000 7f1933e00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f19180390c0 0x7f191803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098c0 tx=0x7f191c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:20.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:20.843+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1924004180 con 0x7f1934106080 2024-07-29T00:27:21.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.040+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1934101a20 con 0x7f1934106080 2024-07-29T00:27:21.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.481+0000 7f1931600640 1 -- 172.21.15.175:0/3928582669 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f192404ac60 con 0x7f1934106080 2024-07-29T00:27:21.483 INFO:teuthology.orchestra.run.smithi175.stderr:adjusted tunables profile to default 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.485+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f19180390c0 msgr2=0x7f191803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f19180390c0 0x7f191803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098c0 tx=0x7f191c006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 msgr2=0x7f19341004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f19341004f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1924009d40 tx=0x7f192402fc10 comp rx=0 tx=0).stop 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 shutdown_connections 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f19180390c0 0x7f191803b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 --2- 172.21.15.175:0/3928582669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1934106080 0x7f19341004f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 >> 172.21.15.175:0/3928582669 conn(0x7f1934075d00 msgr2=0x7f1934102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 shutdown_connections 2024-07-29T00:27:21.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:21.486+0000 7f1939e00640 1 -- 172.21.15.175:0/3928582669 wait complete. 2024-07-29T00:27:21.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3928582669' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-29T00:27:21.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:21.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:21.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:21.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:21.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:27:21.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:21.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:22.274 INFO:tasks.cephadm:Adding mon.a on smithi175 2024-07-29T00:27:22.274 INFO:tasks.cephadm:Adding mon.b on smithi181 2024-07-29T00:27:22.274 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-07-29T00:27:22.275 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch apply mon '3;smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c' 2024-07-29T00:27:22.520 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:22 smithi175 ceph-mon[32138]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:27:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:22 smithi175 ceph-mon[32138]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:22 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3928582669' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-29T00:27:22.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:22 smithi175 ceph-mon[32138]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:22.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:22 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:22.965 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 -- 172.21.15.204:0/3105376001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 msgr2=0x7f43941066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:22.965 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 --2- 172.21.15.204:0/3105376001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941066a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4388009880 tx=0x7f438802a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:22.965 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 -- 172.21.15.204:0/3105376001 shutdown_connections 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 --2- 172.21.15.204:0/3105376001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941066a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 -- 172.21.15.204:0/3105376001 >> 172.21.15.204:0/3105376001 conn(0x7f4394075f40 msgr2=0x7f4394076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 -- 172.21.15.204:0/3105376001 shutdown_connections 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 -- 172.21.15.204:0/3105376001 wait complete. 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 Processor -- start 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.965+0000 7f439ab37640 1 -- start start 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.966+0000 7f439ab37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:22.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.966+0000 7f439ab37640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f439410b9d0 con 0x7f43941062c0 2024-07-29T00:27:22.967 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.966+0000 7f43988d3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:22.967 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.966+0000 7f43988d3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.204:33670/0 (socket says 172.21.15.204:33670) 2024-07-29T00:27:22.967 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.966+0000 7f43988d3640 1 -- 172.21.15.204:0/3184914899 learned_addr learned my addr 172.21.15.204:0/3184914899 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:27:22.967 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.967+0000 7f43988d3640 1 -- 172.21.15.204:0/3184914899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43941023e0 con 0x7f43941062c0 2024-07-29T00:27:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.967+0000 7f43988d3640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941004f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4388009230 tx=0x7f4388003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.967+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f438803f070 con 0x7f43941062c0 2024-07-29T00:27:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.968+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4394100a30 con 0x7f43941062c0 2024-07-29T00:27:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.968+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438803bd40 con 0x7f43941062c0 2024-07-29T00:27:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.968+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4388035030 con 0x7f43941062c0 2024-07-29T00:27:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.968+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4394100ef0 con 0x7f43941062c0 2024-07-29T00:27:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.969+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4394102a20 con 0x7f43941062c0 2024-07-29T00:27:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.969+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4388042070 con 0x7f43941062c0 2024-07-29T00:27:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.969+0000 7f4391ffb640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4384039060 0x7f438403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.970+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f438806cd20 con 0x7f43941062c0 2024-07-29T00:27:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.970+0000 7f4393fff640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4384039060 0x7f438403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:22.971 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.971+0000 7f4393fff640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4384039060 0x7f438403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f43800098c0 tx=0x7f4380006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:22.982 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:22.983+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4388030d60 con 0x7f43941062c0 2024-07-29T00:27:23.168 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.168+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7f4394101ae0 con 0x7f4384039060 2024-07-29T00:27:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.181+0000 7f4391ffb640 1 -- 172.21.15.204:0/3184914899 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4394101ae0 con 0x7f4384039060 2024-07-29T00:27:23.181 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4384039060 msgr2=0x7f438403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4384039060 0x7f438403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f43800098c0 tx=0x7f4380006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 msgr2=0x7f43941004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941004f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4388009230 tx=0x7f4388003d80 comp rx=0 tx=0).stop 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 shutdown_connections 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4384039060 0x7f438403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 --2- 172.21.15.204:0/3184914899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43941062c0 0x7f43941004f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 >> 172.21.15.204:0/3184914899 conn(0x7f4394075f40 msgr2=0x7f4394102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:23.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 shutdown_connections 2024-07-29T00:27:23.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:23.183+0000 7f439ab37640 1 -- 172.21.15.204:0/3184914899 wait complete. 2024-07-29T00:27:23.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:23.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:23.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:27:23.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:23.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:23.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:23.809 DEBUG:teuthology.orchestra.run.smithi181:mon.b> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.b.service 2024-07-29T00:27:23.812 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.c.service 2024-07-29T00:27:23.814 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-29T00:27:23.814 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph mon dump -f json 2024-07-29T00:27:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:24 smithi175 ceph-mon[32138]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:24 smithi175 ceph-mon[32138]: Saving service mon spec with placement smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c;count:3 2024-07-29T00:27:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:24 smithi175 ceph-mon[32138]: Deploying daemon mon.c on smithi204 2024-07-29T00:27:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:24 smithi175 ceph-mon[32138]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:25.989 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:27:26.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.590+0000 7ff45ff31640 1 -- 172.21.15.204:0/2790764143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 msgr2=0x7ff4581008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.590+0000 7ff45ff31640 1 --2- 172.21.15.204:0/2790764143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff4581008b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff454009880 tx=0x7ff45402a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 -- 172.21.15.204:0/2790764143 shutdown_connections 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 --2- 172.21.15.204:0/2790764143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff4581008b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 -- 172.21.15.204:0/2790764143 >> 172.21.15.204:0/2790764143 conn(0x7ff4580fbf70 msgr2=0x7ff4580fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 -- 172.21.15.204:0/2790764143 shutdown_connections 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 -- 172.21.15.204:0/2790764143 wait complete. 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 Processor -- start 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.591+0000 7ff45ff31640 1 -- start start 2024-07-29T00:27:26.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.592+0000 7ff45ff31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff458195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:26.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.592+0000 7ff45ff31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff45810a920 con 0x7ff4581004d0 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.592+0000 7ff45dccd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff458195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.592+0000 7ff45dccd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff458195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.204:33702/0 (socket says 172.21.15.204:33702) 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.592+0000 7ff45dccd640 1 -- 172.21.15.204:0/1998555961 learned_addr learned my addr 172.21.15.204:0/1998555961 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.593+0000 7ff45dccd640 1 -- 172.21.15.204:0/1998555961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff458196350 con 0x7ff4581004d0 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.593+0000 7ff45dccd640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff458195e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff454009230 tx=0x7ff454003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.594+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff45403f070 con 0x7ff4581004d0 2024-07-29T00:27:26.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.594+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff458196640 con 0x7ff4581004d0 2024-07-29T00:27:26.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.594+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45403bc80 con 0x7ff4581004d0 2024-07-29T00:27:26.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.594+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45819a6e0 con 0x7ff4581004d0 2024-07-29T00:27:26.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.594+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff454049060 con 0x7ff4581004d0 2024-07-29T00:27:26.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.595+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c0052c0 con 0x7ff4581004d0 2024-07-29T00:27:26.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.595+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff454042070 con 0x7ff4581004d0 2024-07-29T00:27:26.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.595+0000 7ff44effd640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff444039060 0x7ff44403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:26.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.595+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff45406cc20 con 0x7ff4581004d0 2024-07-29T00:27:26.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.596+0000 7ff45d4cc640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff444039060 0x7ff44403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:26.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.597+0000 7ff45d4cc640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff444039060 0x7ff44403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4480098c0 tx=0x7ff448006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:26.610 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.610+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff454030df0 con 0x7ff4581004d0 2024-07-29T00:27:26.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:26 smithi175 ceph-mon[32138]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:26.859 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.859+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff43c0055b0 con 0x7ff4581004d0 2024-07-29T00:27:26.860 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.861+0000 7ff44effd640 1 -- 172.21.15.204:0/1998555961 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7ff454040260 con 0x7ff4581004d0 2024-07-29T00:27:26.860 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:27:26.861 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","modified":"2024-07-29T00:25:49.691743Z","created":"2024-07-29T00:25:49.691743Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-29T00:27:26.861 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff444039060 msgr2=0x7ff44403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff444039060 0x7ff44403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4480098c0 tx=0x7ff448006db0 comp rx=0 tx=0).stop 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 msgr2=0x7ff458195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff458195e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff454009230 tx=0x7ff454003d40 comp rx=0 tx=0).stop 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 shutdown_connections 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff444039060 0x7ff44403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 --2- 172.21.15.204:0/1998555961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4581004d0 0x7ff458195e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 >> 172.21.15.204:0/1998555961 conn(0x7ff4580fbf70 msgr2=0x7ff458107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 shutdown_connections 2024-07-29T00:27:26.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:26.863+0000 7ff45ff31640 1 -- 172.21.15.204:0/1998555961 wait complete. 2024-07-29T00:27:27.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:27 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/1998555961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:27.846 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).mds e1 print_map 2024-07-29T00:27:27.846 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout: btime 2024-07-29T00:25:53:731837+0000 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-29T00:27:27.847 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-29T00:27:28.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-29T00:27:28.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:27:28.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Deploying cephadm binary to smithi204 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Added host smithi204 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3928582669' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:28.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3928582669' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Saving service mon spec with placement smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c;count:3 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: Deploying daemon mon.c on smithi204 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/1998555961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:28.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 ceph-mon[37368]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-29T00:27:28.343 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 bash[37349]: 0a892ddf45e65019ddbb803ad62840844f03b2a54810d137a6dfe36be046f4e7 2024-07-29T00:27:28.343 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:27 smithi204 systemd[1]: Started Ceph mon.c for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:27:28.362 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-29T00:27:28.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph mon dump -f json 2024-07-29T00:27:28.614 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: mon.a calling monitor election 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: mon.c calling monitor election 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: monmap epoch 2 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: last_changed 2024-07-29T00:27:27.853679+0000 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: min_mon_release 19 (squid) 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: election_strategy: 1 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: fsmap 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: mgrmap e13: x(active, since 43s) 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: overall HEALTH_OK 2024-07-29T00:27:33.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:33.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:33.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:33.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:33.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: mon.a calling monitor election 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: mon.c calling monitor election 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: monmap epoch 2 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: last_changed 2024-07-29T00:27:27.853679+0000 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: min_mon_release 19 (squid) 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: election_strategy: 1 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: fsmap 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:33.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: mgrmap e13: x(active, since 43s) 2024-07-29T00:27:33.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: overall HEALTH_OK 2024-07-29T00:27:33.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:33.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:33.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:33.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:33.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 -- 172.21.15.204:0/195599031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548003530 msgr2=0x7f15480059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 --2- 172.21.15.204:0/195599031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548003530 0x7f15480059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f155c1016e0 tx=0x7f154c02c490 comp rx=0 tx=0).stop 2024-07-29T00:27:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 -- 172.21.15.204:0/195599031 shutdown_connections 2024-07-29T00:27:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 --2- 172.21.15.204:0/195599031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548003530 0x7f15480059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 -- 172.21.15.204:0/195599031 >> 172.21.15.204:0/195599031 conn(0x7f155c0fbf10 msgr2=0x7f155c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 -- 172.21.15.204:0/195599031 shutdown_connections 2024-07-29T00:27:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.436+0000 7f1561207640 1 -- 172.21.15.204:0/195599031 wait complete. 2024-07-29T00:27:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.437+0000 7f1561207640 1 Processor -- start 2024-07-29T00:27:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.437+0000 7f1561207640 1 -- start start 2024-07-29T00:27:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f1561207640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f1561207640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f155c102490 0x7f155c1028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:33.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f1561207640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f155c108360 con 0x7f155c102490 2024-07-29T00:27:33.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f1561207640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1081e0 con 0x7f1548003530 2024-07-29T00:27:33.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:33.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36954/0 (socket says 172.21.15.204:36954) 2024-07-29T00:27:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 -- 172.21.15.204:0/4139545155 learned_addr learned my addr 172.21.15.204:0/4139545155 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:27:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 -- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 msgr2=0x7f155c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-29T00:27:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 -- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 msgr2=0x7f155c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-29T00:27:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-29T00:27:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f155ad9d640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:27:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.438+0000 7f1553fff640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f155c102490 0x7f155c1028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:33.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.439+0000 7f1553fff640 1 -- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 msgr2=0x7f155c101f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.439+0000 7f1553fff640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.439+0000 7f1553fff640 1 -- 172.21.15.204:0/4139545155 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c103e40 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.439+0000 7f1553fff640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f155c102490 0x7f155c1028f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f154400b400 tx=0x7f154400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.440+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1544011630 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.440+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c1a3250 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.440+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c1a37e0 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.440+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1544011cb0 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.440+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1544019e00 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.441+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1548002a90 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.441+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1544018020 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.442+0000 7f1558d99640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f153c038e00 0x7f153c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.442+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f154404b490 con 0x7f155c102490 2024-07-29T00:27:33.444 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.442+0000 7f155ad9d640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f153c038e00 0x7f153c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:33.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.443+0000 7f155ad9d640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f153c038e00 0x7f153c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f154c012c10 tx=0x7f154c0323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:33.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.456+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f154401f030 con 0x7f155c102490 2024-07-29T00:27:33.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.694+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1548002d80 con 0x7f155c102490 2024-07-29T00:27:33.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.695+0000 7f1558d99640 1 -- 172.21.15.204:0/4139545155 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f154400e4c0 con 0x7f155c102490 2024-07-29T00:27:33.695 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:27:33.695 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","modified":"2024-07-29T00:27:27.853679Z","created":"2024-07-29T00:25:49.691743Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-29T00:27:33.695 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f153c038e00 msgr2=0x7f153c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f153c038e00 0x7f153c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f154c012c10 tx=0x7f154c0323a0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f155c102490 msgr2=0x7f155c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f155c102490 0x7f155c1028f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f154400b400 tx=0x7f154400b8c0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 shutdown_connections 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f155c102490 0x7f155c1028f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f153c038e00 0x7f153c03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 --2- 172.21.15.204:0/4139545155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548003530 0x7f155c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:33.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 >> 172.21.15.204:0/4139545155 conn(0x7f155c0fbf10 msgr2=0x7f155c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:33.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 shutdown_connections 2024-07-29T00:27:33.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:33.698+0000 7f1561207640 1 -- 172.21.15.204:0/4139545155 wait complete. 2024-07-29T00:27:34.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:33 smithi204 ceph-mon[37368]: Deploying daemon mon.b on smithi181 2024-07-29T00:27:34.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:33 smithi204 ceph-mon[37368]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:34.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:33 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/4139545155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:34.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:33 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:34.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:33 smithi175 ceph-mon[32138]: Deploying daemon mon.b on smithi181 2024-07-29T00:27:34.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:33 smithi175 ceph-mon[32138]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:34.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:33 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/4139545155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:34.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:33 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:35.263 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-29T00:27:35.264 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph mon dump -f json 2024-07-29T00:27:35.522 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:27:35.894 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 -- 172.21.15.204:0/750165392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 msgr2=0x7f43e81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:35.894 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 --2- 172.21.15.204:0/750165392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e81008b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009880 tx=0x7f43e402a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 -- 172.21.15.204:0/750165392 shutdown_connections 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 --2- 172.21.15.204:0/750165392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e81008b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 -- 172.21.15.204:0/750165392 >> 172.21.15.204:0/750165392 conn(0x7f43e80fbf70 msgr2=0x7f43e80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 -- 172.21.15.204:0/750165392 shutdown_connections 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 -- 172.21.15.204:0/750165392 wait complete. 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 Processor -- start 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.893+0000 7f43ef980640 1 -- start start 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ef980640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e80fc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ef980640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ef980640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43e810a920 con 0x7f43e81004d0 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ef980640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43e810a7a0 con 0x7f43e80fc960 2024-07-29T00:27:35.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36960/0 (socket says 172.21.15.204:36960) 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 -- 172.21.15.204:0/3657490766 learned_addr learned my addr 172.21.15.204:0/3657490766 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 -- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 msgr2=0x7f43e80fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 -- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 msgr2=0x7f43e80fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ecf1b640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.894+0000 7f43ed71c640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e80fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.895+0000 7f43ed71c640 1 -- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 msgr2=0x7f43e80fcdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.895+0000 7f43ed71c640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:35.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.895+0000 7f43ed71c640 1 -- 172.21.15.204:0/3657490766 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e80fe310 con 0x7f43e81004d0 2024-07-29T00:27:35.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.895+0000 7f43ed71c640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e80fc420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000c00 tx=0x7f43e4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:35.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.895+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f43e4035070 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.896+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a3380 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.896+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e4032e40 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.896+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81a3910 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.896+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f43e403ba30 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.896+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e81019b0 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.897+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f43e4042030 con 0x7f43e81004d0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.897+0000 7f43de7fc640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f43d4038e00 0x7f43d403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.898+0000 7f43ecf1b640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f43d4038e00 0x7f43d403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:35.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.898+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f43e4040070 con 0x7f43e81004d0 2024-07-29T00:27:35.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.899+0000 7f43ecf1b640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f43d4038e00 0x7f43d403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009100 tx=0x7f43d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:35.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:35.913+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f43e4044410 con 0x7f43e81004d0 2024-07-29T00:27:36.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.139+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f43e8100930 con 0x7f43e81004d0 2024-07-29T00:27:36.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.140+0000 7f43de7fc640 1 -- 172.21.15.204:0/3657490766 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f43e4038e10 con 0x7f43e81004d0 2024-07-29T00:27:36.140 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:27:36.140 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","modified":"2024-07-29T00:27:27.853679Z","created":"2024-07-29T00:25:49.691743Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-29T00:27:36.140 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f43d4038e00 msgr2=0x7f43d403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f43d4038e00 0x7f43d403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009100 tx=0x7f43d8008040 comp rx=0 tx=0).stop 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 msgr2=0x7f43e80fc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e80fc420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000c00 tx=0x7f43e4032c50 comp rx=0 tx=0).stop 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 shutdown_connections 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f43d4038e00 0x7f43d403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43e80fc960 0x7f43e80fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 --2- 172.21.15.204:0/3657490766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43e81004d0 0x7f43e80fc420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 >> 172.21.15.204:0/3657490766 conn(0x7f43e80fbf70 msgr2=0x7f43e8069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:36.143 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 shutdown_connections 2024-07-29T00:27:36.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:36.143+0000 7f43ef980640 1 -- 172.21.15.204:0/3657490766 wait complete. 2024-07-29T00:27:36.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:36 smithi181 systemd[1]: Started Ceph mon.b for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:27:36.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:36 smithi175 ceph-mon[32138]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:36.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:36 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/3657490766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:36 smithi204 ceph-mon[37368]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:36 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/3657490766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:37.652 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-29T00:27:37.653 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph mon dump -f json 2024-07-29T00:27:37.910 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:27:41.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 -- 172.21.15.204:0/796248942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d8003530 msgr2=0x7f18d80059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:41.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/796248942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d8003530 0x7f18d80059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18ec1027e0 tx=0x7f18e802c3f0 comp rx=0 tx=0).stop 2024-07-29T00:27:41.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 -- 172.21.15.204:0/796248942 shutdown_connections 2024-07-29T00:27:41.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/796248942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d8003530 0x7f18d80059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:41.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 -- 172.21.15.204:0/796248942 >> 172.21.15.204:0/796248942 conn(0x7f18ec0fd010 msgr2=0x7f18ec0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:41.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 -- 172.21.15.204:0/796248942 shutdown_connections 2024-07-29T00:27:41.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.929+0000 7f18f3ff9640 1 -- 172.21.15.204:0/796248942 wait complete. 2024-07-29T00:27:41.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 Processor -- start 2024-07-29T00:27:41.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 -- start start 2024-07-29T00:27:41.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 0x7f18ec19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ec19be30 0x7f18ec19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec109410 con 0x7f18ec19be30 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec109290 con 0x7f18ec197560 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.930+0000 7f18f3ff9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec109590 con 0x7f18d8003530 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:60554/0 (socket says 172.21.15.204:60554) 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 -- 172.21.15.204:0/3841611629 learned_addr learned my addr 172.21.15.204:0/3841611629 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 msgr2=0x7f18ec197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 msgr2=0x7f18ec197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-29T00:27:41.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1d95640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1594640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 0x7f18ec19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1594640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 msgr2=0x7f18ec19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1594640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 msgr2=0x7f18ec19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1594640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 0x7f18ec19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f1594640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 0x7f18ec19b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.931+0000 7f18f2596640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ec19be30 0x7f18ec19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f2596640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 msgr2=0x7f18ec197020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f2596640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f2596640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 msgr2=0x7f18ec19b8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f2596640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 0x7f18ec19b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f2596640 1 -- 172.21.15.204:0/3841611629 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec1a3ba0 con 0x7f18ec19be30 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f2596640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ec19be30 0x7f18ec19c2e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18e400bd50 tx=0x7f18e400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:41.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4019070 con 0x7f18ec19be30 2024-07-29T00:27:41.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e4014920 con 0x7f18ec19be30 2024-07-29T00:27:41.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec1a3dd0 con 0x7f18ec19be30 2024-07-29T00:27:41.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.932+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4012990 con 0x7f18ec19be30 2024-07-29T00:27:41.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.933+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a4280 con 0x7f18ec19be30 2024-07-29T00:27:41.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.934+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f18e401c070 con 0x7f18ec19be30 2024-07-29T00:27:41.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.934+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec10a540 con 0x7f18ec19be30 2024-07-29T00:27:41.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.934+0000 7f18e2ffd640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f18d00394f0 0x7f18d003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:41.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.934+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f18e404be70 con 0x7f18ec19be30 2024-07-29T00:27:41.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.934+0000 7f18f1d95640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f18d00394f0 0x7f18d003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:41.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.936+0000 7f18f1d95640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f18d00394f0 0x7f18d003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18e8014a90 tx=0x7f18e8005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:41.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:41.948+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f18e4013e80 con 0x7f18ec19be30 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: mon.c calling monitor election 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: mon.a calling monitor election 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:41.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: monmap epoch 3 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: last_changed 2024-07-29T00:27:36.614890+0000 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: min_mon_release 19 (squid) 2024-07-29T00:27:41.950 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: election_strategy: 1 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: fsmap 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: mgrmap e13: x(active, since 52s) 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: overall HEALTH_OK 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:41.951 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:42.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: mon.c calling monitor election 2024-07-29T00:27:42.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: mon.a calling monitor election 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: monmap epoch 3 2024-07-29T00:27:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: last_changed 2024-07-29T00:27:36.614890+0000 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: min_mon_release 19 (squid) 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: election_strategy: 1 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: fsmap 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: mgrmap e13: x(active, since 52s) 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: overall HEALTH_OK 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:42.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:42.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:42.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:42.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.194+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f18ec02d5a0 con 0x7f18ec19be30 2024-07-29T00:27:42.195 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.196+0000 7f18e2ffd640 1 -- 172.21.15.204:0/3841611629 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f18e4011df0 con 0x7f18ec19be30 2024-07-29T00:27:42.196 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:27:42.196 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","modified":"2024-07-29T00:27:36.614890Z","created":"2024-07-29T00:25:49.691743Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-29T00:27:42.196 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-07-29T00:27:42.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f18d00394f0 msgr2=0x7f18d003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:42.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f18d00394f0 0x7f18d003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18e8014a90 tx=0x7f18e8005f60 comp rx=0 tx=0).stop 2024-07-29T00:27:42.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ec19be30 msgr2=0x7f18ec19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:42.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ec19be30 0x7f18ec19c2e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18e400bd50 tx=0x7f18e400be40 comp rx=0 tx=0).stop 2024-07-29T00:27:42.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 shutdown_connections 2024-07-29T00:27:42.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ec19be30 0x7f18ec19c2e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:42.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18ec197560 0x7f18ec19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:42.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f18d00394f0 0x7f18d003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:42.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 --2- 172.21.15.204:0/3841611629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18d8003530 0x7f18ec197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:42.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 >> 172.21.15.204:0/3841611629 conn(0x7f18ec0fd010 msgr2=0x7f18ec0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:42.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 shutdown_connections 2024-07-29T00:27:42.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:42.198+0000 7f18f3ff9640 1 -- 172.21.15.204:0/3841611629 wait complete. 2024-07-29T00:27:42.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:42.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:42 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/3841611629' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:42.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:42.825 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-29T00:27:42.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph config generate-minimal-conf 2024-07-29T00:27:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:42 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/3841611629' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:43.271 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:43.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.858+0000 7f25a2a00640 1 -- 172.21.15.175:0/447799938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c101520 msgr2=0x7f259c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:43.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.858+0000 7f25a2a00640 1 --2- 172.21.15.175:0/447799938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c101520 0x7f259c101900 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f258c009880 tx=0x7f258c02a5d0 comp rx=0 tx=0).stop 2024-07-29T00:27:43.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.859+0000 7f25a2a00640 1 -- 172.21.15.175:0/447799938 shutdown_connections 2024-07-29T00:27:43.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.859+0000 7f25a2a00640 1 --2- 172.21.15.175:0/447799938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c101520 0x7f259c101900 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:43.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.859+0000 7f25a2a00640 1 -- 172.21.15.175:0/447799938 >> 172.21.15.175:0/447799938 conn(0x7f259c0fcfa0 msgr2=0x7f259c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:43.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.859+0000 7f25a2a00640 1 -- 172.21.15.175:0/447799938 shutdown_connections 2024-07-29T00:27:43.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.859+0000 7f25a2a00640 1 -- 172.21.15.175:0/447799938 wait complete. 2024-07-29T00:27:43.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.860+0000 7f25a2a00640 1 Processor -- start 2024-07-29T00:27:43.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.860+0000 7f25a2a00640 1 -- start start 2024-07-29T00:27:43.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f25a2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 0x7f259c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:43.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f25a2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c199860 0x7f259c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:43.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f25a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c19e130 0x7f259c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:43.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f25a2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f259c10b970 con 0x7f259c19e130 2024-07-29T00:27:43.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f25a2a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f259c10b7f0 con 0x7f259c101520 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f25a2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f259c10baf0 con 0x7f259c199860 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f259be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 0x7f259c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f259be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 0x7f259c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:34288/0 (socket says 172.21.15.175:34288) 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f259be00640 1 -- 172.21.15.175:0/326911440 learned_addr learned my addr 172.21.15.175:0/326911440 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.861+0000 7f259b400640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c199860 0x7f259c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f25a0a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c19e130 0x7f259c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259b400640 1 -- 172.21.15.175:0/326911440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 msgr2=0x7f259c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259b400640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 0x7f259c199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259b400640 1 -- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c19e130 msgr2=0x7f259c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:43.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259b400640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c19e130 0x7f259c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259b400640 1 -- 172.21.15.175:0/326911440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c1a5ea0 con 0x7f259c199860 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259be00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 0x7f259c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.862+0000 7f259b400640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c199860 0x7f259c19dbf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f258800b450 tx=0x7f258800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.863+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588004230 con 0x7f259c199860 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.863+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25880043d0 con 0x7f259c199860 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.863+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c1a60d0 con 0x7f259c199860 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.863+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588003cb0 con 0x7f259c199860 2024-07-29T00:27:43.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.863+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c1a6630 con 0x7f259c199860 2024-07-29T00:27:43.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.864+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f258800f050 con 0x7f259c199860 2024-07-29T00:27:43.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.864+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c101980 con 0x7f259c199860 2024-07-29T00:27:43.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.864+0000 7f2598c00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25800394f0 0x7f258003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:43.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.864+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f258804bd30 con 0x7f259c199860 2024-07-29T00:27:43.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.865+0000 7f259be00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25800394f0 0x7f258003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:43.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.865+0000 7f259be00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25800394f0 0x7f258003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f258c0096c0 tx=0x7f258c02aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:43.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:43.870+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f258800d040 con 0x7f259c199860 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: mon.c calling monitor election 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: mon.a calling monitor election 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: monmap epoch 3 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: last_changed 2024-07-29T00:27:36.614890+0000 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: min_mon_release 19 (squid) 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: election_strategy: 1 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:44.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: fsmap 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: mgrmap e13: x(active, since 52s) 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: overall HEALTH_OK 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/3841611629' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-29T00:27:44.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:44.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.074+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f259c1a6990 con 0x7f259c199860 2024-07-29T00:27:44.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.074+0000 7f2598c00640 1 -- 172.21.15.175:0/326911440 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f2588011d00 con 0x7f259c199860 2024-07-29T00:27:44.075 INFO:teuthology.orchestra.run.smithi175.stdout:# minimal ceph.conf for f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:44.075 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-07-29T00:27:44.075 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:44.075 INFO:teuthology.orchestra.run.smithi175.stdout: mon_host = [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-07-29T00:27:44.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25800394f0 msgr2=0x7f258003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25800394f0 0x7f258003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f258c0096c0 tx=0x7f258c02aad0 comp rx=0 tx=0).stop 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c199860 msgr2=0x7f259c19dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c199860 0x7f259c19dbf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f258800b450 tx=0x7f258800b910 comp rx=0 tx=0).stop 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 shutdown_connections 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f259c19e130 0x7f259c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25800394f0 0x7f258003b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c199860 0x7f259c19dbf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 --2- 172.21.15.175:0/326911440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c101520 0x7f259c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 >> 172.21.15.175:0/326911440 conn(0x7f259c0fcfa0 msgr2=0x7f259c108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 shutdown_connections 2024-07-29T00:27:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:44.077+0000 7f25a2a00640 1 -- 172.21.15.175:0/326911440 wait complete. 2024-07-29T00:27:44.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: mon.b calling monitor election 2024-07-29T00:27:44.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: mon.b calling monitor election 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: mon.a calling monitor election 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: mon.c calling monitor election 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: monmap epoch 3 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: last_changed 2024-07-29T00:27:36.614890+0000 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:44.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: min_mon_release 19 (squid) 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: election_strategy: 1 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: fsmap 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: mgrmap e13: x(active, since 54s) 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: overall HEALTH_OK 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/326911440' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:44.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: mon.b calling monitor election 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: mon.b calling monitor election 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: mon.a calling monitor election 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: mon.c calling monitor election 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: monmap epoch 3 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: last_changed 2024-07-29T00:27:36.614890+0000 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: min_mon_release 19 (squid) 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: election_strategy: 1 2024-07-29T00:27:44.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: fsmap 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: mgrmap e13: x(active, since 54s) 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: overall HEALTH_OK 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/326911440' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:44.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:44.884 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-29T00:27:44.884 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:27:44.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-07-29T00:27:44.965 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:27:44.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:45.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: mon.b calling monitor election 2024-07-29T00:27:45.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:45.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:27:45.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:27:45.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:27:45.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: mon.b calling monitor election 2024-07-29T00:27:45.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: mon.a calling monitor election 2024-07-29T00:27:45.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: mon.c calling monitor election 2024-07-29T00:27:45.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-29T00:27:45.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: monmap epoch 3 2024-07-29T00:27:45.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:27:45.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: last_changed 2024-07-29T00:27:36.614890+0000 2024-07-29T00:27:45.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: created 2024-07-29T00:25:49.691743+0000 2024-07-29T00:27:45.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: min_mon_release 19 (squid) 2024-07-29T00:27:45.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: election_strategy: 1 2024-07-29T00:27:45.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: fsmap 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: osdmap e4: 0 total, 0 up, 0 in 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: mgrmap e13: x(active, since 54s) 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: overall HEALTH_OK 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.004 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.004 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/326911440' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:45.004 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.074 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:27:45.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-07-29T00:27:45.100 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:27:45.101 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:45.165 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:27:45.165 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-07-29T00:27:45.192 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:27:45.193 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:27:45.257 INFO:tasks.cephadm:Adding mgr.x on smithi175 2024-07-29T00:27:45.257 INFO:tasks.cephadm:Adding mgr.y on smithi181 2024-07-29T00:27:45.257 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-07-29T00:27:45.258 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch apply mgr '3;smithi175=x;smithi181=y;smithi204=z' 2024-07-29T00:27:45.542 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:27:45.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:45.771 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.773 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.773 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.773 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:45.773 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:45.773 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:45.773 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:45.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.954+0000 7f02fffff640 1 -- 172.21.15.204:0/228877982 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0036360 con 0x7f0300101e60 2024-07-29T00:27:45.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.954+0000 7f030731b640 1 -- 172.21.15.204:0/228877982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300101e60 msgr2=0x7f0300104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.954+0000 7f030731b640 1 --2- 172.21.15.204:0/228877982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300101e60 0x7f0300104250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009c20 tx=0x7f02f002aa90 comp rx=0 tx=0).stop 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.954+0000 7f030731b640 1 -- 172.21.15.204:0/228877982 shutdown_connections 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.954+0000 7f030731b640 1 --2- 172.21.15.204:0/228877982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300101e60 0x7f0300104250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.954+0000 7f030731b640 1 -- 172.21.15.204:0/228877982 >> 172.21.15.204:0/228877982 conn(0x7f03000fbaa0 msgr2=0x7f03000fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 -- 172.21.15.204:0/228877982 shutdown_connections 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 -- 172.21.15.204:0/228877982 wait complete. 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 Processor -- start 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 -- start start 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0300101e60 0x7f0300196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300197180 0x7f030019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:45.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 0x7f030019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0300107420 con 0x7f0300197180 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f03001072a0 con 0x7f0300101e60 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f030731b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03001075a0 con 0x7f0300197600 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.955+0000 7f03058b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 0x7f030019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 0x7f030019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:60556/0 (socket says 172.21.15.204:60556) 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 -- 172.21.15.204:0/1190634528 learned_addr learned my addr 172.21.15.204:0/1190634528 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03050b7640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0300101e60 0x7f0300196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 -- 172.21.15.204:0/1190634528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0300101e60 msgr2=0x7f0300196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0300101e60 0x7f0300196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 -- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300197180 msgr2=0x7f030019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:45.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03048b6640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300197180 0x7f030019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300197180 0x7f030019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 -- 172.21.15.204:0/1190634528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030019f000 con 0x7f0300197600 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f03058b8640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 0x7f030019e740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b450 tx=0x7f02f800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8015020 con 0x7f0300197600 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f8011d70 con 0x7f0300197600 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030019f2f0 con 0x7f0300197600 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f800f340 con 0x7f0300197600 2024-07-29T00:27:45.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.956+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030019f860 con 0x7f0300197600 2024-07-29T00:27:45.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.957+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f02f8018020 con 0x7f0300197600 2024-07-29T00:27:45.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.957+0000 7f02fe7fc640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f02e80394f0 0x7f02e803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:45.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.957+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03001901e0 con 0x7f0300197600 2024-07-29T00:27:45.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.957+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f02f804bfa0 con 0x7f0300197600 2024-07-29T00:27:45.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.958+0000 7f03050b7640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f02e80394f0 0x7f02e803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:45.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.959+0000 7f03050b7640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f02e80394f0 0x7f02e803b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000f80 tx=0x7f02f003a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:45.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:45.962+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f02f801f020 con 0x7f0300197600 2024-07-29T00:27:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:27:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:46.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:46.142 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.142+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi175=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f03000623b0 con 0x7f02e80394f0 2024-07-29T00:27:46.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.155+0000 7f02fe7fc640 1 -- 172.21.15.204:0/1190634528 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f03000623b0 con 0x7f02e80394f0 2024-07-29T00:27:46.155 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-07-29T00:27:46.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f02e80394f0 msgr2=0x7f02e803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:46.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f02e80394f0 0x7f02e803b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000f80 tx=0x7f02f003a040 comp rx=0 tx=0).stop 2024-07-29T00:27:46.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 msgr2=0x7f030019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:46.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 0x7f030019e740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b450 tx=0x7f02f800b910 comp rx=0 tx=0).stop 2024-07-29T00:27:46.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 shutdown_connections 2024-07-29T00:27:46.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f02e80394f0 0x7f02e803b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:46.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0300197600 0x7f030019e740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:46.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0300197180 0x7f030019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:46.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 --2- 172.21.15.204:0/1190634528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0300101e60 0x7f0300196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:46.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 >> 172.21.15.204:0/1190634528 conn(0x7f03000fbaa0 msgr2=0x7f03000fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:46.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 shutdown_connections 2024-07-29T00:27:46.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:27:46.157+0000 7f030731b640 1 -- 172.21.15.204:0/1190634528 wait complete. 2024-07-29T00:27:46.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:46 smithi175 ceph-mon[32138]: Reconfiguring mon.a (unknown last config time)... 2024-07-29T00:27:46.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:46 smithi175 ceph-mon[32138]: Reconfiguring daemon mon.a on smithi175 2024-07-29T00:27:46.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:46 smithi175 ceph-mon[32138]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:46.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.709 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:46 smithi204 ceph-mon[37368]: Reconfiguring mon.a (unknown last config time)... 2024-07-29T00:27:46.709 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:46 smithi204 ceph-mon[37368]: Reconfiguring daemon mon.a on smithi175 2024-07-29T00:27:46.709 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:46 smithi204 ceph-mon[37368]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:46.710 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:46.737 DEBUG:teuthology.orchestra.run.smithi181:mgr.y> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.y.service 2024-07-29T00:27:46.739 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.z.service 2024-07-29T00:27:46.742 INFO:tasks.cephadm:Deploying OSDs... 2024-07-29T00:27:46.742 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:27:46.742 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-07-29T00:27:46.787 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-29T00:27:46.787 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-07-29T00:27:46.867 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-29T00:27:46.867 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:46.867 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-07-29T00:27:46.868 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:46.868 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:46.868 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-29 00:27:03.795170736 +0000 2024-07-29T00:27:46.868 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-29 00:21:54.842872981 +0000 2024-07-29T00:27:46.868 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-29 00:21:54.842872981 +0000 2024-07-29T00:27:46.868 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-29 00:21:54.842872981 +0000 2024-07-29T00:27:46.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-29T00:27:46.971 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-29T00:27:46.971 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-29T00:27:46.971 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000334768 s, 1.5 MB/s 2024-07-29T00:27:46.974 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-29T00:27:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:46 smithi181 ceph-mon[36706]: Reconfiguring mon.a (unknown last config time)... 2024-07-29T00:27:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:46 smithi181 ceph-mon[36706]: Reconfiguring daemon mon.a on smithi175 2024-07-29T00:27:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:46 smithi181 ceph-mon[36706]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:47.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.021 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-29 00:27:03.899169158 +0000 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-29 00:21:55.415864239 +0000 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-29 00:21:55.415864239 +0000 2024-07-29T00:27:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-29 00:21:55.415864239 +0000 2024-07-29T00:27:47.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-29T00:27:47.203 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-29T00:27:47.203 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-29T00:27:47.204 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000325677 s, 1.6 MB/s 2024-07-29T00:27:47.207 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-29T00:27:47.248 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-07-29T00:27:47.331 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-29T00:27:47.331 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:47.331 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-07-29T00:27:47.331 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:47.331 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:47.332 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-29 00:27:03.994167716 +0000 2024-07-29T00:27:47.332 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-29 00:21:55.997855360 +0000 2024-07-29T00:27:47.332 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-29 00:21:55.997855360 +0000 2024-07-29T00:27:47.332 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-29 00:21:55.997855360 +0000 2024-07-29T00:27:47.332 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-29T00:27:47.431 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-29T00:27:47.431 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-29T00:27:47.431 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000321668 s, 1.6 MB/s 2024-07-29T00:27:47.434 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-29T00:27:47.479 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-07-29T00:27:47.563 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-29T00:27:47.563 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:47.563 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-07-29T00:27:47.564 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:47.564 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:47.564 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-29 00:27:04.080166411 +0000 2024-07-29T00:27:47.564 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-29 00:21:56.595846236 +0000 2024-07-29T00:27:47.564 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-29 00:21:56.595846236 +0000 2024-07-29T00:27:47.564 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-29 00:21:56.595846236 +0000 2024-07-29T00:27:47.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi175=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: Saving service mgr spec with placement smithi175=x;smithi181=y;smithi204=z;count:3 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:47.622 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:47.674 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-29T00:27:47.674 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-29T00:27:47.674 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000268545 s, 1.9 MB/s 2024-07-29T00:27:47.677 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-29T00:27:47.758 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-07-29T00:27:47.758 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-07-29T00:27:47.779 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-29T00:27:47.779 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-07-29T00:27:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi175=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: Saving service mgr spec with placement smithi175=x;smithi181=y;smithi204=z;count:3 2024-07-29T00:27:47.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:47.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:47.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-07-29 00:27:36.054699428 +0000 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-07-29 00:20:13.658519729 +0000 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-07-29 00:20:13.658519729 +0000 2024-07-29T00:27:47.837 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-07-29 00:20:13.658519729 +0000 2024-07-29T00:27:47.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi175=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: Saving service mgr spec with placement smithi175=x;smithi181=y;smithi204=z;count:3 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:47.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:47.902 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-07-29T00:27:47.902 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-07-29T00:27:47.902 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000280195 s, 1.8 MB/s 2024-07-29T00:27:47.904 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-29T00:27:47.961 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-07-29T00:27:48.017 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-07-29 00:27:36.054699428 +0000 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-07-29 00:20:13.996512756 +0000 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-07-29 00:20:13.996512756 +0000 2024-07-29T00:27:48.018 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-07-29 00:20:13.996512756 +0000 2024-07-29T00:27:48.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-29T00:27:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-07-29T00:27:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-07-29T00:27:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000237302 s, 2.2 MB/s 2024-07-29T00:27:48.083 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-29T00:27:48.139 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-07-29T00:27:48.194 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-07-29 00:27:36.054699428 +0000 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-07-29 00:20:14.333505803 +0000 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-07-29 00:20:14.333505803 +0000 2024-07-29T00:27:48.195 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-07-29 00:20:14.333505803 +0000 2024-07-29T00:27:48.195 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-29T00:27:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-07-29T00:27:48.258 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-07-29T00:27:48.258 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000206897 s, 2.5 MB/s 2024-07-29T00:27:48.259 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-29T00:27:48.314 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-07-29T00:27:48.369 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-07-29 00:27:36.055699408 +0000 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-07-29 00:20:14.665498954 +0000 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-07-29 00:20:14.665498954 +0000 2024-07-29T00:27:48.370 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-07-29 00:20:14.665498954 +0000 2024-07-29T00:27:48.370 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-29T00:27:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-07-29T00:27:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-07-29T00:27:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000349201 s, 1.5 MB/s 2024-07-29T00:27:48.433 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-29T00:27:48.489 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-07-29T00:27:48.489 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-07-29T00:27:48.508 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-29T00:27:48.508 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-07-29T00:27:48.566 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-07-29 00:27:26.968550423 +0000 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-07-29 00:20:16.210626929 +0000 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-07-29 00:20:16.210626929 +0000 2024-07-29T00:27:48.567 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-07-29 00:20:16.210626929 +0000 2024-07-29T00:27:48.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-29T00:27:48.633 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-07-29T00:27:48.633 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-07-29T00:27:48.633 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.00027991 s, 1.8 MB/s 2024-07-29T00:27:48.635 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-29T00:27:48.692 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-07-29T00:27:48.748 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-07-29 00:27:26.968550423 +0000 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-07-29 00:20:16.547622128 +0000 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-07-29 00:20:16.547622128 +0000 2024-07-29T00:27:48.749 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-07-29 00:20:16.547622128 +0000 2024-07-29T00:27:48.749 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-29T00:27:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:48 smithi175 ceph-mon[32138]: Reconfiguring mgr.x (unknown last config time)... 2024-07-29T00:27:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:48 smithi175 ceph-mon[32138]: Reconfiguring daemon mgr.x on smithi175 2024-07-29T00:27:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:48 smithi175 ceph-mon[32138]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:48.810 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:48 smithi204 ceph-mon[37368]: Reconfiguring mgr.x (unknown last config time)... 2024-07-29T00:27:48.810 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:48 smithi204 ceph-mon[37368]: Reconfiguring daemon mgr.x on smithi175 2024-07-29T00:27:48.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:48 smithi204 ceph-mon[37368]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:48.813 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-07-29T00:27:48.813 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-07-29T00:27:48.813 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000219136 s, 2.3 MB/s 2024-07-29T00:27:48.814 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-29T00:27:48.871 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-07-29T00:27:48.927 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-29T00:27:48.927 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-07-29 00:27:26.968550423 +0000 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-07-29 00:20:16.893617199 +0000 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-07-29 00:20:16.893617199 +0000 2024-07-29T00:27:48.928 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-07-29 00:20:16.893617199 +0000 2024-07-29T00:27:48.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-29T00:27:48.990 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-07-29T00:27:48.990 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-07-29T00:27:48.990 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000270398 s, 1.9 MB/s 2024-07-29T00:27:48.992 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-29T00:27:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:48 smithi181 ceph-mon[36706]: Reconfiguring mgr.x (unknown last config time)... 2024-07-29T00:27:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:48 smithi181 ceph-mon[36706]: Reconfiguring daemon mgr.x on smithi175 2024-07-29T00:27:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:48 smithi181 ceph-mon[36706]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:49.047 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-07-29T00:27:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-07-29 00:27:26.969550409 +0000 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-07-29 00:20:17.224612483 +0000 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-07-29 00:20:17.224612483 +0000 2024-07-29T00:27:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-07-29 00:20:17.224612483 +0000 2024-07-29T00:27:49.102 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-29T00:27:49.163 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-07-29T00:27:49.163 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-07-29T00:27:49.163 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000167451 s, 3.1 MB/s 2024-07-29T00:27:49.165 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-29T00:27:49.220 INFO:tasks.cephadm:Deploying osd.0 on smithi175 with /dev/vg_nvme/lv_4... 2024-07-29T00:27:49.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-07-29T00:27:49.658 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: Reconfiguring mon.b (monmap changed)... 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: Reconfiguring daemon mon.b on smithi181 2024-07-29T00:27:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:49 smithi175 ceph-mon[32138]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: Reconfiguring mon.b (monmap changed)... 2024-07-29T00:27:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: Reconfiguring daemon mon.b on smithi181 2024-07-29T00:27:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:49 smithi181 ceph-mon[36706]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: Reconfiguring mon.b (monmap changed)... 2024-07-29T00:27:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: Reconfiguring daemon mon.b on smithi181 2024-07-29T00:27:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:49 smithi204 ceph-mon[37368]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:51.216 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:51.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:51.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: Reconfiguring mon.c (monmap changed)... 2024-07-29T00:27:51.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:51.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:51.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:51.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:50 smithi175 ceph-mon[32138]: Reconfiguring daemon mon.c on smithi204 2024-07-29T00:27:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: Reconfiguring mon.c (monmap changed)... 2024-07-29T00:27:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:51.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:51.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:50 smithi181 ceph-mon[36706]: Reconfiguring daemon mon.c on smithi204 2024-07-29T00:27:51.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:51.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:51.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: Reconfiguring mon.c (monmap changed)... 2024-07-29T00:27:51.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-29T00:27:51.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-29T00:27:51.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:51.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:50 smithi204 ceph-mon[37368]: Reconfiguring daemon mon.c on smithi204 2024-07-29T00:27:52.356 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.356 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.356 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:52.356 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: Deploying daemon mgr.z on smithi204 2024-07-29T00:27:52.357 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:52 smithi175 ceph-mon[32138]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:52.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: Deploying daemon mgr.z on smithi204 2024-07-29T00:27:52.473 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:52 smithi204 ceph-mon[37368]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-29T00:27:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:52.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:52.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: Deploying daemon mgr.z on smithi204 2024-07-29T00:27:52.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:52 smithi181 ceph-mon[36706]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:27:53.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-07-29T00:27:53.336 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 systemd[1]: Starting Ceph mgr.z for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:27:53.617 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:27:53.719 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 podman[39177]: 2024-07-29 00:27:53.303557909 +0000 UTC m=+0.017151438 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:27:53.719 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 podman[39177]: 2024-07-29 00:27:53.40878808 +0000 UTC m=+0.122381604 container create 71ba458fb5a267b942fe01a29dac6b97c6f107203bb13071d65aa6c525689d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:27:53.719 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 podman[39177]: 2024-07-29 00:27:53.603005722 +0000 UTC m=+0.316599238 container init 71ba458fb5a267b942fe01a29dac6b97c6f107203bb13071d65aa6c525689d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, io.buildah.version=1.36.0) 2024-07-29T00:27:53.719 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 podman[39177]: 2024-07-29 00:27:53.606297822 +0000 UTC m=+0.319891347 container start 71ba458fb5a267b942fe01a29dac6b97c6f107203bb13071d65aa6c525689d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-29T00:27:53.719 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 ceph-mgr[39196]: -- 172.21.15.204:0/1236806807 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5e33530e0 con 0x55f5e3331400 2024-07-29T00:27:54.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 bash[39177]: 71ba458fb5a267b942fe01a29dac6b97c6f107203bb13071d65aa6c525689d2d 2024-07-29T00:27:54.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 systemd[1]: Started Ceph mgr.z for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:27:54.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:53.729+0000 7f4e4c6ae040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:27:54.086 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:53.882+0000 7f4e4c6ae040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:27:54.086 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:53.948+0000 7f4e4c6ae040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:27:54.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 -- 172.21.15.175:0/3087096640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114075f10 msgr2=0x7f8114074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:54.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 --2- 172.21.15.175:0/3087096640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114075f10 0x7f8114074370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f810c009900 tx=0x7f810c02a3f0 comp rx=0 tx=0).stop 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 -- 172.21.15.175:0/3087096640 shutdown_connections 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 --2- 172.21.15.175:0/3087096640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 0x7f811410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 --2- 172.21.15.175:0/3087096640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114074940 0x7f8114074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 --2- 172.21.15.175:0/3087096640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114075f10 0x7f8114074370 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 -- 172.21.15.175:0/3087096640 >> 172.21.15.175:0/3087096640 conn(0x7f81140fcee0 msgr2=0x7f81140ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 -- 172.21.15.175:0/3087096640 shutdown_connections 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.214+0000 7f8119000640 1 -- 172.21.15.175:0/3087096640 wait complete. 2024-07-29T00:27:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.215+0000 7f8119000640 1 Processor -- start 2024-07-29T00:27:54.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 -- start start 2024-07-29T00:27:54.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 0x7f8114199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:54.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114075f10 0x7f811419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:54.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 0x7f81141a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:54.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f811410f600 con 0x7f8114075f10 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f811410f480 con 0x7f81141095b0 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8119000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f811410f780 con 0x7f8114074940 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8112a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114075f10 0x7f811419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8113400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 0x7f8114199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8113400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 0x7f8114199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:41666/0 (socket says 172.21.15.175:41666) 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8113400640 1 -- 172.21.15.175:0/2153380553 learned_addr learned my addr 172.21.15.175:0/2153380553 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.216+0000 7f8113e00640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 0x7f81141a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113400640 1 -- 172.21.15.175:0/2153380553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 msgr2=0x7f81141a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113400640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 0x7f81141a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113400640 1 -- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114075f10 msgr2=0x7f811419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113400640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114075f10 0x7f811419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:27:54.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113400640 1 -- 172.21.15.175:0/2153380553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141a1870 con 0x7f8114074940 2024-07-29T00:27:54.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113e00640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 0x7f81141a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:27:54.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8112a00640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114075f10 0x7f811419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:27:54.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8113400640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 0x7f8114199c50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f810c031440 tx=0x7f810c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c03f030 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c003ce0 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141a1b00 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.217+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c035070 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.218+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a2030 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.219+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f810c042030 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.219+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8114108330 con 0x7f8114074940 2024-07-29T00:27:54.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.219+0000 7f810be00640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80f8039290 0x7f80f803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:27:54.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.219+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f810c06cd20 con 0x7f8114074940 2024-07-29T00:27:54.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.219+0000 7f8112a00640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80f8039290 0x7f80f803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:27:54.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.220+0000 7f8112a00640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80f8039290 0x7f80f803b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005fd0 tx=0x7f80fc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:27:54.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.224+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f810c040270 con 0x7f8114074940 2024-07-29T00:27:54.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:54.421+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8114062320 con 0x7f80f8039290 2024-07-29T00:27:54.487 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:54.188+0000 7f4e4c6ae040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:27:54.487 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:54.249+0000 7f4e4c6ae040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:27:54.552 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:54.552 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.552 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.552 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.552 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:54.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-29T00:27:54.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:54.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:54.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:27:54.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:27:54.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:54.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:54.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:54.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-29T00:27:54.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:54.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:54.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:27:54.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:27:54.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:27:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:55.492 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.209+0000 7f4e4c6ae040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:27:55.493 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.338+0000 7f4e4c6ae040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:27:55.493 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.403+0000 7f4e4c6ae040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:27:55.725 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:55 smithi181 ceph-mon[36706]: Deploying daemon mgr.y on smithi181 2024-07-29T00:27:55.725 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:55 smithi181 ceph-mon[36706]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:55.726 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:55 smithi181 systemd[1]: Starting Ceph mgr.y for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:27:55.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-mon[37368]: Deploying daemon mgr.y on smithi181 2024-07-29T00:27:55.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-mon[37368]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:55.767 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.630+0000 7f4e4c6ae040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:27:55.767 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.696+0000 7f4e4c6ae040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:27:55.767 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.767+0000 7f4e4c6ae040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:27:55.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:55 smithi175 ceph-mon[32138]: Deploying daemon mgr.y on smithi181 2024-07-29T00:27:55.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:55 smithi175 ceph-mon[32138]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:27:55.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:55 smithi181 podman[38056]: 2024-07-29 00:27:55.628488865 +0000 UTC m=+0.017431220 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:27:55.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:55 smithi181 podman[38056]: 2024-07-29 00:27:55.72991273 +0000 UTC m=+0.118855084 container create 613bd09f3e12ffaecf445092d8bfb960e6933b86c9065e3f54176296ac3cfb36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-29T00:27:56.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:55 smithi181 podman[38056]: 2024-07-29 00:27:55.940586486 +0000 UTC m=+0.329528841 container init 613bd09f3e12ffaecf445092d8bfb960e6933b86c9065e3f54176296ac3cfb36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-29T00:27:56.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:55 smithi181 podman[38056]: 2024-07-29 00:27:55.943300792 +0000 UTC m=+0.332243150 container start 613bd09f3e12ffaecf445092d8bfb960e6933b86c9065e3f54176296ac3cfb36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-29T00:27:56.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:55 smithi181 ceph-mgr[38075]: -- 172.21.15.181:0/1206171608 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614084a30e0 con 0x561408481400 2024-07-29T00:27:56.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:55.918+0000 7f4e4c6ae040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:27:56.317 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 bash[38056]: 613bd09f3e12ffaecf445092d8bfb960e6933b86c9065e3f54176296ac3cfb36 2024-07-29T00:27:56.317 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 systemd[1]: Started Ceph mgr.y for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:27:56.317 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:56.065+0000 7f31f1507040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:27:56.317 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:56.219+0000 7f31f1507040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:27:56.317 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:56.287+0000 7f31f1507040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:27:56.615 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:56.548+0000 7f31f1507040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:27:56.615 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:56.614+0000 7f31f1507040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:27:56.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-mon[36706]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:56.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:56.357+0000 7f4e4c6ae040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:56.698+0000 7f4e4c6ae040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-mon[37368]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:56.783 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:56 smithi175 ceph-mon[32138]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:57.038 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:56.772+0000 7f4e4c6ae040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:27:57.039 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:56.923+0000 7f4e4c6ae040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:27:57.308 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.244+0000 7f4e4c6ae040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:27:57.308 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.308+0000 7f4e4c6ae040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:27:57.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.501+0000 7f4e4c6ae040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:27:57.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:57.621+0000 7f31f1507040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:27:57.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:57.735+0000 7f31f1507040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:27:57.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:57.795+0000 7f31f1507040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:27:58.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.652+0000 7f4e4c6ae040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:27:58.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.718+0000 7f4e4c6ae040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:27:58.086 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.802+0000 7f4e4c6ae040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:27:58.086 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:57.862+0000 7f4e4c6ae040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:27:58.268 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:58.002+0000 7f31f1507040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:27:58.268 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:58.062+0000 7f31f1507040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:27:58.268 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:58.126+0000 7f31f1507040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:27:58.269 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:58.268+0000 7f31f1507040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:27:58.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/118470939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]: dispatch 2024-07-29T00:27:58.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]: dispatch 2024-07-29T00:27:58.677 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]': finished 2024-07-29T00:27:58.677 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: osdmap e5: 1 total, 0 up, 1 in 2024-07-29T00:27:58.677 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:58.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/118470939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]: dispatch 2024-07-29T00:27:58.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]: dispatch 2024-07-29T00:27:58.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]': finished 2024-07-29T00:27:58.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: osdmap e5: 1 total, 0 up, 1 in 2024-07-29T00:27:58.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:58 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:27:58.835 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:58.395+0000 7f4e4c6ae040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:27:58.835 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:27:58.553+0000 7f4e4c6ae040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:27:58.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:27:58.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:27:58.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/118470939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]: dispatch 2024-07-29T00:27:58.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]: dispatch 2024-07-29T00:27:58.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "563e75c0-0a80-4c6d-948c-6a7d756fa193"}]': finished 2024-07-29T00:27:58.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: osdmap e5: 1 total, 0 up, 1 in 2024-07-29T00:27:58.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:58 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:27:58.982 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:58.675+0000 7f31f1507040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:27:58.982 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:58.982+0000 7f31f1507040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:27:59.249 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.046+0000 7f31f1507040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:27:59.249 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.191+0000 7f31f1507040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:27:59.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:27:59.445+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f810c03b460 con 0x7f8114074940 2024-07-29T00:27:59.736 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.497+0000 7f31f1507040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:27:59.737 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.557+0000 7f31f1507040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:27:59.737 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1991759043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:27:59.737 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-mon[36706]: Standby manager daemon z started 2024-07-29T00:27:59.738 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-29T00:27:59.738 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:27:59.738 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-29T00:27:59.738 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:27:59.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:59 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1991759043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:27:59.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:59 smithi175 ceph-mon[32138]: Standby manager daemon z started 2024-07-29T00:27:59.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:59 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-29T00:27:59.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:59 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:27:59.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:59 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-29T00:27:59.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:27:59 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:27:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:59 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1991759043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:27:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:59 smithi204 ceph-mon[37368]: Standby manager daemon z started 2024-07-29T00:27:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:59 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-29T00:27:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:59 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:27:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:59 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-29T00:27:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:27:59 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/1207241101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:27:59.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.736+0000 7f31f1507040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:27:59.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.880+0000 7f31f1507040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:27:59.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:27:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:27:59.943+0000 7f31f1507040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:28:00.443 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:28:00.024+0000 7f31f1507040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:28:00.443 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:28:00.082+0000 7f31f1507040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:28:00.749 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:28:00.606+0000 7f31f1507040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:28:00.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-mon[36706]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:00.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-mon[36706]: mgrmap e14: x(active, since 70s), standbys: z 2024-07-29T00:28:00.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-29T00:28:00.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:00 smithi175 ceph-mon[32138]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:00 smithi175 ceph-mon[32138]: mgrmap e14: x(active, since 70s), standbys: z 2024-07-29T00:28:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:00 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-29T00:28:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:00 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:00.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:00 smithi204 ceph-mon[37368]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:00.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:00 smithi204 ceph-mon[37368]: mgrmap e14: x(active, since 70s), standbys: z 2024-07-29T00:28:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:00 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-29T00:28:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:00 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:01.249 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:28:00 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:28:00.761+0000 7f31f1507040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:28:01.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:01.466+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f810c03bcc0 con 0x7f8114074940 2024-07-29T00:28:01.733 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:01 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-29T00:28:01.733 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:01 smithi175 ceph-mon[32138]: Standby manager daemon y started 2024-07-29T00:28:01.733 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:01 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:28:01.733 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:01 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-29T00:28:01.733 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:01 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:28:01.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:01 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-29T00:28:01.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:01 smithi181 ceph-mon[36706]: Standby manager daemon y started 2024-07-29T00:28:01.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:01 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:28:01.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:01 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-29T00:28:01.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:01 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:28:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:01 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-29T00:28:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:01 smithi204 ceph-mon[37368]: Standby manager daemon y started 2024-07-29T00:28:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:01 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:28:01.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:01 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-29T00:28:01.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:01 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/4094965296' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:28:02.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:02 smithi181 ceph-mon[36706]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:02.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:02 smithi181 ceph-mon[36706]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-07-29T00:28:02.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:02 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-29T00:28:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:02 smithi175 ceph-mon[32138]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:02 smithi175 ceph-mon[32138]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-07-29T00:28:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:02 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-29T00:28:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:02 smithi204 ceph-mon[37368]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:02 smithi204 ceph-mon[37368]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-07-29T00:28:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:02 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-29T00:28:04.725 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:04 smithi175 ceph-mon[32138]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:04.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:04 smithi181 ceph-mon[36706]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:04.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:04 smithi204 ceph-mon[37368]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:06.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:06 smithi181 ceph-mon[36706]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:06.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-29T00:28:06.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:06.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:06 smithi175 ceph-mon[32138]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:06.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-29T00:28:06.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:06.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:06 smithi204 ceph-mon[37368]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:06.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-29T00:28:06.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:07.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:07 smithi181 ceph-mon[36706]: Deploying daemon osd.0 on smithi175 2024-07-29T00:28:07.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:07 smithi175 ceph-mon[32138]: Deploying daemon osd.0 on smithi175 2024-07-29T00:28:07.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:07 smithi204 ceph-mon[37368]: Deploying daemon osd.0 on smithi175 2024-07-29T00:28:08.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:08 smithi181 ceph-mon[36706]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:08.756 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:08 smithi175 ceph-mon[32138]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:08.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:08 smithi204 ceph-mon[37368]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:10.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:10 smithi181 ceph-mon[36706]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:10.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:10 smithi175 ceph-mon[32138]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:10.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:10 smithi204 ceph-mon[37368]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:12.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:12 smithi181 ceph-mon[36706]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:12.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:12 smithi175 ceph-mon[32138]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:12.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:12 smithi204 ceph-mon[37368]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:14.691 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:14 smithi175 ceph-mon[32138]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:14.691 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:14.691 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:14.691 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:14.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:14 smithi181 ceph-mon[36706]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:14.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:14.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:14.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:14.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:14 smithi204 ceph-mon[37368]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:14.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:14.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:14.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:16 smithi181 ceph-mon[36706]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:16.761 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:16 smithi175 ceph-mon[32138]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:16.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:16 smithi204 ceph-mon[37368]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:18.175 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 0 on host 'smithi175' 2024-07-29T00:28:18.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.174+0000 7f810be00640 1 -- 172.21.15.175:0/2153380553 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8114062320 con 0x7f80f8039290 2024-07-29T00:28:18.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80f8039290 msgr2=0x7f80f803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:18.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80f8039290 0x7f80f803b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005fd0 tx=0x7f80fc009500 comp rx=0 tx=0).stop 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 msgr2=0x7f8114199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 0x7f8114199c50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f810c031440 tx=0x7f810c032ce0 comp rx=0 tx=0).stop 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 shutdown_connections 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141095b0 0x7f81141a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8114075f10 0x7f811419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80f8039290 0x7f80f803b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 --2- 172.21.15.175:0/2153380553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8114074940 0x7f8114199c50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 >> 172.21.15.175:0/2153380553 conn(0x7f81140fcee0 msgr2=0x7f8114105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.179+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 shutdown_connections 2024-07-29T00:28:18.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:28:18.180+0000 7f8119000640 1 -- 172.21.15.175:0/2153380553 wait complete. 2024-07-29T00:28:18.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-29T00:28:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:18 smithi204 ceph-mon[37368]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-29T00:28:18.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-29T00:28:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:18 smithi181 ceph-mon[36706]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-29T00:28:18.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:18 smithi175 ceph-mon[32138]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-29T00:28:18.960 DEBUG:teuthology.orchestra.run.smithi175:osd.0> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.0.service 2024-07-29T00:28:18.964 INFO:tasks.cephadm:Deploying osd.1 on smithi181 with /dev/vg_nvme/lv_4... 2024-07-29T00:28:18.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-07-29T00:28:19.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-29T00:28:19.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:28:19.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: osdmap e6: 1 total, 0 up, 1 in 2024-07-29T00:28:19.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:19.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:28:19.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:19.159 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:19.159 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:19.220 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:28:19.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-29T00:28:19.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:28:19.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: osdmap e6: 1 total, 0 up, 1 in 2024-07-29T00:28:19.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:19.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:28:19.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:19.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:19.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: osdmap e6: 1 total, 0 up, 1 in 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:19.785 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0[46025]: 2024-07-29T00:28:19.595+0000 7f043a000640 -1 osd.0 0 waiting for initial osdmap 2024-07-29T00:28:19.785 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:28:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0[46025]: 2024-07-29T00:28:19.611+0000 7f0430000640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:28:20.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:20 smithi175 ceph-mon[32138]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:28:20.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:20 smithi175 ceph-mon[32138]: osdmap e7: 1 total, 0 up, 1 in 2024-07-29T00:28:20.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:20.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:20.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:20 smithi175 ceph-mon[32138]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:20.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:20 smithi175 ceph-mon[32138]: from='osd.0 ' entity='osd.0' 2024-07-29T00:28:20.354 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:20 smithi181 ceph-mon[36706]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:28:20.354 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:20 smithi181 ceph-mon[36706]: osdmap e7: 1 total, 0 up, 1 in 2024-07-29T00:28:20.355 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:20.355 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:20.355 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:20 smithi181 ceph-mon[36706]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:20.355 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:20 smithi181 ceph-mon[36706]: from='osd.0 ' entity='osd.0' 2024-07-29T00:28:20.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:20 smithi204 ceph-mon[37368]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:28:20.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:20 smithi204 ceph-mon[37368]: osdmap e7: 1 total, 0 up, 1 in 2024-07-29T00:28:20.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:20.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:20.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:20 smithi204 ceph-mon[37368]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-29T00:28:20.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:20 smithi204 ceph-mon[37368]: from='osd.0 ' entity='osd.0' 2024-07-29T00:28:21.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:21 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:28:21.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:21 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:28:21.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:21.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:21 smithi181 ceph-mon[36706]: osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] boot 2024-07-29T00:28:21.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:21 smithi181 ceph-mon[36706]: osdmap e8: 1 total, 1 up, 1 in 2024-07-29T00:28:21.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:21 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:28:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:21 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:28:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:21 smithi175 ceph-mon[32138]: osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] boot 2024-07-29T00:28:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:21 smithi175 ceph-mon[32138]: osdmap e8: 1 total, 1 up, 1 in 2024-07-29T00:28:21.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:21.559 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:28:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:21 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:28:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:21 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:28:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:21 smithi204 ceph-mon[37368]: osd.0 [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] boot 2024-07-29T00:28:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:21 smithi204 ceph-mon[37368]: osdmap e8: 1 total, 1 up, 1 in 2024-07-29T00:28:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:28:21.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-07-29T00:28:21.840 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:28:22.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 -- 172.21.15.181:0/3689111255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 msgr2=0x7fa2fc107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:22.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 --2- 172.21.15.181:0/3689111255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc107150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009900 tx=0x7fa2ec02a410 comp rx=0 tx=0).stop 2024-07-29T00:28:22.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 -- 172.21.15.181:0/3689111255 shutdown_connections 2024-07-29T00:28:22.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 --2- 172.21.15.181:0/3689111255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2fc107690 0x7fa2fc109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:22.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 --2- 172.21.15.181:0/3689111255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc107150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:22.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 --2- 172.21.15.181:0/3689111255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 0x7fa2fc104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 -- 172.21.15.181:0/3689111255 >> 172.21.15.181:0/3689111255 conn(0x7fa2fc0fbf90 msgr2=0x7fa2fc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 -- 172.21.15.181:0/3689111255 shutdown_connections 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.304+0000 7fa3032c4640 1 -- 172.21.15.181:0/3689111255 wait complete. 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 Processor -- start 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 -- start start 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 0x7fa2fc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2fc107690 0x7fa2fc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc10d5d0 con 0x7fa2fc102430 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc10d450 con 0x7fa2fc104d60 2024-07-29T00:28:22.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa3032c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc10d750 con 0x7fa2fc107690 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51604/0 (socket says 172.21.15.181:51604) 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 -- 172.21.15.181:0/768612815 learned_addr learned my addr 172.21.15.181:0/768612815 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa301060640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 0x7fa2fc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa301861640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2fc107690 0x7fa2fc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 -- 172.21.15.181:0/768612815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2fc107690 msgr2=0x7fa2fc19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2fc107690 0x7fa2fc19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 -- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 msgr2=0x7fa2fc197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 0x7fa2fc197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa30085f640 1 -- 172.21.15.181:0/768612815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc19f7c0 con 0x7fa2fc104d60 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.305+0000 7fa301060640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 0x7fa2fc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:28:22.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.306+0000 7fa30085f640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc1980d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec02aa00 tx=0x7fa2ec032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:28:22.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.306+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec03f030 con 0x7fa2fc104d60 2024-07-29T00:28:22.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.306+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ec003ce0 con 0x7fa2fc104d60 2024-07-29T00:28:22.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.306+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec035070 con 0x7fa2fc104d60 2024-07-29T00:28:22.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.306+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc19fa50 con 0x7fa2fc104d60 2024-07-29T00:28:22.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.306+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc19ff30 con 0x7fa2fc104d60 2024-07-29T00:28:22.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.307+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc0fda80 con 0x7fa2fc104d60 2024-07-29T00:28:22.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.308+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fa2ec042030 con 0x7fa2fc104d60 2024-07-29T00:28:22.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.308+0000 7fa2f27fc640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa2e80a2b20 0x7fa2e80a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:22.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.309+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa2ec0edd70 con 0x7fa2fc104d60 2024-07-29T00:28:22.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.309+0000 7fa301060640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa2e80a2b20 0x7fa2e80a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:22.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.310+0000 7fa301060640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa2e80a2b20 0x7fa2e80a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800abe0 tx=0x7fa2f800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:28:22.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.313+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa2ec0bc740 con 0x7fa2fc104d60 2024-07-29T00:28:22.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:22.487+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa2fc062320 con 0x7fa2e80a2b20 2024-07-29T00:28:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:22 smithi181 ceph-mon[36706]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:22 smithi181 ceph-mon[36706]: osdmap e9: 1 total, 1 up, 1 in 2024-07-29T00:28:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:22 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:28:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:22 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:28:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:22 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:22 smithi175 ceph-mon[32138]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:22 smithi175 ceph-mon[32138]: osdmap e9: 1 total, 1 up, 1 in 2024-07-29T00:28:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:22 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:28:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:22 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:28:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:22 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:23.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:22 smithi204 ceph-mon[37368]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:23.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:22 smithi204 ceph-mon[37368]: osdmap e9: 1 total, 1 up, 1 in 2024-07-29T00:28:23.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:22 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:28:23.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:22 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:28:23.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:22 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:23.914 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:28:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:28:24.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:24.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:24.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:28:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:28:24.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:24.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:24.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:24.987 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:24 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi175 2024-07-29T00:28:24.987 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:24 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi175 to 17089M 2024-07-29T00:28:24.987 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:24 smithi181 ceph-mon[36706]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:25.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:24 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi175 2024-07-29T00:28:25.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:24 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi175 to 17089M 2024-07-29T00:28:25.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:24 smithi175 ceph-mon[32138]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:24 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi175 2024-07-29T00:28:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:24 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi175 to 17089M 2024-07-29T00:28:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:24 smithi204 ceph-mon[37368]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:25.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:25 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/1468132686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]: dispatch 2024-07-29T00:28:25.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:25 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]: dispatch 2024-07-29T00:28:25.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:25 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]': finished 2024-07-29T00:28:25.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:25 smithi204 ceph-mon[37368]: osdmap e10: 2 total, 1 up, 2 in 2024-07-29T00:28:25.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:25 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:25.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:25 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/978269987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:28:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:25 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/1468132686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]: dispatch 2024-07-29T00:28:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:25 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]: dispatch 2024-07-29T00:28:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:25 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]': finished 2024-07-29T00:28:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:25 smithi181 ceph-mon[36706]: osdmap e10: 2 total, 1 up, 2 in 2024-07-29T00:28:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:25 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:25 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/978269987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:28:26.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:25 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/1468132686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]: dispatch 2024-07-29T00:28:26.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:25 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]: dispatch 2024-07-29T00:28:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:25 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f644309b-2ac6-416d-bb60-a115681712b5"}]': finished 2024-07-29T00:28:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:25 smithi175 ceph-mon[32138]: osdmap e10: 2 total, 1 up, 2 in 2024-07-29T00:28:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:25 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:25 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/978269987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:28:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:26 smithi181 ceph-mon[36706]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:27.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:26 smithi175 ceph-mon[32138]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:26 smithi204 ceph-mon[37368]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:28.882 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:28 smithi181 ceph-mon[36706]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:28 smithi175 ceph-mon[32138]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:29.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:28 smithi204 ceph-mon[37368]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:30.643 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:30 smithi181 ceph-mon[36706]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:31.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:30 smithi175 ceph-mon[32138]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:31.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:30 smithi204 ceph-mon[37368]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:31.931 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-29T00:28:31.932 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-29T00:28:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-29T00:28:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:32.887 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:32 smithi181 ceph-mon[36706]: Deploying daemon osd.1 on smithi181 2024-07-29T00:28:32.887 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:32 smithi181 ceph-mon[36706]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:32 smithi175 ceph-mon[32138]: Deploying daemon osd.1 on smithi181 2024-07-29T00:28:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:32 smithi175 ceph-mon[32138]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:32 smithi204 ceph-mon[37368]: Deploying daemon osd.1 on smithi181 2024-07-29T00:28:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:32 smithi204 ceph-mon[37368]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:33 smithi181 ceph-mon[36706]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:33 smithi175 ceph-mon[32138]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:33 smithi204 ceph-mon[37368]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:36.689 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:36 smithi181 ceph-mon[36706]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:36.689 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:36.689 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:36.689 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:36.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:36 smithi175 ceph-mon[32138]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:36 smithi204 ceph-mon[37368]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:38.767 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:38 smithi181 ceph-mon[36706]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:38 smithi175 ceph-mon[32138]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:38.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.817+0000 7fa2f27fc640 1 -- 172.21.15.181:0/768612815 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2fc062320 con 0x7fa2e80a2b20 2024-07-29T00:28:38.817 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 1 on host 'smithi181' 2024-07-29T00:28:38.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa2e80a2b20 msgr2=0x7fa2e80a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:38.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa2e80a2b20 0x7fa2e80a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800abe0 tx=0x7fa2f800a5c0 comp rx=0 tx=0).stop 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 msgr2=0x7fa2fc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc1980d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec02aa00 tx=0x7fa2ec032da0 comp rx=0 tx=0).stop 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 shutdown_connections 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa2e80a2b20 0x7fa2e80a4fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2fc107690 0x7fa2fc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2fc104d60 0x7fa2fc1980d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 --2- 172.21.15.181:0/768612815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc102430 0x7fa2fc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 >> 172.21.15.181:0/768612815 conn(0x7fa2fc0fbf90 msgr2=0x7fa2fc102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 shutdown_connections 2024-07-29T00:28:38.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:28:38.819+0000 7fa3032c4640 1 -- 172.21.15.181:0/768612815 wait complete. 2024-07-29T00:28:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:38 smithi204 ceph-mon[37368]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:39.691 DEBUG:teuthology.orchestra.run.smithi181:osd.1> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.1.service 2024-07-29T00:28:39.694 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-07-29T00:28:39.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-07-29T00:28:39.957 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:28:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:39 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:39 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:39 smithi181 ceph-mon[36706]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:39 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:39 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.000 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:28:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1[41140]: 2024-07-29T00:28:39.861+0000 7f41fd840640 -1 osd.1 0 log_to_monitors true 2024-07-29T00:28:40.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:39 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:39 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.072 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:39 smithi204 ceph-mon[37368]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:40.072 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:39 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.072 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:39 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:39 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:39 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:39 smithi175 ceph-mon[32138]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:39 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:39 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:40 smithi204 ceph-mon[37368]: from='osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-29T00:28:41.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:40 smithi204 ceph-mon[37368]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-29T00:28:41.145 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:40 smithi181 ceph-mon[36706]: from='osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-29T00:28:41.145 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:40 smithi181 ceph-mon[36706]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-29T00:28:41.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:40 smithi175 ceph-mon[32138]: from='osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-29T00:28:41.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:40 smithi175 ceph-mon[32138]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-29T00:28:42.105 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-29T00:28:42.105 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: from='osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:28:42.105 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: osdmap e11: 2 total, 1 up, 2 in 2024-07-29T00:28:42.106 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:42.106 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:28:42.106 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:42.106 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-mon[36706]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:28:42.106 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1[41140]: 2024-07-29T00:28:41.827+0000 7f41f97c3640 -1 osd.1 0 waiting for initial osdmap 2024-07-29T00:28:42.106 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:28:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1[41140]: 2024-07-29T00:28:41.831+0000 7f41f55eb640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: from='osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: osdmap e11: 2 total, 1 up, 2 in 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:41 smithi204 ceph-mon[37368]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:28:42.128 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:28:42.155 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-07-29T00:28:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-29T00:28:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: from='osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:28:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: osdmap e11: 2 total, 1 up, 2 in 2024-07-29T00:28:42.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:42.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:28:42.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:42.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:41 smithi175 ceph-mon[32138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:28:42.413 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 -- 172.21.15.204:0/1971586576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1019c0 msgr2=0x7fca8c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 --2- 172.21.15.204:0/1971586576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1019c0 0x7fca8c101e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca74009900 tx=0x7fca7402a410 comp rx=0 tx=0).stop 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 -- 172.21.15.204:0/1971586576 shutdown_connections 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 --2- 172.21.15.204:0/1971586576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c102380 0x7fca8c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 --2- 172.21.15.204:0/1971586576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1019c0 0x7fca8c101e40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 --2- 172.21.15.204:0/1971586576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c1007c0 0x7fca8c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:42.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 -- 172.21.15.204:0/1971586576 >> 172.21.15.204:0/1971586576 conn(0x7fca8c0fbf70 msgr2=0x7fca8c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 -- 172.21.15.204:0/1971586576 shutdown_connections 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.875+0000 7fca920e6640 1 -- 172.21.15.204:0/1971586576 wait complete. 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 Processor -- start 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 -- start start 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 0x7fca8c104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c1019c0 0x7fca8c102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c102380 0x7fca8c1032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10c4a0 con 0x7fca8c1019c0 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10c320 con 0x7fca8c102380 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca920e6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10c620 con 0x7fca8c1007c0 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.876+0000 7fca8b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 0x7fca8c104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 0x7fca8c104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44278/0 (socket says 172.21.15.204:44278) 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 -- 172.21.15.204:0/3135863933 learned_addr learned my addr 172.21.15.204:0/3135863933 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:28:42.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8affd640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c1019c0 0x7fca8c102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8bfff640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c102380 0x7fca8c1032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 -- 172.21.15.204:0/3135863933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c102380 msgr2=0x7fca8c1032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c102380 0x7fca8c1032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 -- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c1019c0 msgr2=0x7fca8c102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c1019c0 0x7fca8c102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 -- 172.21.15.204:0/3135863933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c103ba0 con 0x7fca8c1007c0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca8b7fe640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 0x7fca8c104750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fca8400ccb0 tx=0x7fca8400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84003990 con 0x7fca8c1007c0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca84003b30 con 0x7fca8c1007c0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c1a1e70 con 0x7fca8c1007c0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8400f7b0 con 0x7fca8c1007c0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.877+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c1a2340 con 0x7fca8c1007c0 2024-07-29T00:28:42.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.878+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c1a25d0 con 0x7fca8c1007c0 2024-07-29T00:28:42.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.880+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fca8400f950 con 0x7fca8c1007c0 2024-07-29T00:28:42.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.881+0000 7fca88ff9640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fca780a2b70 0x7fca780a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:28:42.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.881+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fca840cdc60 con 0x7fca8c1007c0 2024-07-29T00:28:42.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.881+0000 7fca8affd640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fca780a2b70 0x7fca780a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:28:42.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.882+0000 7fca8affd640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fca780a2b70 0x7fca780a5030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca74031440 tx=0x7fca74005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:28:42.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:42.883+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fca84014030 con 0x7fca8c1007c0 2024-07-29T00:28:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: osdmap e12: 2 total, 1 up, 2 in 2024-07-29T00:28:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:43.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:43.065+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fca8c06a400 con 0x7fca780a2b70 2024-07-29T00:28:43.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: osdmap e12: 2 total, 1 up, 2 in 2024-07-29T00:28:43.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: osdmap e12: 2 total, 1 up, 2 in 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:43.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:43.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:43.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi181 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi181 to 17088M 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] boot 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: osdmap e13: 2 total, 2 up, 2 in 2024-07-29T00:28:44.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:44.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:43 smithi204 ceph-mon[37368]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi181 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi181 to 17088M 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] boot 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: osdmap e13: 2 total, 2 up, 2 in 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:28:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:28:44.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:44.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:43 smithi181 ceph-mon[36706]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:44.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi181 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi181 to 17088M 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: osd.1 [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] boot 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: osdmap e13: 2 total, 2 up, 2 in 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:28:44.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:28:44.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:44.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:43 smithi175 ceph-mon[32138]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-29T00:28:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:45 smithi181 ceph-mon[36706]: osdmap e14: 2 total, 2 up, 2 in 2024-07-29T00:28:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:44 smithi175 ceph-mon[32138]: osdmap e14: 2 total, 2 up, 2 in 2024-07-29T00:28:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:45 smithi204 ceph-mon[37368]: osdmap e14: 2 total, 2 up, 2 in 2024-07-29T00:28:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/3032936013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]: dispatch 2024-07-29T00:28:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]: dispatch 2024-07-29T00:28:46.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]': finished 2024-07-29T00:28:46.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: osdmap e15: 3 total, 2 up, 3 in 2024-07-29T00:28:46.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:28:46.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:46.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:46 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/4055639738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:28:46.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/3032936013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]: dispatch 2024-07-29T00:28:46.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]: dispatch 2024-07-29T00:28:46.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]': finished 2024-07-29T00:28:46.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: osdmap e15: 3 total, 2 up, 3 in 2024-07-29T00:28:46.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:28:46.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:46.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:46 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/4055639738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/3032936013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]: dispatch 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]: dispatch 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77a7c751-1c7b-45ae-87fc-91eb99bdced4"}]': finished 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: osdmap e15: 3 total, 2 up, 3 in 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:46.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:46 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/4055639738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:28:48.780 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:48 smithi204 ceph-mon[37368]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:48 smithi175 ceph-mon[32138]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:48 smithi181 ceph-mon[36706]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:50.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:50 smithi175 ceph-mon[32138]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:50.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:50 smithi204 ceph-mon[37368]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:50 smithi181 ceph-mon[36706]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:52.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:52 smithi175 ceph-mon[32138]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:52.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-29T00:28:52.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:52 smithi204 ceph-mon[37368]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-29T00:28:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:52 smithi181 ceph-mon[36706]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:53.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-29T00:28:53.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:53.531 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:53 smithi204 ceph-mon[37368]: Deploying daemon osd.2 on smithi204 2024-07-29T00:28:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:53 smithi175 ceph-mon[32138]: Deploying daemon osd.2 on smithi204 2024-07-29T00:28:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:53 smithi181 ceph-mon[36706]: Deploying daemon osd.2 on smithi204 2024-07-29T00:28:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:54 smithi175 ceph-mon[32138]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:54.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:54 smithi204 ceph-mon[37368]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:54.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:54 smithi181 ceph-mon[36706]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:56 smithi175 ceph-mon[32138]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:56.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:56.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:56 smithi204 ceph-mon[37368]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:56 smithi181 ceph-mon[36706]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:28:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:58.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:58 smithi175 ceph-mon[32138]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:58.828 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:58 smithi204 ceph-mon[37368]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:58 smithi181 ceph-mon[36706]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:59.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.731+0000 7fca88ff9640 1 -- 172.21.15.204:0/3135863933 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fca8c06a400 con 0x7fca780a2b70 2024-07-29T00:28:59.732 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.733+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fca780a2b70 msgr2=0x7fca780a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.733+0000 7fca920e6640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fca780a2b70 0x7fca780a5030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca74031440 tx=0x7fca74005be0 comp rx=0 tx=0).stop 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.733+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 msgr2=0x7fca8c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.733+0000 7fca920e6640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 0x7fca8c104750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fca8400ccb0 tx=0x7fca8400a620 comp rx=0 tx=0).stop 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.733+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 shutdown_connections 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.733+0000 7fca920e6640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c102380 0x7fca8c1032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.734+0000 7fca920e6640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca8c1019c0 0x7fca8c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.734+0000 7fca920e6640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fca780a2b70 0x7fca780a5030 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.734+0000 7fca920e6640 1 --2- 172.21.15.204:0/3135863933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8c1007c0 0x7fca8c104750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.734+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 >> 172.21.15.204:0/3135863933 conn(0x7fca8c0fbf70 msgr2=0x7fca8c069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:28:59.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.734+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 shutdown_connections 2024-07-29T00:28:59.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:28:59.734+0000 7fca920e6640 1 -- 172.21.15.204:0/3135863933 wait complete. 2024-07-29T00:28:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:28:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:28:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:59.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:28:59 smithi204 ceph-mon[37368]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:28:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:28:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:00.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:00.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:00.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:28:59 smithi181 ceph-mon[36706]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:00.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:00.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:00.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:28:59 smithi175 ceph-mon[32138]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:00.316 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.2.service 2024-07-29T00:29:00.318 INFO:tasks.cephadm:Deploying osd.3 on smithi175 with /dev/vg_nvme/lv_3... 2024-07-29T00:29:00.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-07-29T00:29:00.763 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:29:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:00 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:00 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:00 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:00 smithi181 ceph-mon[36706]: from='osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-29T00:29:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:00 smithi181 ceph-mon[36706]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-29T00:29:01.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:00 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:01.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:00 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:01.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:00 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:01.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:00 smithi175 ceph-mon[32138]: from='osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-29T00:29:01.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:00 smithi175 ceph-mon[32138]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-29T00:29:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:00 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:00 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:00 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:00 smithi204 ceph-mon[37368]: from='osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-29T00:29:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:00 smithi204 ceph-mon[37368]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-29T00:29:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:01 smithi181 ceph-mon[36706]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-29T00:29:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:01 smithi181 ceph-mon[36706]: osdmap e16: 3 total, 2 up, 3 in 2024-07-29T00:29:02.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:01 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:02.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:01 smithi181 ceph-mon[36706]: from='osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:29:02.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:01 smithi181 ceph-mon[36706]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:29:02.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:01 smithi181 ceph-mon[36706]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:01 smithi175 ceph-mon[32138]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-29T00:29:02.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:01 smithi175 ceph-mon[32138]: osdmap e16: 3 total, 2 up, 3 in 2024-07-29T00:29:02.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:01 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:02.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:01 smithi175 ceph-mon[32138]: from='osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:29:02.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:01 smithi175 ceph-mon[32138]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:29:02.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:01 smithi175 ceph-mon[32138]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-mon[37368]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-29T00:29:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-mon[37368]: osdmap e16: 3 total, 2 up, 3 in 2024-07-29T00:29:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-mon[37368]: from='osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:29:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-mon[37368]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:29:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-mon[37368]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:02.087 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2[42343]: 2024-07-29T00:29:01.731+0000 7f8e4eec2640 -1 osd.2 0 waiting for initial osdmap 2024-07-29T00:29:02.087 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2[42343]: 2024-07-29T00:29:01.735+0000 7f8e4acea640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:29:02.934 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:02 smithi204 ceph-mon[37368]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:29:02.934 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:02 smithi204 ceph-mon[37368]: osdmap e17: 3 total, 2 up, 3 in 2024-07-29T00:29:02.934 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:02 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:02 smithi181 ceph-mon[36706]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:29:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:02 smithi181 ceph-mon[36706]: osdmap e17: 3 total, 2 up, 3 in 2024-07-29T00:29:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:02 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:03.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:02 smithi175 ceph-mon[32138]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:29:03.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:02 smithi175 ceph-mon[32138]: osdmap e17: 3 total, 2 up, 3 in 2024-07-29T00:29:03.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:02 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] boot 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: osdmap e18: 3 total, 3 up, 3 in 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:04.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:04.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.334 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:29:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:29:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:29:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] boot 2024-07-29T00:29:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: osdmap e18: 3 total, 3 up, 3 in 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:04.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-07-29T00:29:04.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: osd.2 [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] boot 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: osdmap e18: 3 total, 3 up, 3 in 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:04.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:04.815 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:29:05.213 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi204 2024-07-29T00:29:05.213 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi204 to 17088M 2024-07-29T00:29:05.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:05.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:29:05.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-29T00:29:05.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: osdmap e19: 3 total, 3 up, 3 in 2024-07-29T00:29:05.214 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:29:05.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi204 2024-07-29T00:29:05.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi204 to 17088M 2024-07-29T00:29:05.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:05.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:29:05.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-29T00:29:05.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: osdmap e19: 3 total, 3 up, 3 in 2024-07-29T00:29:05.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:29:05.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 -- 172.21.15.175:0/2162419076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100640 msgr2=0x7f3f24100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:05.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2162419076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100640 0x7f3f24100a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009900 tx=0x7f3f1802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 -- 172.21.15.175:0/2162419076 shutdown_connections 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2162419076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24102200 0x7f3f24108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2162419076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f24101840 0x7f3f24101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2162419076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100640 0x7f3f24100a40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 -- 172.21.15.175:0/2162419076 >> 172.21.15.175:0/2162419076 conn(0x7f3f240fbdd0 msgr2=0x7f3f240fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 -- 172.21.15.175:0/2162419076 shutdown_connections 2024-07-29T00:29:05.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.475+0000 7f3f2a600640 1 -- 172.21.15.175:0/2162419076 wait complete. 2024-07-29T00:29:05.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.476+0000 7f3f2a600640 1 Processor -- start 2024-07-29T00:29:05.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 -- start start 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 0x7f3f24199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24101840 0x7f3f2419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f24102200 0x7f3f241a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410c2f0 con 0x7f3f24102200 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410c170 con 0x7f3f24101840 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f2a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410c470 con 0x7f3f24100640 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f29200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 0x7f3f24199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f29200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 0x7f3f24199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:47018/0 (socket says 172.21.15.175:47018) 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f29200640 1 -- 172.21.15.175:0/2856169356 learned_addr learned my addr 172.21.15.175:0/2856169356 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:29:05.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.477+0000 7f3f23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24101840 0x7f3f2419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:05.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f29c00640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f24102200 0x7f3f241a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f23e00640 1 -- 172.21.15.175:0/2856169356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 msgr2=0x7f3f24199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f23e00640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 0x7f3f24199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f23e00640 1 -- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f24102200 msgr2=0x7f3f241a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f23e00640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f24102200 0x7f3f241a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f23e00640 1 -- 172.21.15.175:0/2856169356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f241a1840 con 0x7f3f24101840 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f29200640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 0x7f3f24199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f23e00640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24101840 0x7f3f2419a160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000e9f0 tx=0x7f3f1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000ccc0 con 0x7f3f24101840 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f10010910 con 0x7f3f24101840 2024-07-29T00:29:05.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241a1b30 con 0x7f3f24101840 2024-07-29T00:29:05.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10018960 con 0x7f3f24101840 2024-07-29T00:29:05.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.478+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241a20c0 con 0x7f3f24101840 2024-07-29T00:29:05.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.480+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f241a2290 con 0x7f3f24101840 2024-07-29T00:29:05.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.481+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f3f10010ac0 con 0x7f3f24101840 2024-07-29T00:29:05.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.483+0000 7f3f21600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3f0c0a2b70 0x7f3f0c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:05.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.483+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f3f100cdf00 con 0x7f3f24101840 2024-07-29T00:29:05.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.483+0000 7f3f29200640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3f0c0a2b70 0x7f3f0c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:05.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.484+0000 7f3f29200640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3f0c0a2b70 0x7f3f0c0a5030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f18006010 tx=0x7f3f180315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:29:05.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.487+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3f1009c310 con 0x7f3f24101840 2024-07-29T00:29:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi204 2024-07-29T00:29:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi204 to 17088M 2024-07-29T00:29:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-29T00:29:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:29:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-29T00:29:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: osdmap e19: 3 total, 3 up, 3 in 2024-07-29T00:29:05.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:29:05.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:05.690+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3f240fe010 con 0x7f3f0c0a2b70 2024-07-29T00:29:06.087 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:05 smithi175 sudo[50676]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-29T00:29:06.087 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:05 smithi175 sudo[50676]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:06.087 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:05 smithi175 sudo[50676]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:06.087 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:05 smithi175 sudo[50676]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:06.087 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-29T00:29:06.088 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:06 smithi175 ceph-mon[32138]: osdmap e20: 3 total, 3 up, 3 in 2024-07-29T00:29:06.088 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:06 smithi175 ceph-mon[32138]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:06.088 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:06.088 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:06.088 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:06.084+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3f10098280 con 0x7f3f24101840 2024-07-29T00:29:06.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-29T00:29:06.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:06 smithi204 ceph-mon[37368]: osdmap e20: 3 total, 3 up, 3 in 2024-07-29T00:29:06.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:06 smithi204 ceph-mon[37368]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:06.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:06.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:06.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:06.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-29T00:29:06.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:06 smithi181 ceph-mon[36706]: osdmap e20: 3 total, 3 up, 3 in 2024-07-29T00:29:06.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:06 smithi181 ceph-mon[36706]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:06.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:06.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:06.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:06.690 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:06 smithi175 sudo[50759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-29T00:29:06.690 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:06 smithi175 sudo[50759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:06.690 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:06 smithi175 sudo[50759]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:06.690 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:29:06 smithi175 sudo[50759]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:06.963 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44935]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-29T00:29:06.963 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44935]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:06.963 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44935]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:06.963 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44935]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:07.249 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44938]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-29T00:29:07.249 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:07.249 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44938]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:07.250 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:29:06 smithi181 sudo[44938]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:07.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:07 smithi181 ceph-mon[36706]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:07.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:07 smithi181 ceph-mon[36706]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-07-29T00:29:07.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:07 smithi181 ceph-mon[36706]: osdmap e21: 3 total, 3 up, 3 in 2024-07-29T00:29:07.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:07 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:07.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:07 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:07.335 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46213]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-29T00:29:07.335 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46213]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:07.335 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46213]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:07.335 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46213]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:07.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:07 smithi204 ceph-mon[37368]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:07.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:07 smithi204 ceph-mon[37368]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-07-29T00:29:07.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:07 smithi204 ceph-mon[37368]: osdmap e21: 3 total, 3 up, 3 in 2024-07-29T00:29:07.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:07 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:07.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:07 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:07.385 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:07 smithi175 ceph-mon[32138]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:07.385 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:07 smithi175 ceph-mon[32138]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-07-29T00:29:07.385 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:07 smithi175 ceph-mon[32138]: osdmap e21: 3 total, 3 up, 3 in 2024-07-29T00:29:07.385 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:07 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:07.385 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:07 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:07.835 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-29T00:29:07.835 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:07.836 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46216]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:07.836 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:29:07 smithi204 sudo[46216]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:08.187 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50805]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-29T00:29:08.187 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50805]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:08.187 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50805]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:08.187 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50805]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:08.936 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-29T00:29:08.936 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:08.937 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50822]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:08.937 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 sudo[50822]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:08.937 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:08.937 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 ceph-mon[32138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:08.937 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:08 smithi175 ceph-mon[32138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:08 smithi181 ceph-mon[36706]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:08 smithi181 ceph-mon[36706]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:09.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:08 smithi204 sudo[46219]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-29T00:29:09.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:08 smithi204 sudo[46219]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:09.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:08 smithi204 sudo[46219]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:09.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:08 smithi204 sudo[46219]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:09.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 sudo[46222]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-29T00:29:09.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 sudo[46222]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:09.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 sudo[46222]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:09.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 sudo[46222]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:09.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:09.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:09.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:09.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44941]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-29T00:29:09.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44941]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:09.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44941]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:09.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44941]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44944]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-29T00:29:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44944]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-29T00:29:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44944]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 sudo[44944]: pam_unix(sudo:session): session closed for user root 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/680868531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec28d659-e23c-4abf-bc28-9dae71c447c5"}]: dispatch 2024-07-29T00:29:10.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:10.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:09 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/680868531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec28d659-e23c-4abf-bc28-9dae71c447c5"}]': finished 2024-07-29T00:29:10.009 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:10.009 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/680868531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec28d659-e23c-4abf-bc28-9dae71c447c5"}]: dispatch 2024-07-29T00:29:10.010 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:10.011 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:09 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/680868531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec28d659-e23c-4abf-bc28-9dae71c447c5"}]': finished 2024-07-29T00:29:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:29:10.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:29:10.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:29:10.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/680868531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec28d659-e23c-4abf-bc28-9dae71c447c5"}]: dispatch 2024-07-29T00:29:10.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:10.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:09 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/680868531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec28d659-e23c-4abf-bc28-9dae71c447c5"}]': finished 2024-07-29T00:29:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:10 smithi181 ceph-mon[36706]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:10 smithi181 ceph-mon[36706]: osdmap e22: 4 total, 3 up, 4 in 2024-07-29T00:29:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:11.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:10 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1231717058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:29:11.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:10 smithi175 ceph-mon[32138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:11.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:10 smithi175 ceph-mon[32138]: osdmap e22: 4 total, 3 up, 4 in 2024-07-29T00:29:11.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:11.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:10 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1231717058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:29:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:10 smithi204 ceph-mon[37368]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-29T00:29:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:10 smithi204 ceph-mon[37368]: osdmap e22: 4 total, 3 up, 4 in 2024-07-29T00:29:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:10 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1231717058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:29:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:11 smithi181 ceph-mon[36706]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:11 smithi175 ceph-mon[32138]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:11 smithi204 ceph-mon[37368]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:14.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:14 smithi175 ceph-mon[32138]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:14.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:14 smithi204 ceph-mon[37368]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:14 smithi181 ceph-mon[36706]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:16 smithi175 ceph-mon[32138]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-29T00:29:16.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:16 smithi204 ceph-mon[37368]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-29T00:29:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:16 smithi181 ceph-mon[36706]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-29T00:29:18.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:18 smithi175 ceph-mon[32138]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-29T00:29:18.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-29T00:29:18.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:18.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:18 smithi204 ceph-mon[37368]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-29T00:29:18.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-29T00:29:18.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:18 smithi181 ceph-mon[36706]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-29T00:29:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-29T00:29:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:19.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:19 smithi175 ceph-mon[32138]: Deploying daemon osd.3 on smithi175 2024-07-29T00:29:19.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:19 smithi204 ceph-mon[37368]: Deploying daemon osd.3 on smithi175 2024-07-29T00:29:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:19 smithi181 ceph-mon[36706]: Deploying daemon osd.3 on smithi175 2024-07-29T00:29:20.703 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:20 smithi175 ceph-mon[32138]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-29T00:29:20.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:20 smithi204 ceph-mon[37368]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-29T00:29:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:20 smithi181 ceph-mon[36706]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-29T00:29:22.711 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:22 smithi175 ceph-mon[32138]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 869 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:22 smithi204 ceph-mon[37368]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 869 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:22.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:22 smithi181 ceph-mon[36706]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 869 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:24.802 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:24 smithi181 ceph-mon[36706]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:24.829 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:24 smithi175 ceph-mon[32138]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:24.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:24 smithi204 ceph-mon[37368]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:26.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:26 smithi204 ceph-mon[37368]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:26.985 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:26 smithi175 ceph-mon[32138]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:26.985 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:26.985 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:26.985 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:26 smithi181 ceph-mon[36706]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-29T00:29:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:28.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:28 smithi175 ceph-mon[32138]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:28.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:28 smithi204 ceph-mon[37368]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:28 smithi181 ceph-mon[36706]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:30.537 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:30 smithi175 ceph-mon[32138]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:30.537 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.537 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.537 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:30.537 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:30.538 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:30 smithi181 ceph-mon[36706]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:30.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:30.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:30 smithi204 ceph-mon[37368]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:30.862 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 3 on host 'smithi175' 2024-07-29T00:29:30.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.861+0000 7f3f21600640 1 -- 172.21.15.175:0/2856169356 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3f240fe010 con 0x7f3f0c0a2b70 2024-07-29T00:29:30.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.864+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3f0c0a2b70 msgr2=0x7f3f0c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:30.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.864+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3f0c0a2b70 0x7f3f0c0a5030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f18006010 tx=0x7f3f180315b0 comp rx=0 tx=0).stop 2024-07-29T00:29:30.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.864+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24101840 msgr2=0x7f3f2419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:30.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.864+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24101840 0x7f3f2419a160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000e9f0 tx=0x7f3f1000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 shutdown_connections 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f24102200 0x7f3f241a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24101840 0x7f3f2419a160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3f0c0a2b70 0x7f3f0c0a5030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 --2- 172.21.15.175:0/2856169356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f24100640 0x7f3f24199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 >> 172.21.15.175:0/2856169356 conn(0x7f3f240fbdd0 msgr2=0x7f3f240fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.865+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 shutdown_connections 2024-07-29T00:29:30.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:29:30.866+0000 7f3f2a600640 1 -- 172.21.15.175:0/2856169356 wait complete. 2024-07-29T00:29:31.683 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:31 smithi175 ceph-mon[32138]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-29T00:29:31.684 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:31.684 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:31.684 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:31 smithi181 ceph-mon[36706]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-29T00:29:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:31.756 DEBUG:teuthology.orchestra.run.smithi175:osd.3> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.3.service 2024-07-29T00:29:31.760 INFO:tasks.cephadm:Deploying osd.4 on smithi181 with /dev/vg_nvme/lv_3... 2024-07-29T00:29:31.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-07-29T00:29:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:31 smithi204 ceph-mon[37368]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-29T00:29:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:31.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:32.018 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:29:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:32 smithi181 ceph-mon[36706]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:32 smithi181 ceph-mon[36706]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-29T00:29:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:32 smithi181 ceph-mon[36706]: osdmap e23: 4 total, 3 up, 4 in 2024-07-29T00:29:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:32 smithi181 ceph-mon[36706]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:29:32.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:32.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:32 smithi175 ceph-mon[32138]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:32.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:32 smithi175 ceph-mon[32138]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-29T00:29:32.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:32 smithi175 ceph-mon[32138]: osdmap e23: 4 total, 3 up, 4 in 2024-07-29T00:29:32.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:32 smithi175 ceph-mon[32138]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:29:32.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:32.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:32 smithi204 ceph-mon[37368]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:32.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:32 smithi204 ceph-mon[37368]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-29T00:29:32.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:32 smithi204 ceph-mon[37368]: osdmap e23: 4 total, 3 up, 4 in 2024-07-29T00:29:32.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:32 smithi204 ceph-mon[37368]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:29:32.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.110 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:29:33.074+0000 7f678d000640 -1 osd.3 0 waiting for initial osdmap 2024-07-29T00:29:33.110 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:29:33.108+0000 7f6783000640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:29:33.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:33 smithi181 ceph-mon[36706]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:29:33.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:33 smithi181 ceph-mon[36706]: osdmap e24: 4 total, 3 up, 4 in 2024-07-29T00:29:33.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:33 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:33 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:33 smithi181 ceph-mon[36706]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' 2024-07-29T00:29:33.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-mon[32138]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:29:33.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-mon[32138]: osdmap e24: 4 total, 3 up, 4 in 2024-07-29T00:29:33.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:33 smithi175 ceph-mon[32138]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' 2024-07-29T00:29:33.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:33 smithi204 ceph-mon[37368]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:29:33.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:33 smithi204 ceph-mon[37368]: osdmap e24: 4 total, 3 up, 4 in 2024-07-29T00:29:33.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:33 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:33 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:33.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:33 smithi204 ceph-mon[37368]: from='osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757]' entity='osd.3' 2024-07-29T00:29:34.383 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:29:34.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-07-29T00:29:34.660 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:29:34.684 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:34 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:29:34.684 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:34 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:29:34.684 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:34 smithi181 ceph-mon[36706]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:34.684 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:34 smithi181 ceph-mon[36706]: osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] boot 2024-07-29T00:29:34.684 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:34 smithi181 ceph-mon[36706]: osdmap e25: 4 total, 4 up, 4 in 2024-07-29T00:29:34.684 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:34 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:34 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:29:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:34 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:29:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:34 smithi175 ceph-mon[32138]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:34 smithi175 ceph-mon[32138]: osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] boot 2024-07-29T00:29:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:34 smithi175 ceph-mon[32138]: osdmap e25: 4 total, 4 up, 4 in 2024-07-29T00:29:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:34 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:34 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:29:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:34 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:29:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:34 smithi204 ceph-mon[37368]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-29T00:29:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:34 smithi204 ceph-mon[37368]: osd.3 [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] boot 2024-07-29T00:29:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:34 smithi204 ceph-mon[37368]: osdmap e25: 4 total, 4 up, 4 in 2024-07-29T00:29:34.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:34 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:29:35.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2254087829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 msgr2=0x7f60a81047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:35.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2254087829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a81047e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6098009900 tx=0x7f609802a410 comp rx=0 tx=0).stop 2024-07-29T00:29:35.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2254087829 shutdown_connections 2024-07-29T00:29:35.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2254087829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8107650 0x7f60a8109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:35.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2254087829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8104d20 0x7f60a8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:35.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2254087829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a81047e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2254087829 >> 172.21.15.181:0/2254087829 conn(0x7f60a80fbf70 msgr2=0x7f60a80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2254087829 shutdown_connections 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.118+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2254087829 wait complete. 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 Processor -- start 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 -- start start 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a8101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8104d20 0x7f60a81003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8107650 0x7f60a81008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810d570 con 0x7f60a8104d20 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810d3f0 con 0x7f60a81023f0 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60ad1c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810d6f0 con 0x7f60a8107650 2024-07-29T00:29:35.135 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a8101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a8101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40208/0 (socket says 172.21.15.181:40208) 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 -- 172.21.15.181:0/2323540317 learned_addr learned my addr 172.21.15.181:0/2323540317 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a759e640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8107650 0x7f60a81008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f609ffff640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8104d20 0x7f60a81003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 -- 172.21.15.181:0/2323540317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8107650 msgr2=0x7f60a81008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8107650 0x7f60a81008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 -- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8104d20 msgr2=0x7f60a81003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8104d20 0x7f60a81003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f60a6d9d640 1 -- 172.21.15.181:0/2323540317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81011a0 con 0x7f60a81023f0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.119+0000 7f609ffff640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8104d20 0x7f60a81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.120+0000 7f60a6d9d640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a8101d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6098009230 tx=0x7f6098004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.120+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609803f030 con 0x7f60a81023f0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.120+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a81a1d80 con 0x7f60a81023f0 2024-07-29T00:29:35.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.120+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6098003ce0 con 0x7f60a81023f0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.120+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098035070 con 0x7f60a81023f0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.120+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a2180 con 0x7f60a81023f0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.121+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a80fda60 con 0x7f60a81023f0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.123+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6098042030 con 0x7f60a81023f0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.123+0000 7f60a4d99640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f60940a2b80 0x7f60940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.124+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3698+0+0 (secure 0 0 0) 0x7f60980ee850 con 0x7f60a81023f0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.124+0000 7f609ffff640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f60940a2b80 0x7f60940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.124+0000 7f609ffff640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f60940a2b80 0x7f60940a5040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60900058c0 tx=0x7f6090005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:29:35.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.126+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f60980bc9b0 con 0x7f60a81023f0 2024-07-29T00:29:35.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:35.304+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f60a8062320 con 0x7f60940a2b80 2024-07-29T00:29:35.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:35 smithi181 ceph-mon[36706]: osdmap e26: 4 total, 4 up, 4 in 2024-07-29T00:29:35.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:35.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:35.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:35 smithi175 ceph-mon[32138]: osdmap e26: 4 total, 4 up, 4 in 2024-07-29T00:29:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:35.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:35 smithi204 ceph-mon[37368]: osdmap e26: 4 total, 4 up, 4 in 2024-07-29T00:29:35.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:35.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:35.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: osdmap e27: 4 total, 4 up, 4 in 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:36.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: osdmap e27: 4 total, 4 up, 4 in 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: osdmap e27: 4 total, 4 up, 4 in 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:36.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:36.904 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:36.904 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:36.904 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:36.904 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:37.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:37 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi175 2024-07-29T00:29:37.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:37 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi175 to 8544M 2024-07-29T00:29:37.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:37 smithi175 ceph-mon[32138]: osdmap e28: 4 total, 4 up, 4 in 2024-07-29T00:29:37.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:37 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi175 2024-07-29T00:29:37.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:37 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi175 to 8544M 2024-07-29T00:29:37.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:37 smithi204 ceph-mon[37368]: osdmap e28: 4 total, 4 up, 4 in 2024-07-29T00:29:37.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:37 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi175 2024-07-29T00:29:37.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:37 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi175 to 8544M 2024-07-29T00:29:37.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:37 smithi181 ceph-mon[36706]: osdmap e28: 4 total, 4 up, 4 in 2024-07-29T00:29:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: osdmap e29: 4 total, 4 up, 4 in 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/669149013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]: dispatch 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]: dispatch 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]': finished 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: osdmap e30: 5 total, 4 up, 5 in 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:38 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/4005719330' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:29:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: osdmap e29: 4 total, 4 up, 4 in 2024-07-29T00:29:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/669149013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]: dispatch 2024-07-29T00:29:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]: dispatch 2024-07-29T00:29:39.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]': finished 2024-07-29T00:29:39.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: osdmap e30: 5 total, 4 up, 5 in 2024-07-29T00:29:39.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:39.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:38 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/4005719330' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:29:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: osdmap e29: 4 total, 4 up, 4 in 2024-07-29T00:29:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/669149013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]: dispatch 2024-07-29T00:29:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]: dispatch 2024-07-29T00:29:39.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "708e65da-400e-40c5-8991-52201ddf57b1"}]': finished 2024-07-29T00:29:39.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: osdmap e30: 5 total, 4 up, 5 in 2024-07-29T00:29:39.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:39.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:38 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/4005719330' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:29:40.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:40 smithi204 ceph-mon[37368]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-29T00:29:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:40 smithi181 ceph-mon[36706]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-29T00:29:41.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:40 smithi175 ceph-mon[32138]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-29T00:29:42.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:42 smithi181 ceph-mon[36706]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-07-29T00:29:42.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:42 smithi204 ceph-mon[37368]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-07-29T00:29:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:42 smithi175 ceph-mon[32138]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-07-29T00:29:44.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:44 smithi204 ceph-mon[37368]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-07-29T00:29:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:44 smithi175 ceph-mon[32138]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-07-29T00:29:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:44 smithi181 ceph-mon[36706]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-07-29T00:29:45.697 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-29T00:29:45.697 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:45 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:45.697 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:45 smithi181 ceph-mon[36706]: Deploying daemon osd.4 on smithi181 2024-07-29T00:29:45.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-29T00:29:45.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:45 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:45.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:45 smithi204 ceph-mon[37368]: Deploying daemon osd.4 on smithi181 2024-07-29T00:29:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-29T00:29:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:45 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:45 smithi175 ceph-mon[32138]: Deploying daemon osd.4 on smithi181 2024-07-29T00:29:46.591 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:46 smithi181 ceph-mon[36706]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:29:46.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:46 smithi204 ceph-mon[37368]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:29:47.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:46 smithi175 ceph-mon[32138]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:29:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:48 smithi204 ceph-mon[37368]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-29T00:29:48.982 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:48 smithi181 ceph-mon[36706]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-29T00:29:49.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:48 smithi175 ceph-mon[32138]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-29T00:29:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:50 smithi204 ceph-mon[37368]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-29T00:29:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:50.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:50.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:50 smithi181 ceph-mon[36706]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-29T00:29:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:50 smithi175 ceph-mon[32138]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-29T00:29:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:52.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:52 smithi181 ceph-mon[36706]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:52 smithi204 ceph-mon[37368]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:52.837 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:52 smithi175 ceph-mon[32138]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:52.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.842+0000 7f60a4d99640 1 -- 172.21.15.181:0/2323540317 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60a8062320 con 0x7f60940a2b80 2024-07-29T00:29:52.842 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 4 on host 'smithi181' 2024-07-29T00:29:52.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f60940a2b80 msgr2=0x7f60940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:52.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f60940a2b80 0x7f60940a5040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60900058c0 tx=0x7f6090005850 comp rx=0 tx=0).stop 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 msgr2=0x7f60a8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a8101d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6098009230 tx=0x7f6098004360 comp rx=0 tx=0).stop 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 shutdown_connections 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f60940a2b80 0x7f60940a5040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8107650 0x7f60a81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60a8104d20 0x7f60a81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 --2- 172.21.15.181:0/2323540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a81023f0 0x7f60a8101d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 >> 172.21.15.181:0/2323540317 conn(0x7f60a80fbf70 msgr2=0x7f60a8102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 shutdown_connections 2024-07-29T00:29:52.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:29:52.843+0000 7f60ad1c8640 1 -- 172.21.15.181:0/2323540317 wait complete. 2024-07-29T00:29:53.831 DEBUG:teuthology.orchestra.run.smithi181:osd.4> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.4.service 2024-07-29T00:29:53.833 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-07-29T00:29:53.833 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-07-29T00:29:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:53 smithi204 ceph-mon[37368]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:54.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:53 smithi204 ceph-mon[37368]: from='osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-29T00:29:54.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:53 smithi204 ceph-mon[37368]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-29T00:29:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:29:54.210 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:54.210 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:54.210 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:53 smithi175 ceph-mon[32138]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:54.210 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:53 smithi175 ceph-mon[32138]: from='osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-29T00:29:54.210 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:53 smithi175 ceph-mon[32138]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-29T00:29:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:53 smithi181 ceph-mon[36706]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:53 smithi181 ceph-mon[36706]: from='osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-29T00:29:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:53 smithi181 ceph-mon[36706]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: from='osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: osdmap e31: 5 total, 4 up, 5 in 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:55.111 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:55.112 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:29:54.862+0000 7fbe13411640 -1 osd.4 0 waiting for initial osdmap 2024-07-29T00:29:55.112 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:29:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:29:54.868+0000 7fbe0f239640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:29:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-29T00:29:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: from='osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:29:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: osdmap e31: 5 total, 4 up, 5 in 2024-07-29T00:29:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:29:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:55.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-29T00:29:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: from='osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:29:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: osdmap e31: 5 total, 4 up, 5 in 2024-07-29T00:29:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:29:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:55.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:55 smithi181 ceph-mon[36706]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:29:56.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:55 smithi181 ceph-mon[36706]: osdmap e32: 5 total, 4 up, 5 in 2024-07-29T00:29:56.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:55 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:56.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:55 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:56.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:55 smithi181 ceph-mon[36706]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:56.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:55 smithi204 ceph-mon[37368]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:29:56.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:55 smithi204 ceph-mon[37368]: osdmap e32: 5 total, 4 up, 5 in 2024-07-29T00:29:56.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:55 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:56.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:55 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:56.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:55 smithi204 ceph-mon[37368]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:56.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:55 smithi175 ceph-mon[32138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:29:56.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:55 smithi175 ceph-mon[32138]: osdmap e32: 5 total, 4 up, 5 in 2024-07-29T00:29:56.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:55 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:56.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:55 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:56.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:55 smithi175 ceph-mon[32138]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-29T00:29:56.466 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:29:56.497 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-07-29T00:29:56.757 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:29:57.038 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:56 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:29:57.038 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:56 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:29:57.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:56 smithi204 ceph-mon[37368]: osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] boot 2024-07-29T00:29:57.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:56 smithi204 ceph-mon[37368]: osdmap e33: 5 total, 5 up, 5 in 2024-07-29T00:29:57.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:56 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:57.196 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:56 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:29:57.196 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:56 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:29:57.196 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:56 smithi181 ceph-mon[36706]: osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] boot 2024-07-29T00:29:57.196 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:56 smithi181 ceph-mon[36706]: osdmap e33: 5 total, 5 up, 5 in 2024-07-29T00:29:57.196 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:56 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:57.208 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 -- 172.21.15.204:0/2052896051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e01007c0 msgr2=0x7f63e0100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:57.208 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 --2- 172.21.15.204:0/2052896051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e01007c0 0x7f63e0100bc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009900 tx=0x7f63d002a410 comp rx=0 tx=0).stop 2024-07-29T00:29:57.208 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 -- 172.21.15.204:0/2052896051 shutdown_connections 2024-07-29T00:29:57.208 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 --2- 172.21.15.204:0/2052896051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e0102380 0x7f63e0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:57.208 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 --2- 172.21.15.204:0/2052896051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e01019c0 0x7f63e0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:57.208 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 --2- 172.21.15.204:0/2052896051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e01007c0 0x7f63e0100bc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 -- 172.21.15.204:0/2052896051 >> 172.21.15.204:0/2052896051 conn(0x7f63e00fbf70 msgr2=0x7f63e00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 -- 172.21.15.204:0/2052896051 shutdown_connections 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.201+0000 7f63e6074640 1 -- 172.21.15.204:0/2052896051 wait complete. 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 Processor -- start 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 -- start start 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e01007c0 0x7f63e006a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e01019c0 0x7f63e019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 0x7f63e019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63e010c4c0 con 0x7f63e01019c0 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63e010c340 con 0x7f63e01007c0 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63e6074640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63e010c640 con 0x7f63e0102380 2024-07-29T00:29:57.209 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 0x7f63e019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:57.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 0x7f63e019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:58590/0 (socket says 172.21.15.204:58590) 2024-07-29T00:29:57.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 -- 172.21.15.204:0/3778559997 learned_addr learned my addr 172.21.15.204:0/3778559997 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:29:57.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63df7fe640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e01007c0 0x7f63e006a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:57.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 -- 172.21.15.204:0/3778559997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e01007c0 msgr2=0x7f63e006a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:57.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e01007c0 0x7f63e006a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:57.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 -- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e01019c0 msgr2=0x7f63e019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:29:57.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e01019c0 0x7f63e019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:29:57.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.202+0000 7f63dffff640 1 -- 172.21.15.204:0/3778559997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e019f8c0 con 0x7f63e0102380 2024-07-29T00:29:57.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.203+0000 7f63dffff640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 0x7f63e019efb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63d800eb10 tx=0x7f63d800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:29:57.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.203+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8004b20 con 0x7f63e0102380 2024-07-29T00:29:57.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.203+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d8004cc0 con 0x7f63e0102380 2024-07-29T00:29:57.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.203+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e019fb50 con 0x7f63e0102380 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.203+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8010760 con 0x7f63e0102380 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.203+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01a0110 con 0x7f63e0102380 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.204+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e0107bc0 con 0x7f63e0102380 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.206+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f63d8005000 con 0x7f63e0102380 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.206+0000 7f63dcff9640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.206+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f63d80cea80 con 0x7f63e0102380 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.207+0000 7f63df7fe640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.208+0000 7f63df7fe640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63c80a2b80 0x7f63c80a5040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005ab0 tx=0x7f63d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:29:57.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.210+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f63d809ca50 con 0x7f63e0102380 2024-07-29T00:29:57.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:56 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:29:57.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:56 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:29:57.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:56 smithi175 ceph-mon[32138]: osd.4 [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] boot 2024-07-29T00:29:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:56 smithi175 ceph-mon[32138]: osdmap e33: 5 total, 5 up, 5 in 2024-07-29T00:29:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:56 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:29:57.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:29:57.392+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f63e0062320 con 0x7f63c80a2b80 2024-07-29T00:29:58.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: osdmap e34: 5 total, 5 up, 5 in 2024-07-29T00:29:58.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi181 2024-07-29T00:29:58.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi181 to 8544M 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:58.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:58.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:29:57 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: osdmap e34: 5 total, 5 up, 5 in 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi181 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi181 to 8544M 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:58.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:29:57 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: osdmap e34: 5 total, 5 up, 5 in 2024-07-29T00:29:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi181 2024-07-29T00:29:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi181 to 8544M 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:29:58.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:29:58.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:29:57 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/3750026968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]: dispatch 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]: dispatch 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]': finished 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: osdmap e35: 6 total, 5 up, 6 in 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: overall HEALTH_OK 2024-07-29T00:30:00.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:00 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/446362487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:00.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/3750026968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]: dispatch 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]: dispatch 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]': finished 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: osdmap e35: 6 total, 5 up, 6 in 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: overall HEALTH_OK 2024-07-29T00:30:00.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:00 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/446362487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/3750026968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]: dispatch 2024-07-29T00:30:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]: dispatch 2024-07-29T00:30:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6258db1-4eca-4859-bca9-9291b0c40d4e"}]': finished 2024-07-29T00:30:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: osdmap e35: 6 total, 5 up, 6 in 2024-07-29T00:30:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: overall HEALTH_OK 2024-07-29T00:30:01.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:00 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/446362487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:02 smithi175 ceph-mon[32138]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:02 smithi204 ceph-mon[37368]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:02 smithi181 ceph-mon[36706]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:04.711 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:04 smithi204 ceph-mon[37368]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:04.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:04 smithi175 ceph-mon[32138]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:04.979 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:04 smithi181 ceph-mon[36706]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:06.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:06 smithi204 ceph-mon[37368]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:06.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-29T00:30:06.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:06 smithi181 ceph-mon[36706]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-29T00:30:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:06 smithi175 ceph-mon[32138]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-29T00:30:07.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:07.812 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:07 smithi204 ceph-mon[37368]: Deploying daemon osd.5 on smithi204 2024-07-29T00:30:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:07 smithi181 ceph-mon[36706]: Deploying daemon osd.5 on smithi204 2024-07-29T00:30:08.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:07 smithi175 ceph-mon[32138]: Deploying daemon osd.5 on smithi204 2024-07-29T00:30:08.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:08 smithi204 ceph-mon[37368]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:08 smithi181 ceph-mon[36706]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:09.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:08 smithi175 ceph-mon[32138]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:10.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:10 smithi204 ceph-mon[37368]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:10 smithi181 ceph-mon[36706]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:11.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:10 smithi175 ceph-mon[32138]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:11.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:11 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:11.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:11 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:11.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:11 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:11 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:11 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:11 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:11 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:11 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:11 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:12.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:12 smithi204 ceph-mon[37368]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:12.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:12 smithi181 ceph-mon[36706]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:13.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:12 smithi175 ceph-mon[32138]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:14.284 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.284+0000 7f63dcff9640 1 -- 172.21.15.204:0/3778559997 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f63e0062320 con 0x7f63c80a2b80 2024-07-29T00:30:14.284 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63c80a2b80 msgr2=0x7f63c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63c80a2b80 0x7f63c80a5040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005ab0 tx=0x7f63d0000bc0 comp rx=0 tx=0).stop 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 msgr2=0x7f63e019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 0x7f63e019efb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63d800eb10 tx=0x7f63d800c2d0 comp rx=0 tx=0).stop 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 shutdown_connections 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e0102380 0x7f63e019efb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:14.286 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e01019c0 0x7f63e019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:14.287 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 --2- 172.21.15.204:0/3778559997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e01007c0 0x7f63e006a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:14.287 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 >> 172.21.15.204:0/3778559997 conn(0x7f63e00fbf70 msgr2=0x7f63e0104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:30:14.287 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 shutdown_connections 2024-07-29T00:30:14.287 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:30:14.286+0000 7f63e6074640 1 -- 172.21.15.204:0/3778559997 wait complete. 2024-07-29T00:30:14.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:14 smithi204 ceph-mon[37368]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:14.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:14.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:14 smithi181 ceph-mon[36706]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:15.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:14 smithi175 ceph-mon[32138]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:15.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:15.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:15.252 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.5.service 2024-07-29T00:30:15.255 INFO:tasks.cephadm:Deploying osd.6 on smithi175 with /dev/vg_nvme/lv_2... 2024-07-29T00:30:15.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-07-29T00:30:15.531 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:30:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:30:15.383+0000 7f1379599640 -1 osd.5 0 log_to_monitors true 2024-07-29T00:30:15.712 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:30:16.482 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:16 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:16.482 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:16 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:16.482 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:16 smithi204 ceph-mon[37368]: from='osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-29T00:30:16.482 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:16 smithi204 ceph-mon[37368]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-29T00:30:16.482 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:16 smithi204 ceph-mon[37368]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:16.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:16 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:16.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:16 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:16.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:16 smithi175 ceph-mon[32138]: from='osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-29T00:30:16.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:16 smithi175 ceph-mon[32138]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-29T00:30:16.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:16 smithi175 ceph-mon[32138]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:16 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:16 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:16 smithi181 ceph-mon[36706]: from='osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-29T00:30:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:16 smithi181 ceph-mon[36706]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-29T00:30:16.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:16 smithi181 ceph-mon[36706]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:17.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:17 smithi175 ceph-mon[32138]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-29T00:30:17.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:17 smithi175 ceph-mon[32138]: osdmap e36: 6 total, 5 up, 6 in 2024-07-29T00:30:17.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:17 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:17.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:17 smithi175 ceph-mon[32138]: from='osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:30:17.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:17 smithi175 ceph-mon[32138]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:30:17.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:17 smithi181 ceph-mon[36706]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-29T00:30:17.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:17 smithi181 ceph-mon[36706]: osdmap e36: 6 total, 5 up, 6 in 2024-07-29T00:30:17.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:17 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:17.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:17 smithi181 ceph-mon[36706]: from='osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:30:17.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:17 smithi181 ceph-mon[36706]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:30:17.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-mon[37368]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-29T00:30:17.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-mon[37368]: osdmap e36: 6 total, 5 up, 6 in 2024-07-29T00:30:17.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:17.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-mon[37368]: from='osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:30:17.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-mon[37368]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:30:18.085 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:30:17.830+0000 7f1375d30640 -1 osd.5 0 waiting for initial osdmap 2024-07-29T00:30:18.085 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:30:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:30:17.836+0000 7f136d33c640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: osdmap e37: 6 total, 5 up, 6 in 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: from='osd.5 ' entity='osd.5' 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] boot 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: osdmap e38: 6 total, 6 up, 6 in 2024-07-29T00:30:18.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:18.587 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:30:18.587 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: osdmap e37: 6 total, 5 up, 6 in 2024-07-29T00:30:18.588 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:18.588 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:18.588 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: from='osd.5 ' entity='osd.5' 2024-07-29T00:30:18.588 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] boot 2024-07-29T00:30:18.588 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: osdmap e38: 6 total, 6 up, 6 in 2024-07-29T00:30:18.588 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:18.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:30:18.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: osdmap e37: 6 total, 5 up, 6 in 2024-07-29T00:30:18.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:18.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-29T00:30:18.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: from='osd.5 ' entity='osd.5' 2024-07-29T00:30:18.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: osd.5 [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] boot 2024-07-29T00:30:18.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: osdmap e38: 6 total, 6 up, 6 in 2024-07-29T00:30:18.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:30:19.384 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:30:19.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi204 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:19.497 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi204 to 8544M 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:19.498 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:19.499 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.499 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:19 smithi175 ceph-mon[32138]: osdmap e39: 6 total, 6 up, 6 in 2024-07-29T00:30:19.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi204 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi204 to 8544M 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:19.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:19.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:19.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:19.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:19.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:19 smithi181 ceph-mon[36706]: osdmap e39: 6 total, 6 up, 6 in 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi204 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi204 to 8544M 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:19.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:19.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:19.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:19.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:19.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:19.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:19 smithi204 ceph-mon[37368]: osdmap e39: 6 total, 6 up, 6 in 2024-07-29T00:30:19.862 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 -- 172.21.15.175:0/2942809986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c1074d0 msgr2=0x7fdb2c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 --2- 172.21.15.175:0/2942809986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c1074d0 0x7fdb2c1098c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009ca0 tx=0x7fdb2402a8c0 comp rx=0 tx=0).stop 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 -- 172.21.15.175:0/2942809986 shutdown_connections 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 --2- 172.21.15.175:0/2942809986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c1074d0 0x7fdb2c1098c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 --2- 172.21.15.175:0/2942809986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 --2- 172.21.15.175:0/2942809986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb2c102270 0x7fdb2c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 -- 172.21.15.175:0/2942809986 >> 172.21.15.175:0/2942809986 conn(0x7fdb2c0fbdd0 msgr2=0x7fdb2c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:30:20.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 -- 172.21.15.175:0/2942809986 shutdown_connections 2024-07-29T00:30:20.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.525+0000 7fdb2be00640 1 -- 172.21.15.175:0/2942809986 wait complete. 2024-07-29T00:30:20.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.526+0000 7fdb2be00640 1 Processor -- start 2024-07-29T00:30:20.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.526+0000 7fdb2be00640 1 -- start start 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb2c102270 0x7fdb2c101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb2c1074d0 0x7fdb2c100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c10d440 con 0x7fdb2c104ba0 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c10d2c0 con 0x7fdb2c102270 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c10d5c0 con 0x7fdb2c1074d0 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57352/0 (socket says 172.21.15.175:57352) 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2a000640 1 -- 172.21.15.175:0/207574252 learned_addr learned my addr 172.21.15.175:0/207574252 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2aa00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb2c102270 0x7fdb2c101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:20.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2b400640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb2c1074d0 0x7fdb2c100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2a000640 1 -- 172.21.15.175:0/207574252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb2c1074d0 msgr2=0x7fdb2c100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.527+0000 7fdb2a000640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb2c1074d0 0x7fdb2c100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb2a000640 1 -- 172.21.15.175:0/207574252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb2c102270 msgr2=0x7fdb2c101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb2a000640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb2c102270 0x7fdb2c101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb2a000640 1 -- 172.21.15.175:0/207574252 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c101020 con 0x7fdb2c104ba0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb2a000640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c100220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800e9f0 tx=0x7fdb1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800ccc0 con 0x7fdb2c104ba0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb18005500 con 0x7fdb2c104ba0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18010690 con 0x7fdb2c104ba0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c1a1b70 con 0x7fdb2c104ba0 2024-07-29T00:30:20.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.528+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c1a2030 con 0x7fdb2c104ba0 2024-07-29T00:30:20.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.529+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2c0fd8c0 con 0x7fdb2c104ba0 2024-07-29T00:30:20.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.532+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdb180046c0 con 0x7fdb2c104ba0 2024-07-29T00:30:20.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.533+0000 7fdb23400640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdb100a2b80 0x7fdb100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:20.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.533+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fdb180ce710 con 0x7fdb2c104ba0 2024-07-29T00:30:20.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.533+0000 7fdb2aa00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdb100a2b80 0x7fdb100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:20.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.534+0000 7fdb2aa00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdb100a2b80 0x7fdb100a5040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0095f0 tx=0x7fdb1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:30:20.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.538+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdb18053df0 con 0x7fdb2c104ba0 2024-07-29T00:30:20.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:20.722+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdb2c062320 con 0x7fdb100a2b80 2024-07-29T00:30:20.723 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:20 smithi175 ceph-mon[32138]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:20.723 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:20 smithi175 ceph-mon[32138]: osdmap e40: 6 total, 6 up, 6 in 2024-07-29T00:30:20.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:20 smithi181 ceph-mon[36706]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:20.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:20 smithi181 ceph-mon[36706]: osdmap e40: 6 total, 6 up, 6 in 2024-07-29T00:30:20.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:20 smithi204 ceph-mon[37368]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:20.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:20 smithi204 ceph-mon[37368]: osdmap e40: 6 total, 6 up, 6 in 2024-07-29T00:30:21.692 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:30:21.692 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:30:21.692 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:21.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:30:21.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:30:21.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:30:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:30:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:22 smithi181 ceph-mon[36706]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:30:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:22 smithi181 ceph-mon[36706]: osdmap e41: 6 total, 6 up, 6 in 2024-07-29T00:30:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:22 smithi181 ceph-mon[36706]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:22 smithi175 ceph-mon[32138]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:30:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:22 smithi175 ceph-mon[32138]: osdmap e41: 6 total, 6 up, 6 in 2024-07-29T00:30:22.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:22 smithi175 ceph-mon[32138]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:22 smithi204 ceph-mon[37368]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:30:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:22 smithi204 ceph-mon[37368]: osdmap e41: 6 total, 6 up, 6 in 2024-07-29T00:30:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:22 smithi204 ceph-mon[37368]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:23.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:23 smithi181 ceph-mon[36706]: osdmap e42: 6 total, 6 up, 6 in 2024-07-29T00:30:23.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:23 smithi175 ceph-mon[32138]: osdmap e42: 6 total, 6 up, 6 in 2024-07-29T00:30:23.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:23 smithi204 ceph-mon[37368]: osdmap e42: 6 total, 6 up, 6 in 2024-07-29T00:30:24.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:24 smithi181 ceph-mon[36706]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:24.778 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:24 smithi175 ceph-mon[32138]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:24.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:24 smithi204 ceph-mon[37368]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:25.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:25 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1198297184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]: dispatch 2024-07-29T00:30:25.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:25 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]: dispatch 2024-07-29T00:30:25.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:25 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]': finished 2024-07-29T00:30:25.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:25 smithi175 ceph-mon[32138]: osdmap e43: 7 total, 6 up, 7 in 2024-07-29T00:30:25.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:25 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:25.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:25 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2537770504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:25 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1198297184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]: dispatch 2024-07-29T00:30:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:25 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]: dispatch 2024-07-29T00:30:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:25 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]': finished 2024-07-29T00:30:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:25 smithi181 ceph-mon[36706]: osdmap e43: 7 total, 6 up, 7 in 2024-07-29T00:30:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:25 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:25 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2537770504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:25.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:25 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1198297184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]: dispatch 2024-07-29T00:30:25.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:25 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]: dispatch 2024-07-29T00:30:25.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:25 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d77f5c-49a7-4727-99a5-c86d7bf62e86"}]': finished 2024-07-29T00:30:25.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:25 smithi204 ceph-mon[37368]: osdmap e43: 7 total, 6 up, 7 in 2024-07-29T00:30:25.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:25 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:25.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:25 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2537770504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:26.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:26 smithi181 ceph-mon[36706]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-29T00:30:26.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:26 smithi175 ceph-mon[32138]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-29T00:30:26.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:26 smithi204 ceph-mon[37368]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-29T00:30:28.585 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:28 smithi175 ceph-mon[32138]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-07-29T00:30:28.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:28 smithi204 ceph-mon[37368]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-07-29T00:30:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:28 smithi181 ceph-mon[36706]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-07-29T00:30:30.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:30 smithi175 ceph-mon[32138]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:30:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:30 smithi204 ceph-mon[37368]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:30:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:30 smithi181 ceph-mon[36706]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:30:32.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:32 smithi175 ceph-mon[32138]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-07-29T00:30:32.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:32 smithi204 ceph-mon[37368]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-07-29T00:30:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:32 smithi181 ceph-mon[36706]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-07-29T00:30:33.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:33 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-29T00:30:33.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:33 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:33 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-29T00:30:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:33 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:33 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-29T00:30:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:33 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:34 smithi204 ceph-mon[37368]: Deploying daemon osd.6 on smithi175 2024-07-29T00:30:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:34 smithi204 ceph-mon[37368]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-29T00:30:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:34 smithi181 ceph-mon[36706]: Deploying daemon osd.6 on smithi175 2024-07-29T00:30:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:34 smithi181 ceph-mon[36706]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-29T00:30:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:34 smithi175 ceph-mon[32138]: Deploying daemon osd.6 on smithi175 2024-07-29T00:30:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:34 smithi175 ceph-mon[32138]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-29T00:30:36.600 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:36 smithi175 ceph-mon[32138]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:36 smithi204 ceph-mon[37368]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:36 smithi181 ceph-mon[36706]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:38 smithi175 ceph-mon[32138]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:38 smithi204 ceph-mon[37368]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:38 smithi181 ceph-mon[36706]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:40.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:40 smithi204 ceph-mon[37368]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:40 smithi181 ceph-mon[36706]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:41.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:40 smithi175 ceph-mon[32138]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:42.720 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:42 smithi175 ceph-mon[32138]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:42.721 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:42.721 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:42.721 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:42.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:42 smithi204 ceph-mon[37368]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:42.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:42.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:42.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:42 smithi181 ceph-mon[36706]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:44.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:44 smithi175 ceph-mon[32138]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:44.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:44 smithi204 ceph-mon[37368]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:44 smithi181 ceph-mon[36706]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:46.403 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 6 on host 'smithi175' 2024-07-29T00:30:46.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.402+0000 7fdb23400640 1 -- 172.21.15.175:0/207574252 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdb2c062320 con 0x7fdb100a2b80 2024-07-29T00:30:46.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.405+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdb100a2b80 msgr2=0x7fdb100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:46.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.405+0000 7fdb2be00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdb100a2b80 0x7fdb100a5040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0095f0 tx=0x7fdb1c005f90 comp rx=0 tx=0).stop 2024-07-29T00:30:46.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.405+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 msgr2=0x7fdb2c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:46.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.405+0000 7fdb2be00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c100220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800e9f0 tx=0x7fdb1800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 shutdown_connections 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdb100a2b80 0x7fdb100a5040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb2c1074d0 0x7fdb2c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb2c104ba0 0x7fdb2c100220 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 --2- 172.21.15.175:0/207574252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb2c102270 0x7fdb2c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 >> 172.21.15.175:0/207574252 conn(0x7fdb2c0fbdd0 msgr2=0x7fdb2c102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 shutdown_connections 2024-07-29T00:30:46.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:30:46.406+0000 7fdb2be00640 1 -- 172.21.15.175:0/207574252 wait complete. 2024-07-29T00:30:46.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-29T00:30:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-29T00:30:46.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:46.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:47.246 DEBUG:teuthology.orchestra.run.smithi175:osd.6> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.6.service 2024-07-29T00:30:47.250 INFO:tasks.cephadm:Deploying osd.7 on smithi181 with /dev/vg_nvme/lv_2... 2024-07-29T00:30:47.250 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-07-29T00:30:47.509 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:30:47.631 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:47 smithi181 ceph-mon[36706]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-29T00:30:47.631 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:47 smithi181 ceph-mon[36706]: from='osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:30:47.631 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:47 smithi181 ceph-mon[36706]: osdmap e44: 7 total, 6 up, 7 in 2024-07-29T00:30:47.632 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:47.632 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:47 smithi181 ceph-mon[36706]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:30:47.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-mon[32138]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-29T00:30:47.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-mon[32138]: from='osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:30:47.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-mon[32138]: osdmap e44: 7 total, 6 up, 7 in 2024-07-29T00:30:47.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:47.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-mon[32138]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:30:47.786 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:30:47.492+0000 7f9362800640 -1 osd.6 0 waiting for initial osdmap 2024-07-29T00:30:47.786 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:30:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:30:47.522+0000 7f935d800640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:30:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:47 smithi204 ceph-mon[37368]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-29T00:30:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:47 smithi204 ceph-mon[37368]: from='osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:30:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:47 smithi204 ceph-mon[37368]: osdmap e44: 7 total, 6 up, 7 in 2024-07-29T00:30:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:47.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:47 smithi204 ceph-mon[37368]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:30:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:48 smithi181 ceph-mon[36706]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:30:48.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:48 smithi181 ceph-mon[36706]: osdmap e45: 7 total, 6 up, 7 in 2024-07-29T00:30:48.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:48 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:48.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:48 smithi181 ceph-mon[36706]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:48.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:48 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:48 smithi175 ceph-mon[32138]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:30:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:48 smithi175 ceph-mon[32138]: osdmap e45: 7 total, 6 up, 7 in 2024-07-29T00:30:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:48 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:48 smithi175 ceph-mon[32138]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:48 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:48 smithi204 ceph-mon[37368]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:30:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:48 smithi204 ceph-mon[37368]: osdmap e45: 7 total, 6 up, 7 in 2024-07-29T00:30:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:48 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:48 smithi204 ceph-mon[37368]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:48.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:48 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:49.729 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:49 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:30:49.729 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:49 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:30:49.729 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:49 smithi181 ceph-mon[36706]: osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] boot 2024-07-29T00:30:49.729 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:49 smithi181 ceph-mon[36706]: osdmap e46: 7 total, 7 up, 7 in 2024-07-29T00:30:49.729 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:49.729 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:49.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:49 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:30:49.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:49 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:30:49.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:49 smithi175 ceph-mon[32138]: osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] boot 2024-07-29T00:30:49.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:49 smithi175 ceph-mon[32138]: osdmap e46: 7 total, 7 up, 7 in 2024-07-29T00:30:49.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:49.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:49.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:49 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:30:49.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:49 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:30:49.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:49 smithi204 ceph-mon[37368]: osd.6 [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] boot 2024-07-29T00:30:49.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:49 smithi204 ceph-mon[37368]: osdmap e46: 7 total, 7 up, 7 in 2024-07-29T00:30:49.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:49.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:30:50.222 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:30:50.249 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-07-29T00:30:50.507 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:30:50.699 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:50 smithi175 ceph-mon[32138]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:50.699 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:50 smithi175 ceph-mon[32138]: osdmap e47: 7 total, 7 up, 7 in 2024-07-29T00:30:50.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:50 smithi181 ceph-mon[36706]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:50.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:50 smithi181 ceph-mon[36706]: osdmap e47: 7 total, 7 up, 7 in 2024-07-29T00:30:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:50 smithi204 ceph-mon[37368]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-29T00:30:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:50 smithi204 ceph-mon[37368]: osdmap e47: 7 total, 7 up, 7 in 2024-07-29T00:30:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 -- 172.21.15.181:0/3238718480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c102b00 msgr2=0x7f753c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 --2- 172.21.15.181:0/3238718480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c102b00 0x7f753c102f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f752c009900 tx=0x7f752c02a410 comp rx=0 tx=0).stop 2024-07-29T00:30:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 -- 172.21.15.181:0/3238718480 shutdown_connections 2024-07-29T00:30:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 --2- 172.21.15.181:0/3238718480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c1034c0 0x7f753c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 --2- 172.21.15.181:0/3238718480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c102b00 0x7f753c102f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 --2- 172.21.15.181:0/3238718480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753c101900 0x7f753c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 -- 172.21.15.181:0/3238718480 >> 172.21.15.181:0/3238718480 conn(0x7f753c0fd0b0 msgr2=0x7f753c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 -- 172.21.15.181:0/3238718480 shutdown_connections 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 -- 172.21.15.181:0/3238718480 wait complete. 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 Processor -- start 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 -- start start 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 0x7f753c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.996+0000 7f7543ac8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c102b00 0x7f753c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7543ac8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753c1034c0 0x7f753c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7543ac8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10d600 con 0x7f753c1034c0 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7543ac8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10d480 con 0x7f753c101900 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7543ac8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10d780 con 0x7f753c102b00 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 0x7f753c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 0x7f753c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60330/0 (socket says 172.21.15.181:60330) 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 -- 172.21.15.181:0/98303956 learned_addr learned my addr 172.21.15.181:0/98303956 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541063640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c102b00 0x7f753c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 -- 172.21.15.181:0/98303956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c102b00 msgr2=0x7f753c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c102b00 0x7f753c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 -- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753c1034c0 msgr2=0x7f753c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753c1034c0 0x7f753c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 -- 172.21.15.181:0/98303956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19f860 con 0x7f753c101900 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541063640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c102b00 0x7f753c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:30:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7541864640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 0x7f753c197c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f753800b450 tx=0x7f753800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538015020 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7538011d70 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19fb50 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753800f340 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.997+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1a00e0 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.998+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c108cd0 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:50.999+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7538018020 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:51.000+0000 7f7532ffd640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f75280a2c00 0x7f75280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:51.000+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f75380cde20 con 0x7f753c101900 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:51.000+0000 7f7541063640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f75280a2c00 0x7f75280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:51.001+0000 7f7541063640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f75280a2c00 0x7f75280a50c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f752c000c00 tx=0x7f752c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:30:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:51.004+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f753801f020 con 0x7f753c101900 2024-07-29T00:30:51.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:30:51.178+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f753c062320 con 0x7f75280a2c00 2024-07-29T00:30:51.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:30:51.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:30:51.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:30:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:30:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:51.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:30:51.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:30:51.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:52.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:52.648 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:30:52.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:52.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:30:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:30:52.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:30:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:53 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi175 2024-07-29T00:30:53.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:53 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi175 to 5696M 2024-07-29T00:30:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:53 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi175 2024-07-29T00:30:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:53 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi175 to 5696M 2024-07-29T00:30:53.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:53 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi175 2024-07-29T00:30:53.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:53 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi175 to 5696M 2024-07-29T00:30:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/240493769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]: dispatch 2024-07-29T00:30:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]: dispatch 2024-07-29T00:30:54.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]': finished 2024-07-29T00:30:54.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: osdmap e48: 8 total, 7 up, 8 in 2024-07-29T00:30:54.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:30:54.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:54 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/3587151239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/240493769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]: dispatch 2024-07-29T00:30:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]: dispatch 2024-07-29T00:30:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]': finished 2024-07-29T00:30:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: osdmap e48: 8 total, 7 up, 8 in 2024-07-29T00:30:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:30:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:54 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/3587151239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/240493769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]: dispatch 2024-07-29T00:30:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]: dispatch 2024-07-29T00:30:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393b8cdf-a78f-4d38-b132-f45237f8f088"}]': finished 2024-07-29T00:30:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: osdmap e48: 8 total, 7 up, 8 in 2024-07-29T00:30:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:30:55.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:54 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/3587151239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:30:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:56 smithi175 ceph-mon[32138]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:56 smithi204 ceph-mon[37368]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:56 smithi181 ceph-mon[36706]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:58.652 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:30:58 smithi181 ceph-mon[36706]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:58.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:30:58 smithi175 ceph-mon[32138]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:30:58.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:30:58 smithi204 ceph-mon[37368]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:00 smithi175 ceph-mon[32138]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:00.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:00 smithi204 ceph-mon[37368]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:00 smithi181 ceph-mon[36706]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:01.711 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:01 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-29T00:31:01.711 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:01 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:01.711 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:01 smithi181 ceph-mon[36706]: Deploying daemon osd.7 on smithi181 2024-07-29T00:31:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:01 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-29T00:31:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:01 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:01 smithi204 ceph-mon[37368]: Deploying daemon osd.7 on smithi181 2024-07-29T00:31:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:01 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-29T00:31:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:01 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:01 smithi175 ceph-mon[32138]: Deploying daemon osd.7 on smithi181 2024-07-29T00:31:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:02 smithi204 ceph-mon[37368]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:02.931 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:02 smithi181 ceph-mon[36706]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:03.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:02 smithi175 ceph-mon[32138]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:04.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:04 smithi181 ceph-mon[36706]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:04.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:04 smithi204 ceph-mon[37368]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:05.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:04 smithi175 ceph-mon[32138]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:06.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:06 smithi181 ceph-mon[36706]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:06.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:06.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:06.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:06.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:06 smithi204 ceph-mon[37368]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:06.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:06.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:06.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:06 smithi175 ceph-mon[32138]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:08.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:08 smithi181 ceph-mon[36706]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:08.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:08 smithi204 ceph-mon[37368]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:08.913 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 7 on host 'smithi181' 2024-07-29T00:31:08.914 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.913+0000 7f7532ffd640 1 -- 172.21.15.181:0/98303956 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f753c062320 con 0x7f75280a2c00 2024-07-29T00:31:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f75280a2c00 msgr2=0x7f75280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f75280a2c00 0x7f75280a50c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f752c000c00 tx=0x7f752c031400 comp rx=0 tx=0).stop 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 msgr2=0x7f753c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 0x7f753c197c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f753800b450 tx=0x7f753800b910 comp rx=0 tx=0).stop 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 shutdown_connections 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f753c1034c0 0x7f753c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f75280a2c00 0x7f75280a50c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f753c102b00 0x7f753c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 --2- 172.21.15.181:0/98303956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f753c101900 0x7f753c197c40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 >> 172.21.15.181:0/98303956 conn(0x7f753c0fd0b0 msgr2=0x7f753c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 shutdown_connections 2024-07-29T00:31:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:31:08.915+0000 7f7543ac8640 1 -- 172.21.15.181:0/98303956 wait complete. 2024-07-29T00:31:09.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:08 smithi175 ceph-mon[32138]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:09.899 DEBUG:teuthology.orchestra.run.smithi181:osd.7> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.7.service 2024-07-29T00:31:09.901 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-07-29T00:31:09.901 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-07-29T00:31:10.163 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:31:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:09 smithi181 ceph-mon[36706]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:09 smithi181 ceph-mon[36706]: from='osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-29T00:31:10.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:09 smithi181 ceph-mon[36706]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-29T00:31:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:09 smithi175 ceph-mon[32138]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:09 smithi175 ceph-mon[32138]: from='osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-29T00:31:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:09 smithi175 ceph-mon[32138]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-29T00:31:10.285 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:10.285 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:10.285 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:09 smithi204 ceph-mon[37368]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:10.285 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:09 smithi204 ceph-mon[37368]: from='osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-29T00:31:10.285 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:09 smithi204 ceph-mon[37368]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-29T00:31:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-29T00:31:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: from='osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:31:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: osdmap e49: 8 total, 7 up, 8 in 2024-07-29T00:31:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:31:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:11.250 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:31:10.928+0000 7f54f3fb7640 -1 osd.7 0 waiting for initial osdmap 2024-07-29T00:31:11.250 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:31:10 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:31:10.936+0000 7f54efddf640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:31:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-29T00:31:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: from='osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:31:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: osdmap e49: 8 total, 7 up, 8 in 2024-07-29T00:31:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:31:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:11.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-29T00:31:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: from='osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:31:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: osdmap e49: 8 total, 7 up, 8 in 2024-07-29T00:31:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:31:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:11 smithi181 ceph-mon[36706]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:31:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:11 smithi181 ceph-mon[36706]: osdmap e50: 8 total, 7 up, 8 in 2024-07-29T00:31:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:11 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:11 smithi181 ceph-mon[36706]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:11 smithi181 ceph-mon[36706]: osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] boot 2024-07-29T00:31:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:11 smithi175 ceph-mon[32138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:31:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:11 smithi175 ceph-mon[32138]: osdmap e50: 8 total, 7 up, 8 in 2024-07-29T00:31:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:11 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:11 smithi175 ceph-mon[32138]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:11 smithi175 ceph-mon[32138]: osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] boot 2024-07-29T00:31:12.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:11 smithi204 ceph-mon[37368]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:31:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:11 smithi204 ceph-mon[37368]: osdmap e50: 8 total, 7 up, 8 in 2024-07-29T00:31:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:11 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:11 smithi204 ceph-mon[37368]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-29T00:31:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:11 smithi204 ceph-mon[37368]: osd.7 [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] boot 2024-07-29T00:31:12.692 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:31:12.718 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-07-29T00:31:12.981 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:31:13.184 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:12 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:31:13.184 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:12 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:31:13.184 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:12 smithi181 ceph-mon[36706]: osdmap e51: 8 total, 8 up, 8 in 2024-07-29T00:31:13.184 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:12 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:13.184 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:12 smithi181 ceph-mon[36706]: osdmap e52: 8 total, 8 up, 8 in 2024-07-29T00:31:13.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:12 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:31:13.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:12 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:31:13.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:12 smithi204 ceph-mon[37368]: osdmap e51: 8 total, 8 up, 8 in 2024-07-29T00:31:13.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:12 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:13.227 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:12 smithi204 ceph-mon[37368]: osdmap e52: 8 total, 8 up, 8 in 2024-07-29T00:31:13.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:12 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:31:13.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:12 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:31:13.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:12 smithi175 ceph-mon[32138]: osdmap e51: 8 total, 8 up, 8 in 2024-07-29T00:31:13.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:12 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:31:13.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:12 smithi175 ceph-mon[32138]: osdmap e52: 8 total, 8 up, 8 in 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 -- 172.21.15.204:0/462153416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 msgr2=0x7f65c8101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/462153416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c8101e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009900 tx=0x7f65b802a410 comp rx=0 tx=0).stop 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 -- 172.21.15.204:0/462153416 shutdown_connections 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/462153416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c8102380 0x7f65c8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/462153416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c8101e40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/462153416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c81007c0 0x7f65c8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 -- 172.21.15.204:0/462153416 >> 172.21.15.204:0/462153416 conn(0x7f65c80fbf70 msgr2=0x7f65c80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:31:13.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 -- 172.21.15.204:0/462153416 shutdown_connections 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 -- 172.21.15.204:0/462153416 wait complete. 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 Processor -- start 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.444+0000 7f65cf8ef640 1 -- start start 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cf8ef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c81007c0 0x7f65c8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cf8ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cf8ef640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c8102380 0x7f65c819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cf8ef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c4c0 con 0x7f65c81007c0 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cf8ef640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c340 con 0x7f65c8102380 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cf8ef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c640 con 0x7f65c81019c0 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c81981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:59304/0 (socket says 172.21.15.204:59304) 2024-07-29T00:31:13.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 -- 172.21.15.204:0/524333908 learned_addr learned my addr 172.21.15.204:0/524333908 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cde8c640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c8102380 0x7f65c819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 -- 172.21.15.204:0/524333908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c8102380 msgr2=0x7f65c819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c8102380 0x7f65c819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 -- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c81007c0 msgr2=0x7f65c8197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c81007c0 0x7f65c8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 -- 172.21.15.204:0/524333908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c819f8b0 con 0x7f65c81019c0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65cce8a640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c81981d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009230 tx=0x7f65b8004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.445+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b803f030 con 0x7f65c81019c0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.446+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b8003ce0 con 0x7f65c81019c0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.446+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c819fb40 con 0x7f65c81019c0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.446+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8035070 con 0x7f65c81019c0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.446+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a0020 con 0x7f65c81019c0 2024-07-29T00:31:13.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.447+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c81a0250 con 0x7f65c81019c0 2024-07-29T00:31:13.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.448+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f65b8042030 con 0x7f65c81019c0 2024-07-29T00:31:13.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.448+0000 7f65be7fc640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65b40a2b80 0x7f65b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:13.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.448+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5426+0+0 (secure 0 0 0) 0x7f65b80eeed0 con 0x7f65c81019c0 2024-07-29T00:31:13.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.449+0000 7f65cd68b640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65b40a2b80 0x7f65b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:13.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.450+0000 7f65cd68b640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65b40a2b80 0x7f65b40a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005fd0 tx=0x7f65c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:31:13.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.452+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f65b80bc970 con 0x7f65c81019c0 2024-07-29T00:31:13.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:13.630+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f65c806a440 con 0x7f65b40a2b80 2024-07-29T00:31:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi181 2024-07-29T00:31:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi181 to 5696M 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:31:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:31:14.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:31:14.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:14.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:14 smithi204 ceph-mon[37368]: osdmap e53: 8 total, 8 up, 8 in 2024-07-29T00:31:14.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi181 2024-07-29T00:31:14.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi181 to 5696M 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:31:14.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:31:14.501 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:31:14.501 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:14.501 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:14 smithi181 ceph-mon[36706]: osdmap e53: 8 total, 8 up, 8 in 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi181 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi181 to 5696M 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:14.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:14 smithi175 ceph-mon[32138]: osdmap e53: 8 total, 8 up, 8 in 2024-07-29T00:31:15.538 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:15 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:15.538 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:15 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:15.538 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:15 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:15.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:15 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:15.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:15 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:15.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:15 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:15 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:15 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:15 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:16.536 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: osdmap e54: 8 total, 8 up, 8 in 2024-07-29T00:31:16.536 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:16.536 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/2104528597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]: dispatch 2024-07-29T00:31:16.537 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]: dispatch 2024-07-29T00:31:16.537 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]': finished 2024-07-29T00:31:16.537 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: osdmap e55: 9 total, 8 up, 9 in 2024-07-29T00:31:16.537 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:16 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: osdmap e54: 8 total, 8 up, 8 in 2024-07-29T00:31:16.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:16.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/2104528597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]: dispatch 2024-07-29T00:31:16.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]: dispatch 2024-07-29T00:31:16.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]': finished 2024-07-29T00:31:16.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: osdmap e55: 9 total, 8 up, 9 in 2024-07-29T00:31:16.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:16 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: osdmap e54: 8 total, 8 up, 8 in 2024-07-29T00:31:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/2104528597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]: dispatch 2024-07-29T00:31:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]: dispatch 2024-07-29T00:31:16.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f7f413a-1907-4fd0-8482-293e00ba2c96"}]': finished 2024-07-29T00:31:16.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: osdmap e55: 9 total, 8 up, 9 in 2024-07-29T00:31:16.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:16 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:17.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:17 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/3074109132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:31:17.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:17 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/3074109132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:31:17.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:17 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/3074109132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:31:18.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:18 smithi204 ceph-mon[37368]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:18.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:18 smithi181 ceph-mon[36706]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:18.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:18 smithi175 ceph-mon[32138]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:20.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:20 smithi204 ceph-mon[37368]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:20 smithi181 ceph-mon[36706]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:21.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:20 smithi175 ceph-mon[32138]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:22 smithi204 ceph-mon[37368]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:22.837 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:22 smithi175 ceph-mon[32138]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:22.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:22 smithi181 ceph-mon[36706]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:23.714 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-29T00:31:23.714 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-29T00:31:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-29T00:31:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:24.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:24 smithi204 ceph-mon[37368]: Deploying daemon osd.8 on smithi204 2024-07-29T00:31:24.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:24 smithi204 ceph-mon[37368]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:24.900 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:24 smithi181 ceph-mon[36706]: Deploying daemon osd.8 on smithi204 2024-07-29T00:31:24.901 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:24 smithi181 ceph-mon[36706]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:25.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:24 smithi175 ceph-mon[32138]: Deploying daemon osd.8 on smithi204 2024-07-29T00:31:25.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:24 smithi175 ceph-mon[32138]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:26.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:26 smithi204 ceph-mon[37368]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:26 smithi181 ceph-mon[36706]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:27.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:26 smithi175 ceph-mon[32138]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:28.801 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:28 smithi204 ceph-mon[37368]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:28.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:28.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:28.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:28 smithi181 ceph-mon[36706]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:29.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:28 smithi175 ceph-mon[32138]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:30 smithi204 ceph-mon[37368]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:30 smithi181 ceph-mon[36706]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:31.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:30 smithi175 ceph-mon[32138]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:31.321 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.321+0000 7f65be7fc640 1 -- 172.21.15.204:0/524333908 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65c806a440 con 0x7f65b40a2b80 2024-07-29T00:31:31.321 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65b40a2b80 msgr2=0x7f65b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65b40a2b80 0x7f65b40a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005fd0 tx=0x7f65c400b410 comp rx=0 tx=0).stop 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 msgr2=0x7f65c81981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c81981d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009230 tx=0x7f65b8004360 comp rx=0 tx=0).stop 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 shutdown_connections 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c8102380 0x7f65c819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65b40a2b80 0x7f65b40a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c81019c0 0x7f65c81981d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 --2- 172.21.15.204:0/524333908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c81007c0 0x7f65c8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:31.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 >> 172.21.15.204:0/524333908 conn(0x7f65c80fbf70 msgr2=0x7f65c8069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:31:31.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 shutdown_connections 2024-07-29T00:31:31.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:31:31.324+0000 7f65cf8ef640 1 -- 172.21.15.204:0/524333908 wait complete. 2024-07-29T00:31:31.988 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.8.service 2024-07-29T00:31:31.990 INFO:tasks.cephadm:Deploying osd.9 on smithi175 with /dev/vg_nvme/lv_1... 2024-07-29T00:31:31.990 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-07-29T00:31:32.451 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:31:32.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:32.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: from='osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-29T00:31:32.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-29T00:31:32.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:32.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: from='osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-29T00:31:32.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-29T00:31:32.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:32.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: from='osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-29T00:31:32.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-29T00:31:32.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:32.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:33.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-mon[37368]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-29T00:31:33.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-mon[37368]: from='osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:31:33.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-mon[37368]: osdmap e56: 9 total, 8 up, 9 in 2024-07-29T00:31:33.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:33.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-mon[37368]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:31:33.650 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:33 smithi175 ceph-mon[32138]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-29T00:31:33.650 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:33 smithi175 ceph-mon[32138]: from='osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:31:33.650 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:33 smithi175 ceph-mon[32138]: osdmap e56: 9 total, 8 up, 9 in 2024-07-29T00:31:33.650 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:33 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:33.650 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:33 smithi175 ceph-mon[32138]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:31:33.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:33 smithi181 ceph-mon[36706]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-29T00:31:33.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:33 smithi181 ceph-mon[36706]: from='osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:31:33.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:33 smithi181 ceph-mon[36706]: osdmap e56: 9 total, 8 up, 9 in 2024-07-29T00:31:33.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:33 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:33.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:33 smithi181 ceph-mon[36706]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:31:34.001 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:31:33.733+0000 7f4854cd6640 -1 osd.8 0 waiting for initial osdmap 2024-07-29T00:31:34.002 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:31:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:31:33.744+0000 7f484caf6640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:31:34.634 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:34 smithi175 ceph-mon[32138]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:31:34.634 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:34 smithi175 ceph-mon[32138]: osdmap e57: 9 total, 8 up, 9 in 2024-07-29T00:31:34.635 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:34 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:34.635 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:34 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:34.635 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:34 smithi175 ceph-mon[32138]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:34.635 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:34 smithi175 ceph-mon[32138]: from='osd.8 ' entity='osd.8' 2024-07-29T00:31:34.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:34 smithi181 ceph-mon[36706]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:31:34.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:34 smithi181 ceph-mon[36706]: osdmap e57: 9 total, 8 up, 9 in 2024-07-29T00:31:34.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:34 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:34.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:34 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:34.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:34 smithi181 ceph-mon[36706]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:34.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:34 smithi181 ceph-mon[36706]: from='osd.8 ' entity='osd.8' 2024-07-29T00:31:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:34 smithi204 ceph-mon[37368]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:31:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:34 smithi204 ceph-mon[37368]: osdmap e57: 9 total, 8 up, 9 in 2024-07-29T00:31:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:34 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:34 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:34 smithi204 ceph-mon[37368]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-07-29T00:31:34.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:34 smithi204 ceph-mon[37368]: from='osd.8 ' entity='osd.8' 2024-07-29T00:31:35.393 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] boot 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: osdmap e58: 9 total, 9 up, 9 in 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:35.395 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:35.395 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.395 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:35 smithi175 ceph-mon[32138]: osdmap e59: 9 total, 9 up, 9 in 2024-07-29T00:31:35.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] boot 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: osdmap e58: 9 total, 9 up, 9 in 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:35.587 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:35.587 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.587 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:35 smithi204 ceph-mon[37368]: osdmap e59: 9 total, 9 up, 9 in 2024-07-29T00:31:35.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:31:35.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:31:35.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: osd.8 [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] boot 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: osdmap e58: 9 total, 9 up, 9 in 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:35.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:35.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:35 smithi181 ceph-mon[36706]: osdmap e59: 9 total, 9 up, 9 in 2024-07-29T00:31:36.298 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:31:36.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-07-29T00:31:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi204 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi204 to 5696M 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:36.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:36 smithi181 ceph-mon[36706]: osdmap e60: 9 total, 9 up, 9 in 2024-07-29T00:31:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi204 2024-07-29T00:31:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi204 to 5696M 2024-07-29T00:31:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:36.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:36.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:36 smithi175 ceph-mon[32138]: osdmap e60: 9 total, 9 up, 9 in 2024-07-29T00:31:36.810 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:31:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi204 2024-07-29T00:31:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi204 to 5696M 2024-07-29T00:31:36.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:31:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:36.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:36 smithi204 ceph-mon[37368]: osdmap e60: 9 total, 9 up, 9 in 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 -- 172.21.15.175:0/472579989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 msgr2=0x7f8d48109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 --2- 172.21.15.175:0/472579989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d48109b80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009ca0 tx=0x7f8d4002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 -- 172.21.15.175:0/472579989 shutdown_connections 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 --2- 172.21.15.175:0/472579989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d48109b80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 --2- 172.21.15.175:0/472579989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d48102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 --2- 172.21.15.175:0/472579989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 0x7f8d48101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 -- 172.21.15.175:0/472579989 >> 172.21.15.175:0/472579989 conn(0x7f8d480fcee0 msgr2=0x7f8d480ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:31:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 -- 172.21.15.175:0/472579989 shutdown_connections 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.507+0000 7f8d4f000640 1 -- 172.21.15.175:0/472579989 wait complete. 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.508+0000 7f8d4f000640 1 Processor -- start 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.508+0000 7f8d4f000640 1 -- start start 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 0x7f8d48199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d4819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d481a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4810d400 con 0x7f8d481032f0 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4810d280 con 0x7f8d48102930 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4810d580 con 0x7f8d48101730 2024-07-29T00:31:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d481a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 0x7f8d48199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d4819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d481a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43452/0 (socket says 172.21.15.175:43452) 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 0x7f8d48199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:40446/0 (socket says 172.21.15.175:40446) 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d4819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51930/0 (socket says 172.21.15.175:51930) 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.509+0000 7f8d4e600640 1 -- 172.21.15.175:0/2208474491 learned_addr learned my addr 172.21.15.175:0/2208474491 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4d200640 1 -- 172.21.15.175:0/2208474491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 msgr2=0x7f8d48199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4d200640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 0x7f8d48199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4d200640 1 -- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 msgr2=0x7f8d481a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4d200640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d481a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4d200640 1 -- 172.21.15.175:0/2208474491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d481a1800 con 0x7f8d48102930 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4d200640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d4819a110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005fd0 tx=0x7f8d3400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34009c30 con 0x7f8d48102930 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d34009dd0 con 0x7f8d48102930 2024-07-29T00:31:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34010460 con 0x7f8d48102930 2024-07-29T00:31:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d481a1af0 con 0x7f8d48102930 2024-07-29T00:31:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.510+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d481a2080 con 0x7f8d48102930 2024-07-29T00:31:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.511+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d481a2250 con 0x7f8d48102930 2024-07-29T00:31:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.514+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8d34003f50 con 0x7f8d48102930 2024-07-29T00:31:37.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.515+0000 7f8d46000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8d300a2c30 0x7f8d300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:31:37.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.515+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5829+0+0 (secure 0 0 0) 0x7f8d340ce1d0 con 0x7f8d48102930 2024-07-29T00:31:37.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.515+0000 7f8d4dc00640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8d300a2c30 0x7f8d300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:31:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.516+0000 7f8d4dc00640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8d300a2c30 0x7f8d300a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d48102790 tx=0x7f8d3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:31:37.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.517+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8d34017020 con 0x7f8d48102930 2024-07-29T00:31:37.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:31:37.714+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8d481a2420 con 0x7f8d300a2c30 2024-07-29T00:31:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:38 smithi181 ceph-mon[36706]: osdmap e61: 9 total, 9 up, 9 in 2024-07-29T00:31:38.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:38 smithi181 ceph-mon[36706]: pgmap v195: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-29T00:31:38.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:38 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:31:38.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:38 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:31:38.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:38 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:38.765 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:38 smithi175 ceph-mon[32138]: osdmap e61: 9 total, 9 up, 9 in 2024-07-29T00:31:38.765 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:38 smithi175 ceph-mon[32138]: pgmap v195: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-29T00:31:38.765 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:38 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:31:38.765 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:38 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:31:38.766 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:38 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:38 smithi204 ceph-mon[37368]: osdmap e61: 9 total, 9 up, 9 in 2024-07-29T00:31:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:38 smithi204 ceph-mon[37368]: pgmap v195: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-29T00:31:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:38 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:31:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:38 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:31:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:38 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:39.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:39 smithi181 ceph-mon[36706]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:31:39.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:39 smithi181 ceph-mon[36706]: osdmap e62: 9 total, 9 up, 9 in 2024-07-29T00:31:39.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:39 smithi175 ceph-mon[32138]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:31:39.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:39 smithi175 ceph-mon[32138]: osdmap e62: 9 total, 9 up, 9 in 2024-07-29T00:31:39.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:39 smithi204 ceph-mon[37368]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:31:39.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:39 smithi204 ceph-mon[37368]: osdmap e62: 9 total, 9 up, 9 in 2024-07-29T00:31:40.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:40 smithi181 ceph-mon[36706]: pgmap v197: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-07-29T00:31:40.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:40 smithi175 ceph-mon[32138]: pgmap v197: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-07-29T00:31:40.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:40 smithi204 ceph-mon[37368]: pgmap v197: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-07-29T00:31:41.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:41 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3375786181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c85017-3133-4b51-90da-455c226a0d6f"}]: dispatch 2024-07-29T00:31:41.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:41 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3375786181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c85017-3133-4b51-90da-455c226a0d6f"}]': finished 2024-07-29T00:31:41.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:41 smithi181 ceph-mon[36706]: osdmap e63: 10 total, 9 up, 10 in 2024-07-29T00:31:41.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:41 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:31:41.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:41 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3375786181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c85017-3133-4b51-90da-455c226a0d6f"}]: dispatch 2024-07-29T00:31:41.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:41 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3375786181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c85017-3133-4b51-90da-455c226a0d6f"}]': finished 2024-07-29T00:31:41.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:41 smithi175 ceph-mon[32138]: osdmap e63: 10 total, 9 up, 10 in 2024-07-29T00:31:41.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:31:41.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:41 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3375786181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c85017-3133-4b51-90da-455c226a0d6f"}]: dispatch 2024-07-29T00:31:41.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:41 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3375786181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c85017-3133-4b51-90da-455c226a0d6f"}]': finished 2024-07-29T00:31:41.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:41 smithi204 ceph-mon[37368]: osdmap e63: 10 total, 9 up, 10 in 2024-07-29T00:31:41.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:31:42.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:42 smithi181 ceph-mon[36706]: pgmap v199: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-07-29T00:31:42.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:42 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2267040728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:31:42.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:42 smithi175 ceph-mon[32138]: pgmap v199: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-07-29T00:31:42.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:42 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2267040728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:31:42.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:42 smithi204 ceph-mon[37368]: pgmap v199: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-07-29T00:31:42.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:42 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2267040728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:31:44.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:44 smithi204 ceph-mon[37368]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:44 smithi181 ceph-mon[36706]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:44 smithi175 ceph-mon[32138]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:46 smithi175 ceph-mon[32138]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:46.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:46 smithi204 ceph-mon[37368]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:46 smithi181 ceph-mon[36706]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:48 smithi204 ceph-mon[37368]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:48.862 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:48 smithi175 ceph-mon[32138]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:48 smithi181 ceph-mon[36706]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:50.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:50 smithi175 ceph-mon[32138]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:50.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-29T00:31:50.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:50 smithi204 ceph-mon[37368]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-29T00:31:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:50 smithi181 ceph-mon[36706]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-29T00:31:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:31:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:51 smithi204 ceph-mon[37368]: Deploying daemon osd.9 on smithi175 2024-07-29T00:31:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:51 smithi181 ceph-mon[36706]: Deploying daemon osd.9 on smithi175 2024-07-29T00:31:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:51 smithi175 ceph-mon[32138]: Deploying daemon osd.9 on smithi175 2024-07-29T00:31:52.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:52 smithi204 ceph-mon[37368]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:52 smithi181 ceph-mon[36706]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:53.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:52 smithi175 ceph-mon[32138]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:54 smithi204 ceph-mon[37368]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:54 smithi181 ceph-mon[36706]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:55.017 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:54 smithi175 ceph-mon[32138]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:56 smithi175 ceph-mon[32138]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:56 smithi204 ceph-mon[37368]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:56 smithi181 ceph-mon[36706]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:58.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:58 smithi175 ceph-mon[32138]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:58 smithi181 ceph-mon[36706]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:58 smithi204 ceph-mon[37368]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:31:59.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:59.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:59.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:31:59 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:31:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:31:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:31:59 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:00.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:00.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:31:59 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:00.854 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:00 smithi175 ceph-mon[32138]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:00 smithi181 ceph-mon[36706]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:00 smithi204 ceph-mon[37368]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:02 smithi175 ceph-mon[32138]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:02 smithi181 ceph-mon[36706]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:03.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:02 smithi204 ceph-mon[37368]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:04.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.013+0000 7f8d46000640 1 -- 172.21.15.175:0/2208474491 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d481a2420 con 0x7f8d300a2c30 2024-07-29T00:32:04.013 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 9 on host 'smithi175' 2024-07-29T00:32:04.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8d300a2c30 msgr2=0x7f8d300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:04.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8d300a2c30 0x7f8d300a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d48102790 tx=0x7f8d3c005f90 comp rx=0 tx=0).stop 2024-07-29T00:32:04.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 msgr2=0x7f8d4819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:04.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d4819a110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005fd0 tx=0x7f8d3400dde0 comp rx=0 tx=0).stop 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 shutdown_connections 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d481032f0 0x7f8d481a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48102930 0x7f8d4819a110 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8d300a2c30 0x7f8d300a50f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 --2- 172.21.15.175:0/2208474491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48101730 0x7f8d48199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.016+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 >> 172.21.15.175:0/2208474491 conn(0x7f8d480fcee0 msgr2=0x7f8d48074110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.017+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 shutdown_connections 2024-07-29T00:32:04.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:04.017+0000 7f8d4f000640 1 -- 172.21.15.175:0/2208474491 wait complete. 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-29T00:32:04.046 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:03 smithi175 ceph-mon[32138]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:04.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:04.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:04.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-29T00:32:04.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:03 smithi204 ceph-mon[37368]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:04.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-29T00:32:04.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:03 smithi181 ceph-mon[36706]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:04.948 DEBUG:teuthology.orchestra.run.smithi175:osd.9> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.9.service 2024-07-29T00:32:04.955 INFO:tasks.cephadm:Deploying osd.10 on smithi181 with /dev/vg_nvme/lv_1... 2024-07-29T00:32:04.955 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-07-29T00:32:05.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-29T00:32:05.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: osdmap e64: 10 total, 9 up, 10 in 2024-07-29T00:32:05.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:32:05.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:05.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:05.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:05.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:04 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:05.189 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-29T00:32:05.189 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: osdmap e64: 10 total, 9 up, 10 in 2024-07-29T00:32:05.190 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:32:05.190 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:05.190 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:05.190 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:05.190 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:04 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:05.215 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:32:05.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-29T00:32:05.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: osdmap e64: 10 total, 9 up, 10 in 2024-07-29T00:32:05.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-29T00:32:05.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:05.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:05.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:05.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:04 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:05 smithi181 ceph-mon[36706]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:32:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:05 smithi181 ceph-mon[36706]: osdmap e65: 10 total, 9 up, 10 in 2024-07-29T00:32:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:05 smithi181 ceph-mon[36706]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:05 smithi204 ceph-mon[37368]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:32:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:05 smithi204 ceph-mon[37368]: osdmap e65: 10 total, 9 up, 10 in 2024-07-29T00:32:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:05 smithi204 ceph-mon[37368]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:06.123 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:05 smithi175 ceph-mon[32138]: from='osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-29T00:32:06.123 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:05 smithi175 ceph-mon[32138]: osdmap e65: 10 total, 9 up, 10 in 2024-07-29T00:32:06.123 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:06.123 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:06.123 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:05 smithi175 ceph-mon[32138]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-07-29T00:32:06.926 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:06 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:32:06.926 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:06 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:32:06.926 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:06.926 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:06 smithi181 ceph-mon[36706]: osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] boot 2024-07-29T00:32:06.926 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:06 smithi181 ceph-mon[36706]: osdmap e66: 10 total, 10 up, 10 in 2024-07-29T00:32:06.927 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:06 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:06 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:32:07.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:06 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:32:07.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:07.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:06 smithi204 ceph-mon[37368]: osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] boot 2024-07-29T00:32:07.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:06 smithi204 ceph-mon[37368]: osdmap e66: 10 total, 10 up, 10 in 2024-07-29T00:32:07.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:06 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:07.099 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:06 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:32:07.099 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:06 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:32:07.099 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:07.099 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:06 smithi175 ceph-mon[32138]: osd.9 [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] boot 2024-07-29T00:32:07.099 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:06 smithi175 ceph-mon[32138]: osdmap e66: 10 total, 10 up, 10 in 2024-07-29T00:32:07.099 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:06 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:32:07.856 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-07-29T00:32:07.883 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-07-29T00:32:08.135 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:32:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:07 smithi181 ceph-mon[36706]: osdmap e67: 10 total, 10 up, 10 in 2024-07-29T00:32:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:07 smithi181 ceph-mon[36706]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:08.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:07 smithi175 ceph-mon[32138]: osdmap e67: 10 total, 10 up, 10 in 2024-07-29T00:32:08.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:07 smithi175 ceph-mon[32138]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:07 smithi204 ceph-mon[37368]: osdmap e67: 10 total, 10 up, 10 in 2024-07-29T00:32:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:07 smithi204 ceph-mon[37368]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:08.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 -- 172.21.15.181:0/978655997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 msgr2=0x7fed0810bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 --2- 172.21.15.181:0/978655997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed0810bc40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed00009900 tx=0x7fed0002a410 comp rx=0 tx=0).stop 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 -- 172.21.15.181:0/978655997 shutdown_connections 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 --2- 172.21.15.181:0/978655997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed0810bc40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 --2- 172.21.15.181:0/978655997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed08074940 0x7fed08074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 --2- 172.21.15.181:0/978655997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed08075f10 0x7fed08074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 -- 172.21.15.181:0/978655997 >> 172.21.15.181:0/978655997 conn(0x7fed080fd0b0 msgr2=0x7fed080ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 -- 172.21.15.181:0/978655997 shutdown_connections 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.625+0000 7fed0f860640 1 -- 172.21.15.181:0/978655997 wait complete. 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 Processor -- start 2024-07-29T00:32:08.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 -- start start 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed08074940 0x7fed08199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed08075f10 0x7fed0819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed081a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810f800 con 0x7fed08074940 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810f680 con 0x7fed08109780 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0f860640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810f980 con 0x7fed08075f10 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed081a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed081a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44540/0 (socket says 172.21.15.181:44540) 2024-07-29T00:32:08.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 -- 172.21.15.181:0/1483441030 learned_addr learned my addr 172.21.15.181:0/1483441030 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0cdfb640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed08075f10 0x7fed0819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0d5fc640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed08074940 0x7fed08199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 -- 172.21.15.181:0/1483441030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed08075f10 msgr2=0x7fed0819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed08075f10 0x7fed0819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 -- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed08074940 msgr2=0x7fed08199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed08074940 0x7fed08199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.626+0000 7fed0ddfd640 1 -- 172.21.15.181:0/1483441030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed081a19e0 con 0x7fed08109780 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.627+0000 7fed0ddfd640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed081a1370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed000314e0 tx=0x7fed000043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.627+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0003f030 con 0x7fed08109780 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.627+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00003ce0 con 0x7fed08109780 2024-07-29T00:32:08.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.627+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed081a1c70 con 0x7fed08109780 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.627+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00035070 con 0x7fed08109780 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.627+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed081a2150 con 0x7fed08109780 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.628+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed08108500 con 0x7fed08109780 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.629+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fed00042030 con 0x7fed08109780 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.630+0000 7fed067fc640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fecf00a2bb0 0x7fecf00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.630+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6261+0+0 (secure 0 0 0) 0x7fed000eefa0 con 0x7fed08109780 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.630+0000 7fed0d5fc640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fecf00a2bb0 0x7fecf00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.631+0000 7fed0d5fc640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fecf00a2bb0 0x7fecf00a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fed080762f0 tx=0x7fecf80094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:08.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.634+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fed000bc700 con 0x7fed08109780 2024-07-29T00:32:08.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:08.809+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fed08062320 con 0x7fecf00a2bb0 2024-07-29T00:32:09.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:32:09.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:32:09.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:09.233 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:32:09.233 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:32:09.233 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:32:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:32:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:09 smithi175 ceph-mon[32138]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:32:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:09 smithi175 ceph-mon[32138]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:10.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:09 smithi181 ceph-mon[36706]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:32:10.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:09 smithi181 ceph-mon[36706]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:10.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:09 smithi204 ceph-mon[37368]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:32:10.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:09 smithi204 ceph-mon[37368]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi175 2024-07-29T00:32:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi175 to 4272M 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:11.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:10 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi175 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi175 to 4272M 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:11.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:10 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi175 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi175 to 4272M 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:11.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:11.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:10 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/4088519566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]: dispatch 2024-07-29T00:32:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]: dispatch 2024-07-29T00:32:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]': finished 2024-07-29T00:32:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: osdmap e68: 11 total, 10 up, 11 in 2024-07-29T00:32:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:11 smithi181 ceph-mon[36706]: from='client.? 172.21.15.181:0/671574210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:32:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/4088519566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]: dispatch 2024-07-29T00:32:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]: dispatch 2024-07-29T00:32:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]': finished 2024-07-29T00:32:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: osdmap e68: 11 total, 10 up, 11 in 2024-07-29T00:32:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:11 smithi175 ceph-mon[32138]: from='client.? 172.21.15.181:0/671574210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:32:12.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/4088519566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]: dispatch 2024-07-29T00:32:12.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]: dispatch 2024-07-29T00:32:12.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d33f82ef-0e35-45ca-ba0c-27da92a81a15"}]': finished 2024-07-29T00:32:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: osdmap e68: 11 total, 10 up, 11 in 2024-07-29T00:32:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:11 smithi204 ceph-mon[37368]: from='client.? 172.21.15.181:0/671574210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:32:14.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:14 smithi175 ceph-mon[32138]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:14.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:14 smithi204 ceph-mon[37368]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:14 smithi181 ceph-mon[36706]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:16.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:16 smithi204 ceph-mon[37368]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:16.944 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:16 smithi181 ceph-mon[36706]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:16 smithi175 ceph-mon[32138]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:18.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:18 smithi204 ceph-mon[37368]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:18.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-29T00:32:18.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:18.867 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:18 smithi181 ceph-mon[36706]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:18.867 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-29T00:32:18.867 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:19.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:18 smithi175 ceph-mon[32138]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:19.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-29T00:32:19.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:19.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:19 smithi181 ceph-mon[36706]: Deploying daemon osd.10 on smithi181 2024-07-29T00:32:19.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:19 smithi204 ceph-mon[37368]: Deploying daemon osd.10 on smithi181 2024-07-29T00:32:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:19 smithi175 ceph-mon[32138]: Deploying daemon osd.10 on smithi181 2024-07-29T00:32:20.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:20 smithi181 ceph-mon[36706]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:20.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:20 smithi204 ceph-mon[37368]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:21.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:20 smithi175 ceph-mon[32138]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:22.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:22 smithi181 ceph-mon[36706]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:22 smithi204 ceph-mon[37368]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:22.837 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:22 smithi175 ceph-mon[32138]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:24 smithi175 ceph-mon[32138]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:24.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:24.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:25.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:24 smithi181 ceph-mon[36706]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:25.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:25.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:25.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:24 smithi204 ceph-mon[37368]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:25.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:25.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:25.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:26.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:26 smithi181 ceph-mon[36706]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:26.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:26.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:26.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:26.951 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:26.952 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:26 smithi175 ceph-mon[32138]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:27.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:27.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:27.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:26 smithi204 ceph-mon[37368]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:27.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.343 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 10 on host 'smithi181' 2024-07-29T00:32:27.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.342+0000 7fed067fc640 1 -- 172.21.15.181:0/1483441030 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fed08062320 con 0x7fecf00a2bb0 2024-07-29T00:32:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fecf00a2bb0 msgr2=0x7fecf00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fecf00a2bb0 0x7fecf00a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fed080762f0 tx=0x7fecf80094f0 comp rx=0 tx=0).stop 2024-07-29T00:32:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 msgr2=0x7fed081a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed081a1370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed000314e0 tx=0x7fed000043c0 comp rx=0 tx=0).stop 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 shutdown_connections 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08109780 0x7fed081a1370 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fecf00a2bb0 0x7fecf00a5070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed08075f10 0x7fed0819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 --2- 172.21.15.181:0/1483441030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed08074940 0x7fed08199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 >> 172.21.15.181:0/1483441030 conn(0x7fed080fd0b0 msgr2=0x7fed081056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 shutdown_connections 2024-07-29T00:32:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-07-29T00:32:27.345+0000 7fed0f860640 1 -- 172.21.15.181:0/1483441030 wait complete. 2024-07-29T00:32:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:27 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:27 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:27 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:27 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:27 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:27 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:28.045 DEBUG:teuthology.orchestra.run.smithi181:osd.10> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.10.service 2024-07-29T00:32:28.047 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-07-29T00:32:28.048 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-07-29T00:32:28.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:28.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:28.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:28.310 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:32:28.741 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:28 smithi204 ceph-mon[37368]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:28.741 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:28 smithi204 ceph-mon[37368]: from='osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-29T00:32:28.741 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:28 smithi204 ceph-mon[37368]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-29T00:32:28.936 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:28 smithi181 ceph-mon[36706]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:28.936 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:28 smithi181 ceph-mon[36706]: from='osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-29T00:32:28.936 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:28 smithi181 ceph-mon[36706]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-29T00:32:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:28 smithi175 ceph-mon[32138]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:28 smithi175 ceph-mon[32138]: from='osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-29T00:32:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:28 smithi175 ceph-mon[32138]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-29T00:32:29.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:29 smithi204 ceph-mon[37368]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-29T00:32:29.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:29 smithi204 ceph-mon[37368]: from='osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:32:29.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:29 smithi204 ceph-mon[37368]: osdmap e69: 11 total, 10 up, 11 in 2024-07-29T00:32:29.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:29 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:29.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:29 smithi204 ceph-mon[37368]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:32:29.783 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:29 smithi204 ceph-mon[37368]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:32:30.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-mon[36706]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-29T00:32:30.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-mon[36706]: from='osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:32:30.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-mon[36706]: osdmap e69: 11 total, 10 up, 11 in 2024-07-29T00:32:30.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-mon[36706]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:32:30.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-mon[36706]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:32:30.001 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:32:29.612+0000 7fa0cea34640 -1 osd.10 0 waiting for initial osdmap 2024-07-29T00:32:30.001 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:32:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:32:29.625+0000 7fa0ca85c640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:32:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:29 smithi175 ceph-mon[32138]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-29T00:32:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:29 smithi175 ceph-mon[32138]: from='osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:32:30.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:29 smithi175 ceph-mon[32138]: osdmap e69: 11 total, 10 up, 11 in 2024-07-29T00:32:30.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:29 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:29 smithi175 ceph-mon[32138]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-07-29T00:32:30.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:29 smithi175 ceph-mon[32138]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: osdmap e70: 11 total, 10 up, 11 in 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] boot 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: osdmap e71: 11 total, 11 up, 11 in 2024-07-29T00:32:30.931 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:30 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.934 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-07-29T00:32:30.944 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:30.944 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: osdmap e70: 11 total, 10 up, 11 in 2024-07-29T00:32:30.944 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.945 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.945 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] boot 2024-07-29T00:32:30.945 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: osdmap e71: 11 total, 11 up, 11 in 2024-07-29T00:32:30.945 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:30 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:30.965 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-07-29T00:32:31.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-07-29T00:32:31.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: osdmap e70: 11 total, 10 up, 11 in 2024-07-29T00:32:31.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:31.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:31.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: osd.10 [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] boot 2024-07-29T00:32:31.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: osdmap e71: 11 total, 11 up, 11 in 2024-07-29T00:32:31.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:30 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:32:31.265 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 -- 172.21.15.204:0/1740728325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 msgr2=0x7f304c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 --2- 172.21.15.204:0/1740728325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c100bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3048009900 tx=0x7f304802a410 comp rx=0 tx=0).stop 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 -- 172.21.15.204:0/1740728325 shutdown_connections 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 --2- 172.21.15.204:0/1740728325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f304c102380 0x7f304c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 --2- 172.21.15.204:0/1740728325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c1019c0 0x7f304c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 --2- 172.21.15.204:0/1740728325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c100bc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 -- 172.21.15.204:0/1740728325 >> 172.21.15.204:0/1740728325 conn(0x7f304c0fbf70 msgr2=0x7f304c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 -- 172.21.15.204:0/1740728325 shutdown_connections 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 -- 172.21.15.204:0/1740728325 wait complete. 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 Processor -- start 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.708+0000 7f30540d5640 1 -- start start 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f304c1019c0 0x7f304c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c102380 0x7f304c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10c4a0 con 0x7f304c1019c0 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10c320 con 0x7f304c102380 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10c620 con 0x7f304c1007c0 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55192/0 (socket says 172.21.15.204:55192) 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 -- 172.21.15.204:0/722888829 learned_addr learned my addr 172.21.15.204:0/722888829 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3052672640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c102380 0x7f304c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 -- 172.21.15.204:0/722888829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c102380 msgr2=0x7f304c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c102380 0x7f304c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 -- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f304c1019c0 msgr2=0x7f304c1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f304c1019c0 0x7f304c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 -- 172.21.15.204:0/722888829 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c19e790 con 0x7f304c1007c0 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3052672640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c102380 0x7f304c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3051e71640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c196b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3048031610 tx=0x7f30480043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304803f030 con 0x7f304c1007c0 2024-07-29T00:32:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048003ce0 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048035070 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.709+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c19ea20 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.710+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c19efd0 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.711+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c107c60 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.711+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3048042030 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.712+0000 7f3042ffd640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f30380a2c30 0x7f30380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.712+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6723+0+0 (secure 0 0 0) 0x7f30480eeb70 con 0x7f304c1007c0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.712+0000 7f3051670640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f30380a2c30 0x7f30380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.713+0000 7f3051670640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f30380a2c30 0x7f30380a50f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f303c005fd0 tx=0x7f303c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.716+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f30480bc100 con 0x7f304c1007c0 2024-07-29T00:32:31.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:31.894+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f304c062320 con 0x7f30380a2c30 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:31.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:31.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:31.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:31.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:31.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:31.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:31 smithi204 ceph-mon[37368]: osdmap e72: 11 total, 11 up, 11 in 2024-07-29T00:32:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:32.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:32.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:31 smithi181 ceph-mon[36706]: osdmap e72: 11 total, 11 up, 11 in 2024-07-29T00:32:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:32.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:31 smithi175 ceph-mon[32138]: osdmap e72: 11 total, 11 up, 11 in 2024-07-29T00:32:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi181 2024-07-29T00:32:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi181 to 4272M 2024-07-29T00:32:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:32:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:32:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:32 smithi181 ceph-mon[36706]: osdmap e73: 11 total, 11 up, 11 in 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi181 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi181 to 4272M 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:32 smithi175 ceph-mon[32138]: osdmap e73: 11 total, 11 up, 11 in 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi181 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi181 to 4272M 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:33.042 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:32 smithi204 ceph-mon[37368]: osdmap e73: 11 total, 11 up, 11 in 2024-07-29T00:32:33.706 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:33 smithi204 ceph-mon[37368]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:32:33.706 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:33 smithi204 ceph-mon[37368]: osdmap e74: 11 total, 11 up, 11 in 2024-07-29T00:32:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:33 smithi181 ceph-mon[36706]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:32:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:33 smithi181 ceph-mon[36706]: osdmap e74: 11 total, 11 up, 11 in 2024-07-29T00:32:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:33 smithi175 ceph-mon[32138]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:32:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:33 smithi175 ceph-mon[32138]: osdmap e74: 11 total, 11 up, 11 in 2024-07-29T00:32:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: osdmap e75: 11 total, 11 up, 11 in 2024-07-29T00:32:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/1742313888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]: dispatch 2024-07-29T00:32:35.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]: dispatch 2024-07-29T00:32:35.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]': finished 2024-07-29T00:32:35.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: osdmap e76: 12 total, 11 up, 12 in 2024-07-29T00:32:35.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:34 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: osdmap e75: 11 total, 11 up, 11 in 2024-07-29T00:32:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/1742313888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]: dispatch 2024-07-29T00:32:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]: dispatch 2024-07-29T00:32:35.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]': finished 2024-07-29T00:32:35.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: osdmap e76: 12 total, 11 up, 12 in 2024-07-29T00:32:35.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:34 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: osdmap e75: 11 total, 11 up, 11 in 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/1742313888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]: dispatch 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]: dispatch 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a28bd9a6-023c-4dd3-8acd-441f4bc106b2"}]': finished 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: osdmap e76: 12 total, 11 up, 12 in 2024-07-29T00:32:35.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:34 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:35 smithi175 ceph-mon[32138]: from='client.? 172.21.15.204:0/810066608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:32:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:35 smithi181 ceph-mon[36706]: from='client.? 172.21.15.204:0/810066608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:32:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:35 smithi204 ceph-mon[37368]: from='client.? 172.21.15.204:0/810066608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-29T00:32:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:36 smithi181 ceph-mon[36706]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:37.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:36 smithi175 ceph-mon[32138]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:36 smithi204 ceph-mon[37368]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:38 smithi181 ceph-mon[36706]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-29T00:32:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:38 smithi175 ceph-mon[32138]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-29T00:32:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:38 smithi204 ceph-mon[37368]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-29T00:32:40.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:40 smithi204 ceph-mon[37368]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-07-29T00:32:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:40 smithi181 ceph-mon[36706]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-07-29T00:32:41.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:40 smithi175 ceph-mon[32138]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-07-29T00:32:41.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:41 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-29T00:32:41.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:41 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:42.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-29T00:32:42.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:42.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-29T00:32:42.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:42.973 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:42 smithi204 ceph-mon[37368]: Deploying daemon osd.11 on smithi204 2024-07-29T00:32:42.973 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:42 smithi204 ceph-mon[37368]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:32:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:42 smithi181 ceph-mon[36706]: Deploying daemon osd.11 on smithi204 2024-07-29T00:32:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:42 smithi181 ceph-mon[36706]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:32:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:42 smithi175 ceph-mon[32138]: Deploying daemon osd.11 on smithi204 2024-07-29T00:32:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:42 smithi175 ceph-mon[32138]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-29T00:32:44.838 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:44 smithi204 ceph-mon[37368]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-07-29T00:32:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:44 smithi181 ceph-mon[36706]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-07-29T00:32:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:44 smithi175 ceph-mon[32138]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-07-29T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:46 smithi181 ceph-mon[36706]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-07-29T00:32:47.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:46 smithi175 ceph-mon[32138]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-07-29T00:32:47.059 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:46 smithi204 ceph-mon[37368]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-07-29T00:32:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:48.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:48 smithi181 ceph-mon[36706]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-29T00:32:49.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:48 smithi175 ceph-mon[32138]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-29T00:32:49.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:48 smithi204 ceph-mon[37368]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-29T00:32:50.482 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-07-29T00:32:50.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.482+0000 7f3042ffd640 1 -- 172.21.15.204:0/722888829 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f304c062320 con 0x7f30380a2c30 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f30380a2c30 msgr2=0x7f30380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f30380a2c30 0x7f30380a50f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f303c005fd0 tx=0x7f303c005850 comp rx=0 tx=0).stop 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 msgr2=0x7f304c196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c196b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3048031610 tx=0x7f30480043b0 comp rx=0 tx=0).stop 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 shutdown_connections 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f304c102380 0x7f304c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f304c1019c0 0x7f304c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f30380a2c30 0x7f30380a50f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:50.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 --2- 172.21.15.204:0/722888829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f304c1007c0 0x7f304c196b60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:50.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 >> 172.21.15.204:0/722888829 conn(0x7f304c0fbf70 msgr2=0x7f304c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:50.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 shutdown_connections 2024-07-29T00:32:50.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:32:50.484+0000 7f30540d5640 1 -- 172.21.15.204:0/722888829 wait complete. 2024-07-29T00:32:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-29T00:32:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:50 smithi181 ceph-mon[36706]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-29T00:32:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-29T00:32:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:50 smithi175 ceph-mon[32138]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-29T00:32:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-29T00:32:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:50 smithi204 ceph-mon[37368]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-29T00:32:51.156 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.11.service 2024-07-29T00:32:51.158 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-07-29T00:32:51.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd stat -f json 2024-07-29T00:32:51.620 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 -- 172.21.15.175:0/4057381348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c0101730 msgr2=0x7fc0c0101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 --2- 172.21.15.175:0/4057381348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c0101730 0x7fc0c0101b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009900 tx=0x7fc0b402a3f0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 -- 172.21.15.175:0/4057381348 shutdown_connections 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 --2- 172.21.15.175:0/4057381348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c01032f0 0x7fc0c0109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 --2- 172.21.15.175:0/4057381348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c0102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 --2- 172.21.15.175:0/4057381348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c0101730 0x7fc0c0101b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 -- 172.21.15.175:0/4057381348 >> 172.21.15.175:0/4057381348 conn(0x7fc0c00fcee0 msgr2=0x7fc0c00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 -- 172.21.15.175:0/4057381348 shutdown_connections 2024-07-29T00:32:52.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.271+0000 7fc0c6400640 1 -- 172.21.15.175:0/4057381348 wait complete. 2024-07-29T00:32:52.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 Processor -- start 2024-07-29T00:32:52.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 -- start start 2024-07-29T00:32:52.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c0101730 0x7fc0c0199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:52.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c01032f0 0x7fc0c01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c010d410 con 0x7fc0c01032f0 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c010d290 con 0x7fc0c0102930 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.272+0000 7fc0c6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c010d590 con 0x7fc0c0101730 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:45272/0 (socket says 172.21.15.175:45272) 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 -- 172.21.15.175:0/1361446822 learned_addr learned my addr 172.21.15.175:0/1361446822 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0c5a00640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c01032f0 0x7fc0c01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0c5000640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c0101730 0x7fc0c0199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 -- 172.21.15.175:0/1361446822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c0101730 msgr2=0x7fc0c0199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:52.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c0101730 0x7fc0c0199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 -- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c01032f0 msgr2=0x7fc0c01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c01032f0 0x7fc0c01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.273+0000 7fc0bfe00640 1 -- 172.21.15.175:0/1361446822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c01a1850 con 0x7fc0c0102930 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.274+0000 7fc0bfe00640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c019a170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00d950 tx=0x7fc0ac00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.274+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac017030 con 0x7fc0c0102930 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.274+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c01a1b40 con 0x7fc0c0102930 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.274+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac004020 con 0x7fc0c0102930 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.274+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0047a0 con 0x7fc0c0102930 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.274+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c01a20a0 con 0x7fc0c0102930 2024-07-29T00:32:52.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.275+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c01a22d0 con 0x7fc0c0102930 2024-07-29T00:32:52.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.277+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc0ac0041d0 con 0x7fc0c0102930 2024-07-29T00:32:52.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.279+0000 7fc0bd600640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc0a80a2b80 0x7fc0a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:52.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.279+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6831+0+0 (secure 0 0 0) 0x7fc0ac0ce740 con 0x7fc0c0102930 2024-07-29T00:32:52.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.280+0000 7fc0c5000640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc0a80a2b80 0x7fc0a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:52.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.280+0000 7fc0c5000640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc0a80a2b80 0x7fc0a80a5040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0b402a9e0 tx=0x7fc0b40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:52.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.281+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc0ac09bbd0 con 0x7fc0c0102930 2024-07-29T00:32:52.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.471+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc0c01a24a0 con 0x7fc0c0102930 2024-07-29T00:32:52.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.472+0000 7fc0bd600640 1 -- 172.21.15.175:0/1361446822 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7fc0ac09b4d0 con 0x7fc0c0102930 2024-07-29T00:32:52.473 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:32:52.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.475+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc0a80a2b80 msgr2=0x7fc0a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:52.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.475+0000 7fc0c6400640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc0a80a2b80 0x7fc0a80a5040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0b402a9e0 tx=0x7fc0b40315b0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.475+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 msgr2=0x7fc0c019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:52.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.475+0000 7fc0c6400640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c019a170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00d950 tx=0x7fc0ac00de10 comp rx=0 tx=0).stop 2024-07-29T00:32:52.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 shutdown_connections 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0c01032f0 0x7fc0c01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0c0102930 0x7fc0c019a170 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc0a80a2b80 0x7fc0a80a5040 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 --2- 172.21.15.175:0/1361446822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c0101730 0x7fc0c0199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 >> 172.21.15.175:0/1361446822 conn(0x7fc0c00fcee0 msgr2=0x7fc0c0074110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 shutdown_connections 2024-07-29T00:32:52.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:52.476+0000 7fc0c6400640 1 -- 172.21.15.175:0/1361446822 wait complete. 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: osdmap e77: 12 total, 11 up, 12 in 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: from='osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:52.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:52 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1361446822' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-29T00:32:52.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-29T00:32:52.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: osdmap e77: 12 total, 11 up, 12 in 2024-07-29T00:32:52.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:52.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: from='osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:32:52.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:32:52.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:52.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1361446822' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-29T00:32:52.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-29T00:32:52.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: osdmap e77: 12 total, 11 up, 12 in 2024-07-29T00:32:52.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: from='osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:32:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-07-29T00:32:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:52.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:52 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1361446822' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-29T00:32:52.951 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:32:52.937+0000 7f11abccf640 -1 osd.11 0 waiting for initial osdmap 2024-07-29T00:32:53.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:32:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:32:52.951+0000 7f11a3aef640 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-29T00:32:53.470 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1722213150,"num_in_osds":12,"osd_in_since":1722213154,"num_remapped_pgs":0} 2024-07-29T00:32:53.636 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:53 smithi175 ceph-mon[32138]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:32:53.636 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:53 smithi175 ceph-mon[32138]: osdmap e78: 12 total, 11 up, 12 in 2024-07-29T00:32:53.636 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:53.636 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:53 smithi175 ceph-mon[32138]: from='osd.11 ' entity='osd.11' 2024-07-29T00:32:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:53 smithi181 ceph-mon[36706]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:32:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:53 smithi181 ceph-mon[36706]: osdmap e78: 12 total, 11 up, 12 in 2024-07-29T00:32:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:53.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:53 smithi181 ceph-mon[36706]: from='osd.11 ' entity='osd.11' 2024-07-29T00:32:53.812 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:53 smithi204 ceph-mon[37368]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-07-29T00:32:53.812 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:53 smithi204 ceph-mon[37368]: osdmap e78: 12 total, 11 up, 12 in 2024-07-29T00:32:53.812 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:53.812 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:53 smithi204 ceph-mon[37368]: from='osd.11 ' entity='osd.11' 2024-07-29T00:32:54.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd stat -f json 2024-07-29T00:32:54.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: purged_snaps scrub starts 2024-07-29T00:32:54.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: purged_snaps scrub ok 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] boot 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: osdmap e79: 12 total, 12 up, 12 in 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.536 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:54.536 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:54.536 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:54 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: purged_snaps scrub starts 2024-07-29T00:32:54.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: purged_snaps scrub ok 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] boot 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: osdmap e79: 12 total, 12 up, 12 in 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:54.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:54.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:54 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: purged_snaps scrub starts 2024-07-29T00:32:54.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: purged_snaps scrub ok 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: osd.11 [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] boot 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: osdmap e79: 12 total, 12 up, 12 in 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:32:54.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:32:54.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:54 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:32:54.923 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:32:55.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 -- 172.21.15.175:0/820166473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8078480 msgr2=0x7fc2a8078d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:55.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 --2- 172.21.15.175:0/820166473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8078480 0x7fc2a8078d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009ca0 tx=0x7fc2a002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 -- 172.21.15.175:0/820166473 shutdown_connections 2024-07-29T00:32:55.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 --2- 172.21.15.175:0/820166473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8078480 0x7fc2a8078d20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 --2- 172.21.15.175:0/820166473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 0x7fc2a8077f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 --2- 172.21.15.175:0/820166473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a8076890 0x7fc2a8076c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 -- 172.21.15.175:0/820166473 >> 172.21.15.175:0/820166473 conn(0x7fc2a80ff2a0 msgr2=0x7fc2a81016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 -- 172.21.15.175:0/820166473 shutdown_connections 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.585+0000 7fc2aca00640 1 -- 172.21.15.175:0/820166473 wait complete. 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 Processor -- start 2024-07-29T00:32:55.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 -- start start 2024-07-29T00:32:55.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8076890 0x7fc2a819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:55.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 0x7fc2a819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:55.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a8078480 0x7fc2a81a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8113c00 con 0x7fc2a8077ae0 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8113a80 con 0x7fc2a8078480 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.586+0000 7fc2aca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8113d80 con 0x7fc2a8076890 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc29d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 0x7fc2a819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc29d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 0x7fc2a819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37292/0 (socket says 172.21.15.175:37292) 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc29d400640 1 -- 172.21.15.175:0/1244576639 learned_addr learned my addr 172.21.15.175:0/1244576639 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8076890 0x7fc2a819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a6800640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a8078480 0x7fc2a81a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 -- 172.21.15.175:0/1244576639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a8078480 msgr2=0x7fc2a81a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a8078480 0x7fc2a81a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 -- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 msgr2=0x7fc2a819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 0x7fc2a819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 -- 172.21.15.175:0/1244576639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a81a5da0 con 0x7fc2a8076890 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.587+0000 7fc2a5e00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8076890 0x7fc2a819e170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9f0 tx=0x7fc29800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.588+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29800ccc0 con 0x7fc2a8076890 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.588+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298005020 con 0x7fc2a8076890 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.588+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980049e0 con 0x7fc2a8076890 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.588+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a81a6090 con 0x7fc2a8076890 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.588+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a6580 con 0x7fc2a8076890 2024-07-29T00:32:55.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.589+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a8110470 con 0x7fc2a8076890 2024-07-29T00:32:55.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.591+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc298005560 con 0x7fc2a8076890 2024-07-29T00:32:55.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.592+0000 7fc29f400640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc28c0a2bb0 0x7fc28c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:55.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.592+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fc2980ceb60 con 0x7fc2a8076890 2024-07-29T00:32:55.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.592+0000 7fc29d400640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc28c0a2bb0 0x7fc28c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:55.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.593+0000 7fc29d400640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc28c0a2bb0 0x7fc28c0a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2a819f690 tx=0x7fc290005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:55.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.597+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc2980051d0 con 0x7fc2a8076890 2024-07-29T00:32:55.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:55 smithi175 ceph-mon[32138]: Detected new or changed devices on smithi204 2024-07-29T00:32:55.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:55 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi204 to 4272M 2024-07-29T00:32:55.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.787+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc2a8062ae0 con 0x7fc2a8076890 2024-07-29T00:32:55.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.788+0000 7fc29f400640 1 -- 172.21.15.175:0/1244576639 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7fc29809bec0 con 0x7fc2a8076890 2024-07-29T00:32:55.789 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:32:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.791+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc28c0a2bb0 msgr2=0x7fc28c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.791+0000 7fc2aca00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc28c0a2bb0 0x7fc28c0a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2a819f690 tx=0x7fc290005f90 comp rx=0 tx=0).stop 2024-07-29T00:32:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.791+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8076890 msgr2=0x7fc2a819e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.791+0000 7fc2aca00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8076890 0x7fc2a819e170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9f0 tx=0x7fc29800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.791+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 shutdown_connections 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.791+0000 7fc2aca00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a8078480 0x7fc2a81a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.792+0000 7fc2aca00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8077ae0 0x7fc2a819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.792+0000 7fc2aca00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc28c0a2bb0 0x7fc28c0a5070 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.792+0000 7fc2aca00640 1 --2- 172.21.15.175:0/1244576639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2a8076890 0x7fc2a819e170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.792+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 >> 172.21.15.175:0/1244576639 conn(0x7fc2a80ff2a0 msgr2=0x7fc2a8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.792+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 shutdown_connections 2024-07-29T00:32:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:55.792+0000 7fc2aca00640 1 -- 172.21.15.175:0/1244576639 wait complete. 2024-07-29T00:32:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:55 smithi204 ceph-mon[37368]: Detected new or changed devices on smithi204 2024-07-29T00:32:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:55 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi204 to 4272M 2024-07-29T00:32:55.923 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:55 smithi181 ceph-mon[36706]: Detected new or changed devices on smithi204 2024-07-29T00:32:55.923 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:55 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi204 to 4272M 2024-07-29T00:32:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:56 smithi175 ceph-mon[32138]: osdmap e80: 12 total, 12 up, 12 in 2024-07-29T00:32:56.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:56 smithi175 ceph-mon[32138]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:32:56.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:56 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1244576639' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-29T00:32:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:56 smithi204 ceph-mon[37368]: osdmap e80: 12 total, 12 up, 12 in 2024-07-29T00:32:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:56 smithi204 ceph-mon[37368]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:32:56.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:56 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1244576639' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-29T00:32:56.852 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:56 smithi181 ceph-mon[36706]: osdmap e80: 12 total, 12 up, 12 in 2024-07-29T00:32:56.852 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:56 smithi181 ceph-mon[36706]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:32:56.852 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:56 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1244576639' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-29T00:32:56.943 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1722213173,"num_in_osds":12,"osd_in_since":1722213154,"num_remapped_pgs":0} 2024-07-29T00:32:56.944 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:32:57.396 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:32:58.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 -- 172.21.15.175:0/3771661619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728104430 msgr2=0x7f9728110960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:58.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 --2- 172.21.15.175:0/3771661619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728104430 0x7f9728110960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9720009cd0 tx=0x7f972002a8e0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 -- 172.21.15.175:0/3771661619 shutdown_connections 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 --2- 172.21.15.175:0/3771661619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728104430 0x7f9728110960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 --2- 172.21.15.175:0/3771661619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9728103a90 0x7f9728103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 --2- 172.21.15.175:0/3771661619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728109a90 0x7f9728109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 -- 172.21.15.175:0/3771661619 >> 172.21.15.175:0/3771661619 conn(0x7f97280ff710 msgr2=0x7f9728101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 -- 172.21.15.175:0/3771661619 shutdown_connections 2024-07-29T00:32:58.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.036+0000 7f9730200640 1 -- 172.21.15.175:0/3771661619 wait complete. 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.037+0000 7f9730200640 1 Processor -- start 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.037+0000 7f9730200640 1 -- start start 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f9730200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 0x7f972819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f9730200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9728104430 0x7f972819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f9730200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728109a90 0x7f972819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f9730200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9728113ee0 con 0x7f9728109a90 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f9730200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9728113d60 con 0x7f9728104430 2024-07-29T00:32:58.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f9730200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9728114060 con 0x7f9728103a90 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f972d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 0x7f972819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f972d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 0x7f972819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42696/0 (socket says 172.21.15.175:42696) 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f972d800640 1 -- 172.21.15.175:0/551175617 learned_addr learned my addr 172.21.15.175:0/551175617 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f972e200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728109a90 0x7f972819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.038+0000 7f972ce00640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9728104430 0x7f972819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f972d800640 1 -- 172.21.15.175:0/551175617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9728104430 msgr2=0x7f972819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f972d800640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9728104430 0x7f972819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f972d800640 1 -- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728109a90 msgr2=0x7f972819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f972d800640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728109a90 0x7f972819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f972d800640 1 -- 172.21.15.175:0/551175617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972819f2e0 con 0x7f9728103a90 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f972d800640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 0x7f972819a330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f971c00ea60 tx=0x7f971c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:58.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c00cde0 con 0x7f9728103a90 2024-07-29T00:32:58.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972819f5d0 con 0x7f9728103a90 2024-07-29T00:32:58.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f971c0035f0 con 0x7f9728103a90 2024-07-29T00:32:58.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c010670 con 0x7f9728103a90 2024-07-29T00:32:58.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.039+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a6eb0 con 0x7f9728103a90 2024-07-29T00:32:58.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.041+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9728103ef0 con 0x7f9728103a90 2024-07-29T00:32:58.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.042+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f971c0037a0 con 0x7f9728103a90 2024-07-29T00:32:58.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.044+0000 7f9726000640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f97100a2c00 0x7f97100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:32:58.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.044+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f971c0cee30 con 0x7f9728103a90 2024-07-29T00:32:58.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.044+0000 7f972ce00640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f97100a2c00 0x7f97100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:32:58.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.045+0000 7f972ce00640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f97100a2c00 0x7f97100a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9728109090 tx=0x7f9714005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:32:58.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.048+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f971c09c190 con 0x7f9728103a90 2024-07-29T00:32:58.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.237+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f972810ba90 con 0x7f9728103a90 2024-07-29T00:32:58.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.238+0000 7f9726000640 1 -- 172.21.15.175:0/551175617 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+18500 (secure 0 0 0) 0x7f971c09ba90 con 0x7f9728103a90 2024-07-29T00:32:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:32:58.240 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":80,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:32:55.498667+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:32:58.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f97100a2c00 msgr2=0x7f97100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:58.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f97100a2c00 0x7f97100a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9728109090 tx=0x7f9714005f90 comp rx=0 tx=0).stop 2024-07-29T00:32:58.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 msgr2=0x7f972819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:32:58.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 0x7f972819a330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f971c00ea60 tx=0x7f971c00ef20 comp rx=0 tx=0).stop 2024-07-29T00:32:58.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 shutdown_connections 2024-07-29T00:32:58.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728109a90 0x7f972819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9728104430 0x7f972819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f97100a2c00 0x7f97100a50c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 --2- 172.21.15.175:0/551175617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9728103a90 0x7f972819a330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:32:58.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 >> 172.21.15.175:0/551175617 conn(0x7f97280ff710 msgr2=0x7f9728100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:32:58.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 shutdown_connections 2024-07-29T00:32:58.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:32:58.242+0000 7f9730200640 1 -- 172.21.15.175:0/551175617 wait complete. 2024-07-29T00:32:58.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:58 smithi175 ceph-mon[32138]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:32:58.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:32:58 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/551175617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:32:58.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:58 smithi204 ceph-mon[37368]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:32:58.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:32:58 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/551175617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:32:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:58 smithi181 ceph-mon[36706]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:32:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:32:58 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/551175617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:32:59.126 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-29T00:29:04.051354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-29T00:32:59.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-29T00:32:59.587 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:00.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.270+0000 7f65a9e00640 1 -- 172.21.15.175:0/3676547811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 msgr2=0x7f65a4109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:00.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.270+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3676547811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a4109e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6594009930 tx=0x7f659402a440 comp rx=0 tx=0).stop 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 -- 172.21.15.175:0/3676547811 shutdown_connections 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3676547811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 0x7f65a4110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3676547811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a4103a80 0x7f65a4103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3676547811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a4109e60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 -- 172.21.15.175:0/3676547811 >> 172.21.15.175:0/3676547811 conn(0x7f65a40ff760 msgr2=0x7f65a4101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 -- 172.21.15.175:0/3676547811 shutdown_connections 2024-07-29T00:33:00.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.271+0000 7f65a9e00640 1 -- 172.21.15.175:0/3676547811 wait complete. 2024-07-29T00:33:00.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.272+0000 7f65a9e00640 1 Processor -- start 2024-07-29T00:33:00.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.272+0000 7f65a9e00640 1 -- start start 2024-07-29T00:33:00.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a4103a80 0x7f65a419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:00.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 0x7f65a419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:00.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4113e90 con 0x7f65a4109a80 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a9e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4113d10 con 0x7f65a4103a80 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a9e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4114010 con 0x7f65a4104420 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37236/0 (socket says 172.21.15.175:37236) 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a3400640 1 -- 172.21.15.175:0/3688570785 learned_addr learned my addr 172.21.15.175:0/3688570785 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f659be00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 0x7f65a419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:00.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.273+0000 7f65a2a00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a4103a80 0x7f65a419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a3400640 1 -- 172.21.15.175:0/3688570785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 msgr2=0x7f65a419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a3400640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 0x7f65a419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a3400640 1 -- 172.21.15.175:0/3688570785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a4103a80 msgr2=0x7f65a419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a3400640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a4103a80 0x7f65a419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a3400640 1 -- 172.21.15.175:0/3688570785 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a419f340 con 0x7f65a4109a80 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f659be00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 0x7f65a419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:00.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a3400640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a419ec60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f659c00eb10 tx=0x7f659c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:00.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c004b20 con 0x7f65a4109a80 2024-07-29T00:33:00.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c004cc0 con 0x7f65a4109a80 2024-07-29T00:33:00.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c010760 con 0x7f65a4109a80 2024-07-29T00:33:00.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.274+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a419f630 con 0x7f65a4109a80 2024-07-29T00:33:00.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.275+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a41a6f10 con 0x7f65a4109a80 2024-07-29T00:33:00.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.276+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a4103ee0 con 0x7f65a4109a80 2024-07-29T00:33:00.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.278+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f659c005020 con 0x7f65a4109a80 2024-07-29T00:33:00.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.279+0000 7f65a8a00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65880a2c00 0x7f65880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:00.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.279+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f659c0cf030 con 0x7f65a4109a80 2024-07-29T00:33:00.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.280+0000 7f65a2a00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65880a2c00 0x7f65880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:00.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.280+0000 7f65a2a00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65880a2c00 0x7f65880a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6594006010 tx=0x7f6594000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:00.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.284+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f659c09c430 con 0x7f65a4109a80 2024-07-29T00:33:00.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.471+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65a410ba60 con 0x7f65a4109a80 2024-07-29T00:33:00.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.472+0000 7f65a8a00640 1 -- 172.21.15.175:0/3688570785 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f659c09bd30 con 0x7f65a4109a80 2024-07-29T00:33:00.473 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 1 2024-07-29T00:33:00.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65880a2c00 msgr2=0x7f65880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:00.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65880a2c00 0x7f65880a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6594006010 tx=0x7f6594000bc0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 msgr2=0x7f65a419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:00.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a419ec60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f659c00eb10 tx=0x7f659c00c2d0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 shutdown_connections 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a4109a80 0x7f65a419ec60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65880a2c00 0x7f65880a50c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a4104420 0x7f65a419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 --2- 172.21.15.175:0/3688570785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a4103a80 0x7f65a419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 >> 172.21.15.175:0/3688570785 conn(0x7f65a40ff760 msgr2=0x7f65a4100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.475+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 shutdown_connections 2024-07-29T00:33:00.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:00.476+0000 7f65a9e00640 1 -- 172.21.15.175:0/3688570785 wait complete. 2024-07-29T00:33:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:00 smithi175 ceph-mon[32138]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:00.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:00 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3688570785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:33:00.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:00 smithi204 ceph-mon[37368]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:00.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:00 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3688570785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:33:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:00 smithi181 ceph-mon[36706]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:00 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3688570785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:33:01.273 INFO:tasks.cephadm:Adding mds.a on smithi175 2024-07-29T00:33:01.273 INFO:tasks.cephadm:Adding mds.d on smithi175 2024-07-29T00:33:01.273 INFO:tasks.cephadm:Adding mds.g on smithi175 2024-07-29T00:33:01.274 INFO:tasks.cephadm:Adding mds.j on smithi175 2024-07-29T00:33:01.274 INFO:tasks.cephadm:Adding mds.b on smithi181 2024-07-29T00:33:01.274 INFO:tasks.cephadm:Adding mds.e on smithi181 2024-07-29T00:33:01.274 INFO:tasks.cephadm:Adding mds.h on smithi181 2024-07-29T00:33:01.274 INFO:tasks.cephadm:Adding mds.k on smithi181 2024-07-29T00:33:01.275 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-07-29T00:33:01.275 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-07-29T00:33:01.275 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-07-29T00:33:01.275 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-07-29T00:33:01.275 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph orch apply mds all '12;smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-07-29T00:33:01.541 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:33:02.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 -- 172.21.15.204:0/2244522230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2604102380 msgr2=0x7f2604108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:02.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 --2- 172.21.15.204:0/2244522230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2604102380 0x7f2604108c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ca0 tx=0x7f25fc02a8c0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.051 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 -- 172.21.15.204:0/2244522230 shutdown_connections 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 --2- 172.21.15.204:0/2244522230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2604102380 0x7f2604108c10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 --2- 172.21.15.204:0/2244522230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26041019c0 0x7f2604101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 --2- 172.21.15.204:0/2244522230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26041007c0 0x7f2604100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.043+0000 7f260ba20640 1 -- 172.21.15.204:0/2244522230 >> 172.21.15.204:0/2244522230 conn(0x7f26040fbf70 msgr2=0x7f26040fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 -- 172.21.15.204:0/2244522230 shutdown_connections 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 -- 172.21.15.204:0/2244522230 wait complete. 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 Processor -- start 2024-07-29T00:33:02.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 -- start start 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26041007c0 0x7f2604104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 0x7f2604102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2604102380 0x7f26041032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f260410c4f0 con 0x7f26041007c0 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f260410c370 con 0x7f2604102380 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f260ba20640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f260410c670 con 0x7f26041019c0 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f2608fbb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 0x7f2604102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.044+0000 7f2608fbb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 0x7f2604102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56314/0 (socket says 172.21.15.204:56314) 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 -- 172.21.15.204:0/2661661120 learned_addr learned my addr 172.21.15.204:0/2661661120 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2609fbd640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2604102380 0x7f26041032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 -- 172.21.15.204:0/2661661120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2604102380 msgr2=0x7f26041032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:02.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2604102380 0x7f26041032e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 -- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26041007c0 msgr2=0x7f2604104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26041007c0 0x7f2604104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 -- 172.21.15.204:0/2661661120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2604103ba0 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f2608fbb640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 0x7f2604102da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e9f0 tx=0x7f25f400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400cd80 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f4005500 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041a1dd0 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4010690 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.045+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a22b0 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.046+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26041a2540 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.047+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f25f40046c0 con 0x7f26041019c0 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.048+0000 7f25fa7fc640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25f00a2b80 0x7f25f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:02.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.049+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f25f40cf3a0 con 0x7f26041019c0 2024-07-29T00:33:02.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.049+0000 7f26097bc640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25f00a2b80 0x7f25f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:02.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.050+0000 7f26097bc640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25f00a2b80 0x7f25f00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2604101820 tx=0x7f2600006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:02.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.052+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f25f4053df0 con 0x7f26041019c0 2024-07-29T00:33:02.231 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.230+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7f26040fe190 con 0x7f25f00a2b80 2024-07-29T00:33:02.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.245+0000 7f25fa7fc640 1 -- 172.21.15.204:0/2661661120 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f26040fe190 con 0x7f25f00a2b80 2024-07-29T00:33:02.246 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25f00a2b80 msgr2=0x7f25f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25f00a2b80 0x7f25f00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2604101820 tx=0x7f2600006c90 comp rx=0 tx=0).stop 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 msgr2=0x7f2604102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 0x7f2604102da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e9f0 tx=0x7f25f400edf0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 shutdown_connections 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2604102380 0x7f26041032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25f00a2b80 0x7f25f00a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26041019c0 0x7f2604102da0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 --2- 172.21.15.204:0/2661661120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26041007c0 0x7f2604104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 >> 172.21.15.204:0/2661661120 conn(0x7f26040fbf70 msgr2=0x7f26040fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 shutdown_connections 2024-07-29T00:33:02.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-07-29T00:33:02.248+0000 7f260ba20640 1 -- 172.21.15.204:0/2661661120 wait complete. 2024-07-29T00:33:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:02 smithi175 ceph-mon[32138]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:02 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:02.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:02 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:02 smithi204 ceph-mon[37368]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:02 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:02.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:02 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:02.900 DEBUG:teuthology.orchestra.run.smithi175:mds.a> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.a.service 2024-07-29T00:33:02.908 DEBUG:teuthology.orchestra.run.smithi175:mds.d> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.d.service 2024-07-29T00:33:02.953 DEBUG:teuthology.orchestra.run.smithi175:mds.g> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.g.service 2024-07-29T00:33:02.999 DEBUG:teuthology.orchestra.run.smithi175:mds.j> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.j.service 2024-07-29T00:33:03.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:02 smithi181 ceph-mon[36706]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:03.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:02 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:03.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:02 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:03.044 DEBUG:teuthology.orchestra.run.smithi181:mds.b> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.b.service 2024-07-29T00:33:03.047 DEBUG:teuthology.orchestra.run.smithi181:mds.e> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.e.service 2024-07-29T00:33:03.090 DEBUG:teuthology.orchestra.run.smithi181:mds.h> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.h.service 2024-07-29T00:33:03.133 DEBUG:teuthology.orchestra.run.smithi181:mds.k> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.k.service 2024-07-29T00:33:03.177 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.c.service 2024-07-29T00:33:03.179 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.f.service 2024-07-29T00:33:03.222 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.i.service 2024-07-29T00:33:03.265 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.l.service 2024-07-29T00:33:03.308 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-07-29T00:33:03.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:03.756 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:33:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: Saving service mds.all spec with placement smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-07-29T00:33:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:03.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:03.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:03.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:03.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:03 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:03.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: Saving service mds.all spec with placement smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:03 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:33:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: Saving service mds.all spec with placement smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-07-29T00:33:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:04.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:04.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:04.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:04.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:03 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.394+0000 7eff88000640 1 -- 172.21.15.175:0/2461508302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80102200 msgr2=0x7eff8010e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 --2- 172.21.15.175:0/2461508302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80102200 0x7eff8010e730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff78009c10 tx=0x7eff7802a6e0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 -- 172.21.15.175:0/2461508302 shutdown_connections 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 --2- 172.21.15.175:0/2461508302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80102200 0x7eff8010e730 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 --2- 172.21.15.175:0/2461508302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80101860 0x7eff80101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 --2- 172.21.15.175:0/2461508302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80107860 0x7eff80107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 -- 172.21.15.175:0/2461508302 >> 172.21.15.175:0/2461508302 conn(0x7eff800fd5a0 msgr2=0x7eff800ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:04.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 -- 172.21.15.175:0/2461508302 shutdown_connections 2024-07-29T00:33:04.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.395+0000 7eff88000640 1 -- 172.21.15.175:0/2461508302 wait complete. 2024-07-29T00:33:04.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.396+0000 7eff88000640 1 Processor -- start 2024-07-29T00:33:04.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.396+0000 7eff88000640 1 -- start start 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.396+0000 7eff88000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80101860 0x7eff8019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.396+0000 7eff88000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80102200 0x7eff8019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.396+0000 7eff88000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 0x7eff8019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff88000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff800745c0 con 0x7eff80101860 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff88000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff80074440 con 0x7eff80107860 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff88000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff80074740 con 0x7eff80102200 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff84c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80102200 0x7eff8019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:04.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 0x7eff8019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff85600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80101860 0x7eff8019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff84c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80102200 0x7eff8019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46996/0 (socket says 172.21.15.175:46996) 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 0x7eff8019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:52140/0 (socket says 172.21.15.175:52140) 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff84c00640 1 -- 172.21.15.175:0/610699939 learned_addr learned my addr 172.21.15.175:0/610699939 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 -- 172.21.15.175:0/610699939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80102200 msgr2=0x7eff8019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80102200 0x7eff8019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 -- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80101860 msgr2=0x7eff8019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80101860 0x7eff8019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.397+0000 7eff86000640 1 -- 172.21.15.175:0/610699939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff801a04f0 con 0x7eff80107860 2024-07-29T00:33:04.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.398+0000 7eff86000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 0x7eff8019fc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7eff78034040 tx=0x7eff780305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:04.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.398+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78041020 con 0x7eff80107860 2024-07-29T00:33:04.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.398+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff801a0720 con 0x7eff80107860 2024-07-29T00:33:04.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.398+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff78003e50 con 0x7eff80107860 2024-07-29T00:33:04.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.398+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78036040 con 0x7eff80107860 2024-07-29T00:33:04.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.398+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff801a6e20 con 0x7eff80107860 2024-07-29T00:33:04.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.400+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff80101cc0 con 0x7eff80107860 2024-07-29T00:33:04.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.402+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7eff78044020 con 0x7eff80107860 2024-07-29T00:33:04.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.403+0000 7eff7e000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff680a2b80 0x7eff680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:04.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.403+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7eff780ef870 con 0x7eff80107860 2024-07-29T00:33:04.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.403+0000 7eff85600640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff680a2b80 0x7eff680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:04.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.404+0000 7eff85600640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff680a2b80 0x7eff680a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff740096c0 tx=0x7eff74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:04.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.407+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7eff7804b020 con 0x7eff80107860 2024-07-29T00:33:04.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.637+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff801a7050 con 0x7eff80107860 2024-07-29T00:33:04.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.638+0000 7eff7e000640 1 -- 172.21.15.175:0/610699939 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7eff780bcbd0 con 0x7eff80107860 2024-07-29T00:33:04.639 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:04.639 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"btime":"2024-07-29T00:25:53:731837+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-07-29T00:33:04.639 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 1 2024-07-29T00:33:04.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff680a2b80 msgr2=0x7eff680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff680a2b80 0x7eff680a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff740096c0 tx=0x7eff74009260 comp rx=0 tx=0).stop 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 msgr2=0x7eff8019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 0x7eff8019fc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7eff78034040 tx=0x7eff780305d0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 shutdown_connections 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff80107860 0x7eff8019fc80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff680a2b80 0x7eff680a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff80102200 0x7eff8019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 --2- 172.21.15.175:0/610699939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff80101860 0x7eff8019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 >> 172.21.15.175:0/610699939 conn(0x7eff800fd5a0 msgr2=0x7eff800feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 shutdown_connections 2024-07-29T00:33:04.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:04.641+0000 7eff88000640 1 -- 172.21.15.175:0/610699939 wait complete. 2024-07-29T00:33:04.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:04 smithi204 ceph-mon[37368]: Deploying daemon mds.l on smithi204 2024-07-29T00:33:04.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:04 smithi204 ceph-mon[37368]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:04.836 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:04 smithi204 systemd[1]: Starting Ceph mds.l for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:04.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:04 smithi181 ceph-mon[36706]: Deploying daemon mds.l on smithi204 2024-07-29T00:33:04.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:04 smithi181 ceph-mon[36706]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:05.022 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:04 smithi175 ceph-mon[32138]: Deploying daemon mds.l on smithi204 2024-07-29T00:33:05.023 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:04 smithi175 ceph-mon[32138]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:05.179 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:04 smithi204 podman[66676]: 2024-07-29 00:33:04.82536804 +0000 UTC m=+0.017387258 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:05.179 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:04 smithi204 podman[66676]: 2024-07-29 00:33:04.951794935 +0000 UTC m=+0.143814145 container create cce0a62972a38f5fd3eb147fcb049e5b2344edffef21f9649ca6a3b5bdd59f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-29T00:33:05.179 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:05 smithi204 podman[66676]: 2024-07-29 00:33:05.179710874 +0000 UTC m=+0.371730096 container init cce0a62972a38f5fd3eb147fcb049e5b2344edffef21f9649ca6a3b5bdd59f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-29T00:33:05.541 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:05 smithi204 podman[66676]: 2024-07-29 00:33:05.182333953 +0000 UTC m=+0.374353168 container start cce0a62972a38f5fd3eb147fcb049e5b2344edffef21f9649ca6a3b5bdd59f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-07-29T00:33:05.542 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mds[66695]: -- 172.21.15.204:0/1037552388 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55805c241860 con 0x55805c21e800 2024-07-29T00:33:05.542 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l[66691]: starting mds.l at 2024-07-29T00:33:05.542 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:05 smithi204 bash[66676]: cce0a62972a38f5fd3eb147fcb049e5b2344edffef21f9649ca6a3b5bdd59f56 2024-07-29T00:33:05.542 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:33:05 smithi204 systemd[1]: Started Ceph mds.l for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/610699939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6832/1301583270,v1:172.21.15.204:6833/1301583270] up:boot 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: fsmap 1 up:standby 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:05.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:05 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/610699939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6832/1301583270,v1:172.21.15.204:6833/1301583270] up:boot 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: fsmap 1 up:standby 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:05.803 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:05 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/610699939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6832/1301583270,v1:172.21.15.204:6833/1301583270] up:boot 2024-07-29T00:33:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: fsmap 1 up:standby 2024-07-29T00:33:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:33:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:05.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:05.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:05.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:05 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:06.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:06 smithi181 ceph-mon[36706]: Deploying daemon mds.k on smithi181 2024-07-29T00:33:06.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:06 smithi181 ceph-mon[36706]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:06.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:06 smithi204 ceph-mon[37368]: Deploying daemon mds.k on smithi181 2024-07-29T00:33:06.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:06 smithi204 ceph-mon[37368]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:06 smithi175 ceph-mon[32138]: Deploying daemon mds.k on smithi181 2024-07-29T00:33:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:06 smithi175 ceph-mon[32138]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:07.499 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 systemd[1]: Starting Ceph mds.k for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:07.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:07.849 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 podman[65495]: 2024-07-29 00:33:07.486708624 +0000 UTC m=+0.016728431 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:07.849 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 podman[65495]: 2024-07-29 00:33:07.589160281 +0000 UTC m=+0.119180118 container create c9c81f1f128478150efee1c521bb8fa9fe9c5cb3fbf0203e966d5ec77fbe3e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, GIT_CLEAN=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-29T00:33:07.849 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 podman[65495]: 2024-07-29 00:33:07.850087945 +0000 UTC m=+0.380107739 container init c9c81f1f128478150efee1c521bb8fa9fe9c5cb3fbf0203e966d5ec77fbe3e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe) 2024-07-29T00:33:08.058 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:08.249 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 podman[65495]: 2024-07-29 00:33:07.852983614 +0000 UTC m=+0.383003424 container start c9c81f1f128478150efee1c521bb8fa9fe9c5cb3fbf0203e966d5ec77fbe3e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS) 2024-07-29T00:33:08.250 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 ceph-mds[65514]: -- 172.21.15.181:0/1761175074 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0af4cd860 con 0x55b0af4ab400 2024-07-29T00:33:08.250 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k[65510]: starting mds.k at 2024-07-29T00:33:08.250 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 bash[65495]: c9c81f1f128478150efee1c521bb8fa9fe9c5cb3fbf0203e966d5ec77fbe3e57 2024-07-29T00:33:08.250 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:33:07 smithi181 systemd[1]: Started Ceph mds.k for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:08.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 -- 172.21.15.175:0/3443947444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a010bcb0 msgr2=0x7f67a010e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 --2- 172.21.15.175:0/3443947444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a010bcb0 0x7f67a010e150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6798009ca0 tx=0x7f679802a950 comp rx=0 tx=0).stop 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 -- 172.21.15.175:0/3443947444 shutdown_connections 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 --2- 172.21.15.175:0/3443947444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a010bcb0 0x7f67a010e150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 --2- 172.21.15.175:0/3443947444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 0x7f67a010b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 --2- 172.21.15.175:0/3443947444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67a0102880 0x7f67a0102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 -- 172.21.15.175:0/3443947444 >> 172.21.15.175:0/3443947444 conn(0x7f67a00fc300 msgr2=0x7f67a00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 -- 172.21.15.175:0/3443947444 shutdown_connections 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.695+0000 7f67a5000640 1 -- 172.21.15.175:0/3443947444 wait complete. 2024-07-29T00:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.696+0000 7f67a5000640 1 Processor -- start 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.696+0000 7f67a5000640 1 -- start start 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f67a5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 0x7f67a019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f67a5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 0x7f67a019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f67a5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67a010bcb0 0x7f67a019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f67a5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0111a70 con 0x7f67a0102880 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f67a5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01118f0 con 0x7f67a01031a0 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f67a5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0111bf0 con 0x7f67a010bcb0 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 0x7f67a019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 0x7f67a019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37262/0 (socket says 172.21.15.175:37262) 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679f400640 1 -- 172.21.15.175:0/2054489020 learned_addr learned my addr 172.21.15.175:0/2054489020 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679ea00640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 0x7f67a019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679fe00640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67a010bcb0 0x7f67a019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679ea00640 1 -- 172.21.15.175:0/2054489020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67a010bcb0 msgr2=0x7f67a019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.697+0000 7f679ea00640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67a010bcb0 0x7f67a019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f679ea00640 1 -- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 msgr2=0x7f67a019a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f679ea00640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 0x7f67a019a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f679ea00640 1 -- 172.21.15.175:0/2054489020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a019f0f0 con 0x7f67a01031a0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f679f400640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 0x7f67a019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f679ea00640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 0x7f67a019a680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f678c00e9f0 tx=0x7f678c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c00ccc0 con 0x7f67a01031a0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f678c0035f0 con 0x7f67a01031a0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a019f3e0 con 0x7f67a01031a0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c00ce60 con 0x7f67a01031a0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.698+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01a6cc0 con 0x7f67a01031a0 2024-07-29T00:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.699+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a0102c60 con 0x7f67a01031a0 2024-07-29T00:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.701+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f678c003b30 con 0x7f67a01031a0 2024-07-29T00:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.702+0000 7f6797e00640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67840a2c00 0x7f67840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.702+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f678c0cede0 con 0x7f67a01031a0 2024-07-29T00:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.703+0000 7f679f400640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67840a2c00 0x7f67840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.703+0000 7f679f400640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67840a2c00 0x7f67840a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6790011550 tx=0x7f6790005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.706+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f678c09c140 con 0x7f67a01031a0 2024-07-29T00:33:08.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:08.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:08.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:08.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:08 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:08 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:08.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.961+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67a01a7050 con 0x7f67a01031a0 2024-07-29T00:33:08.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.962+0000 7f6797e00640 1 -- 172.21.15.175:0/2054489020 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f678c09ba40 con 0x7f67a01031a0 2024-07-29T00:33:08.962 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:08.962 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"btime":"2024-07-29T00:33:08:546150+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-07-29T00:33:08.963 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 3 2024-07-29T00:33:08.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.964+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67840a2c00 msgr2=0x7f67840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:08.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.964+0000 7f67a5000640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67840a2c00 0x7f67840a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6790011550 tx=0x7f6790005f90 comp rx=0 tx=0).stop 2024-07-29T00:33:08.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 msgr2=0x7f67a019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:08.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 0x7f67a019a680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f678c00e9f0 tx=0x7f678c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 shutdown_connections 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f67840a2c00 0x7f67840a50c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67a010bcb0 0x7f67a019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a01031a0 0x7f67a019a680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 --2- 172.21.15.175:0/2054489020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a0102880 0x7f67a019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 >> 172.21.15.175:0/2054489020 conn(0x7f67a00fc300 msgr2=0x7f67a010cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 shutdown_connections 2024-07-29T00:33:08.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:08.965+0000 7f67a5000640 1 -- 172.21.15.175:0/2054489020 wait complete. 2024-07-29T00:33:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:09.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:09.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:08 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:09.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:09 smithi175 ceph-mon[32138]: Deploying daemon mds.j on smithi175 2024-07-29T00:33:09.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6832/4123110499,v1:172.21.15.181:6833/4123110499] up:boot 2024-07-29T00:33:09.680 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:09 smithi175 ceph-mon[32138]: fsmap 2 up:standby 2024-07-29T00:33:09.680 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:09 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:33:09.680 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:09 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2054489020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:09 smithi204 ceph-mon[37368]: Deploying daemon mds.j on smithi175 2024-07-29T00:33:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6832/4123110499,v1:172.21.15.181:6833/4123110499] up:boot 2024-07-29T00:33:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:09 smithi204 ceph-mon[37368]: fsmap 2 up:standby 2024-07-29T00:33:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:09 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:33:09.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:09 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2054489020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:09.994 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:09 smithi181 ceph-mon[36706]: Deploying daemon mds.j on smithi175 2024-07-29T00:33:09.994 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6832/4123110499,v1:172.21.15.181:6833/4123110499] up:boot 2024-07-29T00:33:09.994 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:09 smithi181 ceph-mon[36706]: fsmap 2 up:standby 2024-07-29T00:33:09.994 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:09 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:33:09.994 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:09 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2054489020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:10.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:10 smithi175 ceph-mon[32138]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:10.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:10 smithi204 ceph-mon[37368]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:10 smithi181 ceph-mon[36706]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:12.012 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:11 smithi175 systemd[1]: Starting Ceph mds.j for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:12.284 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 podman[71297]: 2024-07-29 00:33:11.915846298 +0000 UTC m=+0.036679165 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:12.284 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 podman[71297]: 2024-07-29 00:33:12.048096029 +0000 UTC m=+0.168928935 container create ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:33:12.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:12 smithi175 ceph-mon[32138]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 podman[71297]: 2024-07-29 00:33:12.368718687 +0000 UTC m=+0.489551596 container init ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 podman[71297]: 2024-07-29 00:33:12.378692577 +0000 UTC m=+0.499525483 container start ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 ceph-mds[71316]: -- 172.21.15.175:0/759242210 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56055c951860 con 0x56055c92f000 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j[71312]: starting mds.j at 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 bash[71297]: ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 2024-07-29T00:33:12.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:33:12 smithi175 systemd[1]: Started Ceph mds.j for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:12.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:12 smithi204 ceph-mon[37368]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:12.905 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:12.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:12 smithi181 ceph-mon[36706]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:13.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 -- 172.21.15.175:0/2836423773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 msgr2=0x7f865810e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:13.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 --2- 172.21.15.175:0/2836423773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 0x7f865810e300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8650009ca0 tx=0x7f865002a950 comp rx=0 tx=0).stop 2024-07-29T00:33:13.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 -- 172.21.15.175:0/2836423773 shutdown_connections 2024-07-29T00:33:13.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 --2- 172.21.15.175:0/2836423773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 0x7f865810e300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 --2- 172.21.15.175:0/2836423773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8658103350 0x7f865810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 --2- 172.21.15.175:0/2836423773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658102a30 0x7f8658102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 -- 172.21.15.175:0/2836423773 >> 172.21.15.175:0/2836423773 conn(0x7f86580fc490 msgr2=0x7f86580fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:13.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 -- 172.21.15.175:0/2836423773 shutdown_connections 2024-07-29T00:33:13.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.696+0000 7f865da00640 1 -- 172.21.15.175:0/2836423773 wait complete. 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.698+0000 7f865da00640 1 Processor -- start 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.698+0000 7f865da00640 1 -- start start 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f865da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8658102a30 0x7f86581980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f865da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 0x7f8658198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f865da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 0x7f865819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f865da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111c60 con 0x7f865810be60 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f865da00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111ae0 con 0x7f8658102a30 2024-07-29T00:33:13.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f865da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111de0 con 0x7f8658103350 2024-07-29T00:33:13.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 0x7f8658198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:13.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8658102a30 0x7f86581980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:13.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8657400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 0x7f865819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:13.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 0x7f8658198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:36868/0 (socket says 172.21.15.175:36868) 2024-07-29T00:33:13.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 -- 172.21.15.175:0/2793537526 learned_addr learned my addr 172.21.15.175:0/2793537526 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 -- 172.21.15.175:0/2793537526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8658102a30 msgr2=0x7f86581980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8658102a30 0x7f86581980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 -- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 msgr2=0x7f865819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 0x7f865819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.699+0000 7f8656000640 1 -- 172.21.15.175:0/2793537526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865819d070 con 0x7f8658103350 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.700+0000 7f8656000640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 0x7f8658198600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9f0 tx=0x7f864000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.700+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000cd80 con 0x7f8658103350 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.700+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865819d360 con 0x7f8658103350 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.700+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86400035f0 con 0x7f8658103350 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.700+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640010670 con 0x7f8658103350 2024-07-29T00:33:13.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.700+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86581a4c40 con 0x7f8658103350 2024-07-29T00:33:13.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.702+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8658102e10 con 0x7f8658103350 2024-07-29T00:33:13.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.703+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8640004020 con 0x7f8658103350 2024-07-29T00:33:13.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.707+0000 7f864f400640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f863c0a2c00 0x7f863c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:13.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.707+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f86400cedc0 con 0x7f8658103350 2024-07-29T00:33:13.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.707+0000 7f8656a00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f863c0a2c00 0x7f863c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:13.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.708+0000 7f8656a00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f863c0a2c00 0x7f863c0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8648009bd0 tx=0x7f8648005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:13.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.708+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f864009c120 con 0x7f8658103350 2024-07-29T00:33:13.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6834/284852041,v1:172.21.15.175:6835/284852041] up:boot 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: fsmap 3 up:standby 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:13.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:13 smithi204 ceph-mon[37368]: Deploying daemon mds.d on smithi175 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6834/284852041,v1:172.21.15.175:6835/284852041] up:boot 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: fsmap 3 up:standby 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:13.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:13 smithi175 ceph-mon[32138]: Deploying daemon mds.d on smithi175 2024-07-29T00:33:13.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.946+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f865802d5a0 con 0x7f8658103350 2024-07-29T00:33:13.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.947+0000 7f864f400640 1 -- 172.21.15.175:0/2793537526 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f864009ba20 con 0x7f8658103350 2024-07-29T00:33:13.948 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:13.948 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":4,"btime":"2024-07-29T00:33:12:571498+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-07-29T00:33:13.948 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 4 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f863c0a2c00 msgr2=0x7f863c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f863c0a2c00 0x7f863c0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8648009bd0 tx=0x7f8648005f90 comp rx=0 tx=0).stop 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 msgr2=0x7f8658198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 0x7f8658198600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9f0 tx=0x7f864000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 shutdown_connections 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f865810be60 0x7f865819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f863c0a2c00 0x7f863c0a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8658103350 0x7f8658198600 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 --2- 172.21.15.175:0/2793537526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8658102a30 0x7f86581980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.950+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 >> 172.21.15.175:0/2793537526 conn(0x7f86580fc490 msgr2=0x7f86580fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.951+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 shutdown_connections 2024-07-29T00:33:13.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:13.951+0000 7f865da00640 1 -- 172.21.15.175:0/2793537526 wait complete. 2024-07-29T00:33:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6834/284852041,v1:172.21.15.175:6835/284852041] up:boot 2024-07-29T00:33:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: fsmap 3 up:standby 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:13 smithi181 ceph-mon[36706]: Deploying daemon mds.d on smithi175 2024-07-29T00:33:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:14 smithi181 ceph-mon[36706]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:14 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2793537526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:15.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:14 smithi175 ceph-mon[32138]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:15.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:14 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2793537526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:15.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:14 smithi204 ceph-mon[37368]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:15.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:14 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2793537526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:16.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:16 smithi204 ceph-mon[37368]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:16.964 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:16 smithi175 ceph-mon[32138]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:17.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:16 smithi181 ceph-mon[36706]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:17.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:17.677 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 podman[71733]: 2024-07-29 00:33:17.380001511 +0000 UTC m=+0.188587430 container create 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:33:17.677 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 podman[71733]: 2024-07-29 00:33:17.676300902 +0000 UTC m=+0.484886827 container init 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724) 2024-07-29T00:33:17.933 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:18.034 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 podman[71733]: 2024-07-29 00:33:17.684952574 +0000 UTC m=+0.493538464 container start 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-29T00:33:18.034 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 ceph-mds[71778]: -- 172.21.15.175:0/2633913437 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9dfde3c20 con 0x55e9dfdc1000 2024-07-29T00:33:18.034 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d[71761]: starting mds.d at 2024-07-29T00:33:18.034 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 bash[71733]: 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 2024-07-29T00:33:18.035 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:33:17 smithi175 systemd[1]: Started Ceph mds.d for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 -- 172.21.15.175:0/3805041699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 msgr2=0x7f01d0109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 --2- 172.21.15.175:0/3805041699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 0x7f01d0109e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009930 tx=0x7f01c002a440 comp rx=0 tx=0).stop 2024-07-29T00:33:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 -- 172.21.15.175:0/3805041699 shutdown_connections 2024-07-29T00:33:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 --2- 172.21.15.175:0/3805041699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 0x7f01d0110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 --2- 172.21.15.175:0/3805041699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d0103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 --2- 172.21.15.175:0/3805041699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 0x7f01d0109e60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 -- 172.21.15.175:0/3805041699 >> 172.21.15.175:0/3805041699 conn(0x7f01d00ff760 msgr2=0x7f01d0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 -- 172.21.15.175:0/3805041699 shutdown_connections 2024-07-29T00:33:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.678+0000 7f01d7800640 1 -- 172.21.15.175:0/3805041699 wait complete. 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 Processor -- start 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 -- start start 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 0x7f01d019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 0x7f01d019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0113ed0 con 0x7f01d0109a80 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0113d50 con 0x7f01d0103a80 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.680+0000 7f01d7800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0114050 con 0x7f01d0104420 2024-07-29T00:33:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01d4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01d4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58486/0 (socket says 172.21.15.175:58486) 2024-07-29T00:33:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01d4e00640 1 -- 172.21.15.175:0/4114600913 learned_addr learned my addr 172.21.15.175:0/4114600913 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01c7e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 0x7f01d019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01d5800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 0x7f01d019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01c7e00640 1 -- 172.21.15.175:0/4114600913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 msgr2=0x7f01d019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01c7e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01c7e00640 1 -- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 msgr2=0x7f01d019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01c7e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 0x7f01d019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01c7e00640 1 -- 172.21.15.175:0/4114600913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d019f300 con 0x7f01d0104420 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.681+0000 7f01d4e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.682+0000 7f01c7e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 0x7f01d019a890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01b800e9f0 tx=0x7f01b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.682+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b800ccc0 con 0x7f01d0104420 2024-07-29T00:33:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.682+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8010910 con 0x7f01d0104420 2024-07-29T00:33:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.682+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8018960 con 0x7f01d0104420 2024-07-29T00:33:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.682+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d019f5f0 con 0x7f01d0104420 2024-07-29T00:33:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.682+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d01a6ed0 con 0x7f01d0104420 2024-07-29T00:33:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.683+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d0103ee0 con 0x7f01d0104420 2024-07-29T00:33:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.685+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f01b8010ac0 con 0x7f01d0104420 2024-07-29T00:33:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.686+0000 7f01ce000640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f01b40a2c00 0x7f01b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.687+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f01b80cee90 con 0x7f01d0104420 2024-07-29T00:33:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.687+0000 7f01d4e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f01b40a2c00 0x7f01b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.687+0000 7f01d4e00640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f01b40a2c00 0x7f01b40a50c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01c0000c00 tx=0x7f01c0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.691+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f01b809bc40 con 0x7f01d0104420 2024-07-29T00:33:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.917+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01d010bff0 con 0x7f01d0104420 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:18.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:18.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.918+0000 7f01ce000640 1 -- 172.21.15.175:0/4114600913 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f01b809ff40 con 0x7f01d0104420 2024-07-29T00:33:18.919 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:18.919 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":5,"btime":"2024-07-29T00:33:18:628216+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-07-29T00:33:18.920 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 5 2024-07-29T00:33:18.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f01b40a2c00 msgr2=0x7f01b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f01b40a2c00 0x7f01b40a50c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01c0000c00 tx=0x7f01c0031510 comp rx=0 tx=0).stop 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 msgr2=0x7f01d019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 0x7f01d019a890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01b800e9f0 tx=0x7f01b800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 shutdown_connections 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d0109a80 0x7f01d019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f01b40a2c00 0x7f01b40a50c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01d0104420 0x7f01d019a890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 --2- 172.21.15.175:0/4114600913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d0103a80 0x7f01d019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 >> 172.21.15.175:0/4114600913 conn(0x7f01d00ff760 msgr2=0x7f01d0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 shutdown_connections 2024-07-29T00:33:18.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:18.921+0000 7f01d7800640 1 -- 172.21.15.175:0/4114600913 wait complete. 2024-07-29T00:33:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:19.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:19.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:19.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:19.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:19.815 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:19 smithi181 ceph-mon[36706]: Deploying daemon mds.e on smithi181 2024-07-29T00:33:19.815 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:19 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6836/1855710459,v1:172.21.15.175:6837/1855710459] up:boot 2024-07-29T00:33:19.815 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:19 smithi181 ceph-mon[36706]: fsmap 4 up:standby 2024-07-29T00:33:19.815 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-29T00:33:19.815 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:19 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/4114600913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:19.912 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:19 smithi175 ceph-mon[32138]: Deploying daemon mds.e on smithi181 2024-07-29T00:33:19.912 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:19 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6836/1855710459,v1:172.21.15.175:6837/1855710459] up:boot 2024-07-29T00:33:19.912 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:19 smithi175 ceph-mon[32138]: fsmap 4 up:standby 2024-07-29T00:33:19.912 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-29T00:33:19.912 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:19 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/4114600913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:19 smithi204 ceph-mon[37368]: Deploying daemon mds.e on smithi181 2024-07-29T00:33:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:19 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6836/1855710459,v1:172.21.15.175:6837/1855710459] up:boot 2024-07-29T00:33:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:19 smithi204 ceph-mon[37368]: fsmap 4 up:standby 2024-07-29T00:33:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-29T00:33:20.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:19 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/4114600913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:20.201 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:19 smithi181 systemd[1]: Starting Ceph mds.e for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:20.474 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 podman[65809]: 2024-07-29 00:33:20.103784471 +0000 UTC m=+0.017129330 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:20.474 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 podman[65809]: 2024-07-29 00:33:20.203382512 +0000 UTC m=+0.116727380 container create 14b08d6e5b5234f6c388014f2d8e501066cecfcae16629ca199bd58b028b368c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-29T00:33:20.474 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 podman[65809]: 2024-07-29 00:33:20.447998257 +0000 UTC m=+0.361343117 container init 14b08d6e5b5234f6c388014f2d8e501066cecfcae16629ca199bd58b028b368c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:33:20.474 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 podman[65809]: 2024-07-29 00:33:20.450474029 +0000 UTC m=+0.363818897 container start 14b08d6e5b5234f6c388014f2d8e501066cecfcae16629ca199bd58b028b368c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-29T00:33:20.474 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mds[65828]: -- 172.21.15.181:0/2800829231 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56086e98b860 con 0x56086e969400 2024-07-29T00:33:20.749 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e[65824]: starting mds.e at 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 bash[65809]: 14b08d6e5b5234f6c388014f2d8e501066cecfcae16629ca199bd58b028b368c 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:33:20 smithi181 systemd[1]: Started Ceph mds.e for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:20.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:20 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:21.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:20 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:20 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:21 smithi204 ceph-mon[37368]: Deploying daemon mds.f on smithi204 2024-07-29T00:33:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:21 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6834/568693704,v1:172.21.15.181:6835/568693704] up:boot 2024-07-29T00:33:21.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:21 smithi204 ceph-mon[37368]: fsmap 5 up:standby 2024-07-29T00:33:21.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:33:21.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:21.998 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:21 smithi175 ceph-mon[32138]: Deploying daemon mds.f on smithi204 2024-07-29T00:33:21.998 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:21 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6834/568693704,v1:172.21.15.181:6835/568693704] up:boot 2024-07-29T00:33:21.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:21 smithi175 ceph-mon[32138]: fsmap 5 up:standby 2024-07-29T00:33:21.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:33:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:21 smithi181 ceph-mon[36706]: Deploying daemon mds.f on smithi204 2024-07-29T00:33:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:21 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6834/568693704,v1:172.21.15.181:6835/568693704] up:boot 2024-07-29T00:33:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:21 smithi181 ceph-mon[36706]: fsmap 5 up:standby 2024-07-29T00:33:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:33:22.384 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:22.804 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:22 smithi175 ceph-mon[32138]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:22.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:22 smithi204 ceph-mon[37368]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:22.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:22 smithi181 ceph-mon[36706]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:23.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 -- 172.21.15.175:0/2637426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1076b0 msgr2=0x7f7ddc107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:23.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 --2- 172.21.15.175:0/2637426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc107a90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009930 tx=0x7f7dd002a420 comp rx=0 tx=0).stop 2024-07-29T00:33:23.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 -- 172.21.15.175:0/2637426507 shutdown_connections 2024-07-29T00:33:23.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 --2- 172.21.15.175:0/2637426507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 0x7f7ddc10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 --2- 172.21.15.175:0/2637426507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1016b0 0x7f7ddc101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 --2- 172.21.15.175:0/2637426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc107a90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 -- 172.21.15.175:0/2637426507 >> 172.21.15.175:0/2637426507 conn(0x7f7ddc0fd430 msgr2=0x7f7ddc0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 -- 172.21.15.175:0/2637426507 shutdown_connections 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.063+0000 7f7de2c00640 1 -- 172.21.15.175:0/2637426507 wait complete. 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.065+0000 7f7de2c00640 1 Processor -- start 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.065+0000 7f7de2c00640 1 -- start start 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.065+0000 7f7de2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1016b0 0x7f7ddc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.065+0000 7f7de2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 0x7f7ddc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de2c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc111b00 con 0x7f7ddc1016b0 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de2c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc111980 con 0x7f7ddc1076b0 2024-07-29T00:33:23.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc111c80 con 0x7f7ddc102050 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1016b0 0x7f7ddc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1016b0 0x7f7ddc19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35232/0 (socket says 172.21.15.175:35232) 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de1800640 1 -- 172.21.15.175:0/19287730 learned_addr learned my addr 172.21.15.175:0/19287730 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de0e00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 0x7f7ddc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de2200640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de0e00640 1 -- 172.21.15.175:0/19287730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1076b0 msgr2=0x7f7ddc19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de0e00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de0e00640 1 -- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1016b0 msgr2=0x7f7ddc19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de0e00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1016b0 0x7f7ddc19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de0e00640 1 -- 172.21.15.175:0/19287730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc19f190 con 0x7f7ddc102050 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.066+0000 7f7de2200640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.067+0000 7f7de0e00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 0x7f7ddc19a720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800e9f0 tx=0x7f7dc800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.067+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800ccc0 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.067+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc8010910 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.067+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8018960 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.067+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc19f480 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.067+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc1a6d60 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.068+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc101b10 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.070+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7dc8010ac0 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.072+0000 7f7dda000640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7dc40a2c00 0x7f7dc40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.072+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f7dc80cefe0 con 0x7f7ddc102050 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.072+0000 7f7de1800640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7dc40a2c00 0x7f7dc40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.073+0000 7f7de1800640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7dc40a2c00 0x7f7dc40a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00096c0 tx=0x7f7dd0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:23.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.076+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7dc809c340 con 0x7f7ddc102050 2024-07-29T00:33:23.151 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:33:23 smithi204 podman[66989]: 2024-07-29 00:33:23.129309406 +0000 UTC m=+0.356878415 container start f0a1c0c1381d544fc0c1e04e6f8dc19dbd0c1a3dfd437721558e174556f3f95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-07-29T00:33:23.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.309+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ddc1a6fc0 con 0x7f7ddc102050 2024-07-29T00:33:23.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.310+0000 7f7dda000640 1 -- 172.21.15.175:0/19287730 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7f7dc809bc40 con 0x7f7ddc102050 2024-07-29T00:33:23.310 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:23.310 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":6,"btime":"2024-07-29T00:33:20:652324+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-07-29T00:33:23.311 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 6 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.312+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7dc40a2c00 msgr2=0x7f7dc40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.312+0000 7f7de2c00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7dc40a2c00 0x7f7dc40a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00096c0 tx=0x7f7dd0005b50 comp rx=0 tx=0).stop 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.312+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 msgr2=0x7f7ddc19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.312+0000 7f7de2c00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 0x7f7ddc19a720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800e9f0 tx=0x7f7dc800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 shutdown_connections 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc1076b0 0x7f7ddc19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7dc40a2c00 0x7f7dc40a50c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc102050 0x7f7ddc19a720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 --2- 172.21.15.175:0/19287730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc1016b0 0x7f7ddc19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:23.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 >> 172.21.15.175:0/19287730 conn(0x7f7ddc0fd430 msgr2=0x7f7ddc0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:23.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 shutdown_connections 2024-07-29T00:33:23.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:23.313+0000 7f7de2c00640 1 -- 172.21.15.175:0/19287730 wait complete. 2024-07-29T00:33:23.585 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:33:23 smithi204 ceph-mds[67007]: -- 172.21.15.204:0/222206182 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56438f1ed860 con 0x56438f1cb400 2024-07-29T00:33:23.585 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:33:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f[67003]: starting mds.f at 2024-07-29T00:33:23.585 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:33:23 smithi204 bash[66989]: f0a1c0c1381d544fc0c1e04e6f8dc19dbd0c1a3dfd437721558e174556f3f95e 2024-07-29T00:33:23.586 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:33:23 smithi204 systemd[1]: Started Ceph mds.f for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:24.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: Deploying daemon mds.c on smithi204 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/19287730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6834/1977765890,v1:172.21.15.204:6835/1977765890] up:boot 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: fsmap 6 up:standby 2024-07-29T00:33:24.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:24 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:33:24.379 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.379 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.379 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.379 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:24.379 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:24.379 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:24.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: Deploying daemon mds.c on smithi204 2024-07-29T00:33:24.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/19287730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:24.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:24.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6834/1977765890,v1:172.21.15.204:6835/1977765890] up:boot 2024-07-29T00:33:24.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: fsmap 6 up:standby 2024-07-29T00:33:24.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:24 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:33:24.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: Deploying daemon mds.c on smithi204 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/19287730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6834/1977765890,v1:172.21.15.204:6835/1977765890] up:boot 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: fsmap 6 up:standby 2024-07-29T00:33:24.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:24 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:33:25.335 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 systemd[1]: Starting Ceph mds.c for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:25.616 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 podman[67289]: 2024-07-29 00:33:25.251243751 +0000 UTC m=+0.017156122 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:25.616 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 podman[67289]: 2024-07-29 00:33:25.371385305 +0000 UTC m=+0.137297707 container create 48119f7fae9d83e86e892dd5d433cd5579de7ca7933bfd996e42b6990f8bd9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724) 2024-07-29T00:33:25.616 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 podman[67289]: 2024-07-29 00:33:25.591049988 +0000 UTC m=+0.356962348 container init 48119f7fae9d83e86e892dd5d433cd5579de7ca7933bfd996e42b6990f8bd9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-29T00:33:25.616 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 podman[67289]: 2024-07-29 00:33:25.59381013 +0000 UTC m=+0.359722505 container start 48119f7fae9d83e86e892dd5d433cd5579de7ca7933bfd996e42b6990f8bd9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-29T00:33:25.616 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 ceph-mds[67308]: -- 172.21.15.204:0/1207380497 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558fae91b860 con 0x558fae8f9400 2024-07-29T00:33:26.085 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c[67304]: starting mds.c at 2024-07-29T00:33:26.085 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 bash[67289]: 48119f7fae9d83e86e892dd5d433cd5579de7ca7933bfd996e42b6990f8bd9de 2024-07-29T00:33:26.086 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:33:25 smithi204 systemd[1]: Started Ceph mds.c for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:26.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:26.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:26.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:26.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:26.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:26.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:26.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:26.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:26 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:26.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:26 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:26.913 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:27.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:27.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:27.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:27.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:27.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:27.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:27.038 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:26 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:27.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 -- 172.21.15.175:0/3190020623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 msgr2=0x7f7ebc103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:27.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 --2- 172.21.15.175:0/3190020623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc103ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009900 tx=0x7f7ea802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 -- 172.21.15.175:0/3190020623 shutdown_connections 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 --2- 172.21.15.175:0/3190020623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc104420 0x7f7ebc110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 --2- 172.21.15.175:0/3190020623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc103ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 --2- 172.21.15.175:0/3190020623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ebc109a80 0x7f7ebc109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 -- 172.21.15.175:0/3190020623 >> 172.21.15.175:0/3190020623 conn(0x7f7ebc0ff760 msgr2=0x7f7ebc101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 -- 172.21.15.175:0/3190020623 shutdown_connections 2024-07-29T00:33:27.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.638+0000 7f7ec2800640 1 -- 172.21.15.175:0/3190020623 wait complete. 2024-07-29T00:33:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 Processor -- start 2024-07-29T00:33:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 -- start start 2024-07-29T00:33:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ebc104420 0x7f7ebc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc109a80 0x7f7ebc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc113ed0 con 0x7f7ebc109a80 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc113d50 con 0x7f7ebc103a80 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.639+0000 7f7ec2800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc114050 con 0x7f7ebc104420 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:33456/0 (socket says 172.21.15.175:33456) 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 -- 172.21.15.175:0/824418239 learned_addr learned my addr 172.21.15.175:0/824418239 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebaa00640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ebc104420 0x7f7ebc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebbe00640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc109a80 0x7f7ebc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 -- 172.21.15.175:0/824418239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ebc104420 msgr2=0x7f7ebc19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ebc104420 0x7f7ebc19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 -- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc109a80 msgr2=0x7f7ebc19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc109a80 0x7f7ebc19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 -- 172.21.15.175:0/824418239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc19f320 con 0x7f7ebc103a80 2024-07-29T00:33:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.640+0000 7f7ebb400640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc19a370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9f0 tx=0x7f7eac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.641+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00cd80 con 0x7f7ebc103a80 2024-07-29T00:33:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.641+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac005500 con 0x7f7ebc103a80 2024-07-29T00:33:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.641+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc19f610 con 0x7f7ebc103a80 2024-07-29T00:33:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.641+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac010690 con 0x7f7ebc103a80 2024-07-29T00:33:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.641+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a6ef0 con 0x7f7ebc103a80 2024-07-29T00:33:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.642+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc103ee0 con 0x7f7ebc103a80 2024-07-29T00:33:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.644+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7eac0046c0 con 0x7f7ebc103a80 2024-07-29T00:33:27.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.645+0000 7f7eb3e00640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7ea00a2c00 0x7f7ea00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:27.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.645+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f7eac0cefe0 con 0x7f7ebc103a80 2024-07-29T00:33:27.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.645+0000 7f7ebaa00640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7ea00a2c00 0x7f7ea00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:27.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.646+0000 7f7ebaa00640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7ea00a2c00 0x7f7ea00a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8000c00 tx=0x7f7ea8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.649+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7eac09c340 con 0x7f7ebc103a80 2024-07-29T00:33:27.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.876+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ebc10ba60 con 0x7f7ebc103a80 2024-07-29T00:33:27.877 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:27 smithi175 ceph-mon[32138]: Deploying daemon mds.h on smithi181 2024-07-29T00:33:27.877 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:27 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] up:boot 2024-07-29T00:33:27.877 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:27 smithi175 ceph-mon[32138]: fsmap 7 up:standby 2024-07-29T00:33:27.877 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:27 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:33:27.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.877+0000 7f7eb3e00640 1 -- 172.21.15.175:0/824418239 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f7eac09bc40 con 0x7f7ebc103a80 2024-07-29T00:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":8,"btime":"2024-07-29T00:33:26:580640+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-07-29T00:33:27.879 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 8 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7ea00a2c00 msgr2=0x7f7ea00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7ea00a2c00 0x7f7ea00a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8000c00 tx=0x7f7ea8005b50 comp rx=0 tx=0).stop 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 msgr2=0x7f7ebc19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc19a370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9f0 tx=0x7f7eac00edf0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 shutdown_connections 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc109a80 0x7f7ebc19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f7ea00a2c00 0x7f7ea00a50c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ebc104420 0x7f7ebc19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 --2- 172.21.15.175:0/824418239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc103a80 0x7f7ebc19a370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.880+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 >> 172.21.15.175:0/824418239 conn(0x7f7ebc0ff760 msgr2=0x7f7ebc0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:27.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.881+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 shutdown_connections 2024-07-29T00:33:27.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:27.881+0000 7f7ec2800640 1 -- 172.21.15.175:0/824418239 wait complete. 2024-07-29T00:33:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:27 smithi181 ceph-mon[36706]: Deploying daemon mds.h on smithi181 2024-07-29T00:33:28.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:27 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] up:boot 2024-07-29T00:33:28.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:27 smithi181 ceph-mon[36706]: fsmap 7 up:standby 2024-07-29T00:33:28.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:27 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:33:28.000 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:27 smithi181 systemd[1]: Starting Ceph mds.h for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:27 smithi204 ceph-mon[37368]: Deploying daemon mds.h on smithi181 2024-07-29T00:33:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:27 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] up:boot 2024-07-29T00:33:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:27 smithi204 ceph-mon[37368]: fsmap 7 up:standby 2024-07-29T00:33:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:27 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:33:28.345 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 podman[66129]: 2024-07-29 00:33:27.961721541 +0000 UTC m=+0.017267403 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:28.346 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 podman[66129]: 2024-07-29 00:33:28.085444139 +0000 UTC m=+0.140990040 container create 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-07-29T00:33:28.346 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 podman[66129]: 2024-07-29 00:33:28.34645217 +0000 UTC m=+0.401998028 container init 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-07-29T00:33:28.613 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 podman[66129]: 2024-07-29 00:33:28.349261158 +0000 UTC m=+0.404807019 container start 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:33:28.613 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mds[66148]: -- 172.21.15.181:0/219396479 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55727ba81860 con 0x55727ba5f400 2024-07-29T00:33:28.613 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h[66144]: starting mds.h at 2024-07-29T00:33:28.614 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 bash[66129]: 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce 2024-07-29T00:33:28.614 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:33:28 smithi181 systemd[1]: Started Ceph mds.h for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/824418239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:29.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:28 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/824418239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:29.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:28 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/824418239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:29.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:29.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:29.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:28 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:29 smithi181 ceph-mon[36706]: Deploying daemon mds.i on smithi204 2024-07-29T00:33:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:29 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] up:boot 2024-07-29T00:33:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:29 smithi181 ceph-mon[36706]: fsmap 8 up:standby 2024-07-29T00:33:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:29 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:33:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:29 smithi204 ceph-mon[37368]: Deploying daemon mds.i on smithi204 2024-07-29T00:33:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:29 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] up:boot 2024-07-29T00:33:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:29 smithi204 ceph-mon[37368]: fsmap 8 up:standby 2024-07-29T00:33:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:29 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:33:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:29 smithi175 ceph-mon[32138]: Deploying daemon mds.i on smithi204 2024-07-29T00:33:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:29 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] up:boot 2024-07-29T00:33:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:29 smithi175 ceph-mon[32138]: fsmap 8 up:standby 2024-07-29T00:33:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:29 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:33:30.586 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 systemd[1]: Starting Ceph mds.i for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:30.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:30.923 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:30 smithi204 ceph-mon[37368]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:30.923 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 podman[67596]: 2024-07-29 00:33:30.695648772 +0000 UTC m=+0.115062179 container create 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-29T00:33:30.923 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 podman[67596]: 2024-07-29 00:33:30.598228573 +0000 UTC m=+0.017641989 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:30.923 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 podman[67596]: 2024-07-29 00:33:30.923821833 +0000 UTC m=+0.343235259 container init 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, ceph=True) 2024-07-29T00:33:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:30 smithi181 ceph-mon[36706]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:31.033 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:30 smithi175 ceph-mon[32138]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:31.178 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:31.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 podman[67596]: 2024-07-29 00:33:30.926854686 +0000 UTC m=+0.346268094 container start 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:33:31.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 ceph-mds[67615]: -- 172.21.15.204:0/4105015619 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f01053860 con 0x555f01031400 2024-07-29T00:33:31.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i[67611]: starting mds.i at 2024-07-29T00:33:31.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:31 smithi204 bash[67596]: 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee 2024-07-29T00:33:31.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:33:31 smithi204 systemd[1]: Started Ceph mds.i for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:31.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 -- 172.21.15.175:0/1209816101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 msgr2=0x7fe1900767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 --2- 172.21.15.175:0/1209816101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe1900767f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe188009ca0 tx=0x7fe18802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 -- 172.21.15.175:0/1209816101 shutdown_connections 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 --2- 172.21.15.175:0/1209816101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190112560 0x7fe190114950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 --2- 172.21.15.175:0/1209816101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190076d30 0x7fe190112020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 --2- 172.21.15.175:0/1209816101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe1900767f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 -- 172.21.15.175:0/1209816101 >> 172.21.15.175:0/1209816101 conn(0x7fe1900ff600 msgr2=0x7fe190101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 -- 172.21.15.175:0/1209816101 shutdown_connections 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.761+0000 7fe194e00640 1 -- 172.21.15.175:0/1209816101 wait complete. 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.762+0000 7fe194e00640 1 Processor -- start 2024-07-29T00:33:31.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.762+0000 7fe194e00640 1 -- start start 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.762+0000 7fe194e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe19019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.762+0000 7fe194e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190076d30 0x7fe19019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.762+0000 7fe194e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190112560 0x7fe1901a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe194e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901180e0 con 0x7fe190076410 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe194e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe190117f60 con 0x7fe190076d30 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe194e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe190118260 con 0x7fe190112560 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe19019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe19019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:36754/0 (socket says 172.21.15.175:36754) 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 -- 172.21.15.175:0/3228970603 learned_addr learned my addr 172.21.15.175:0/3228970603 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18fe00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190112560 0x7fe1901a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18ea00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190076d30 0x7fe19019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:31.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 -- 172.21.15.175:0/3228970603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190112560 msgr2=0x7fe1901a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190112560 0x7fe1901a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 -- 172.21.15.175:0/3228970603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190076d30 msgr2=0x7fe19019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190076d30 0x7fe19019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18f400640 1 -- 172.21.15.175:0/3228970603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1901a35b0 con 0x7fe190076410 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18fe00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190112560 0x7fe1901a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.763+0000 7fe18ea00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190076d30 0x7fe19019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.764+0000 7fe18f400640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe19019e600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe18802ade0 tx=0x7fe1880319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.764+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18802fa20 con 0x7fe190076410 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.764+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18802fbc0 con 0x7fe190076410 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.764+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18803a660 con 0x7fe190076410 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.764+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1901a3840 con 0x7fe190076410 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.764+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901ab180 con 0x7fe190076410 2024-07-29T00:33:31.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.765+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190077b50 con 0x7fe190076410 2024-07-29T00:33:31.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.767+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe1880305c0 con 0x7fe190076410 2024-07-29T00:33:31.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.768+0000 7fe187e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe1740a2c00 0x7fe1740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:31.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.769+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fe1880eff70 con 0x7fe190076410 2024-07-29T00:33:31.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.769+0000 7fe18ea00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe1740a2c00 0x7fe1740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:31.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.769+0000 7fe18ea00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe1740a2c00 0x7fe1740a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe190079060 tx=0x7fe17c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:31.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:31.773+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe1880bd2d0 con 0x7fe190076410 2024-07-29T00:33:32.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.004+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe19010d6f0 con 0x7fe190076410 2024-07-29T00:33:32.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.005+0000 7fe187e00640 1 -- 172.21.15.175:0/3228970603 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7fe1880bcbd0 con 0x7fe190076410 2024-07-29T00:33:32.005 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:32.006 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":10,"btime":"2024-07-29T00:33:31:625251+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-07-29T00:33:32.006 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 10 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe1740a2c00 msgr2=0x7fe1740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe1740a2c00 0x7fe1740a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe190079060 tx=0x7fe17c005f90 comp rx=0 tx=0).stop 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 msgr2=0x7fe19019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe19019e600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe18802ade0 tx=0x7fe1880319f0 comp rx=0 tx=0).stop 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 shutdown_connections 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe1740a2c00 0x7fe1740a50c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190112560 0x7fe1901a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe190076d30 0x7fe19019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:32.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 --2- 172.21.15.175:0/3228970603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe190076410 0x7fe19019e600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:32.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.008+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 >> 172.21.15.175:0/3228970603 conn(0x7fe1900ff600 msgr2=0x7fe1901019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:32.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.009+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 shutdown_connections 2024-07-29T00:33:32.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:32.009+0000 7fe194e00640 1 -- 172.21.15.175:0/3228970603 wait complete. 2024-07-29T00:33:32.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: Deploying daemon mds.g on smithi175 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] up:boot 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: fsmap 9 up:standby 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:33:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3228970603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:32.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: Deploying daemon mds.g on smithi175 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] up:boot 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: fsmap 9 up:standby 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:33:32.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3228970603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:32.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:32.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: Deploying daemon mds.g on smithi175 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] up:boot 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: fsmap 9 up:standby 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:33:32.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3228970603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:34 smithi204 ceph-mon[37368]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:34.854 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:34 smithi175 ceph-mon[32138]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:34 smithi181 ceph-mon[36706]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:35.149 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:35.207 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:34 smithi175 systemd[1]: Starting Ceph mds.g for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:35.518 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 podman[72483]: 2024-07-29 00:33:35.157903216 +0000 UTC m=+0.037446384 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:35.518 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 podman[72483]: 2024-07-29 00:33:35.287610986 +0000 UTC m=+0.167154163 container create e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-29T00:33:35.784 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 podman[72483]: 2024-07-29 00:33:35.584516111 +0000 UTC m=+0.464059263 container init e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-29T00:33:35.784 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 podman[72483]: 2024-07-29 00:33:35.595954852 +0000 UTC m=+0.475498027 container start e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-29T00:33:35.784 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 ceph-mds[72546]: -- 172.21.15.175:0/3241451304 <== mon.0 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56455cc6f860 con 0x56455cc4c800 2024-07-29T00:33:35.784 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g[72522]: starting mds.g at 2024-07-29T00:33:35.785 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 bash[72483]: e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb 2024-07-29T00:33:35.785 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:33:35 smithi175 systemd[1]: Started Ceph mds.g for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:35.797 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:36.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 -- 172.21.15.175:0/1374866500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 msgr2=0x7fa9b8109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:36.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 --2- 172.21.15.175:0/1374866500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 0x7fa9b8109e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009930 tx=0x7fa9a802a420 comp rx=0 tx=0).stop 2024-07-29T00:33:36.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 -- 172.21.15.175:0/1374866500 shutdown_connections 2024-07-29T00:33:36.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 --2- 172.21.15.175:0/1374866500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8104420 0x7fa9b8110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 --2- 172.21.15.175:0/1374866500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8103a80 0x7fa9b8103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 --2- 172.21.15.175:0/1374866500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 0x7fa9b8109e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 -- 172.21.15.175:0/1374866500 >> 172.21.15.175:0/1374866500 conn(0x7fa9b80ff780 msgr2=0x7fa9b8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:36.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 -- 172.21.15.175:0/1374866500 shutdown_connections 2024-07-29T00:33:36.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.502+0000 7fa9bde00640 1 -- 172.21.15.175:0/1374866500 wait complete. 2024-07-29T00:33:36.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 Processor -- start 2024-07-29T00:33:36.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 -- start start 2024-07-29T00:33:36.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 0x7fa9b819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:36.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8104420 0x7fa9b819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:36.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 0x7fa9b819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:36.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8113ed0 con 0x7fa9b8109a80 2024-07-29T00:33:36.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8113d50 con 0x7fa9b8103a80 2024-07-29T00:33:36.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.503+0000 7fa9bde00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8114050 con 0x7fa9b8104420 2024-07-29T00:33:36.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 0x7fa9b819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:36.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9afe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8104420 0x7fa9b819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:36.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 0x7fa9b819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38810/0 (socket says 172.21.15.175:38810) 2024-07-29T00:33:36.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9afe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8104420 0x7fa9b819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33974/0 (socket says 172.21.15.175:33974) 2024-07-29T00:33:36.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 0x7fa9b819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:36.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 -- 172.21.15.175:0/3184226302 learned_addr learned my addr 172.21.15.175:0/3184226302 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:36.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 -- 172.21.15.175:0/3184226302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8104420 msgr2=0x7fa9b819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:36.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8104420 0x7fa9b819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 -- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 msgr2=0x7fa9b819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:36.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 0x7fa9b819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 -- 172.21.15.175:0/3184226302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b819f320 con 0x7fa9b8103a80 2024-07-29T00:33:36.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.504+0000 7fa9b6a00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 0x7fa9b819a370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9a802a940 tx=0x7fa9a8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:36.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.505+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a803f070 con 0x7fa9b8103a80 2024-07-29T00:33:36.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.505+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9a8031c90 con 0x7fa9b8103a80 2024-07-29T00:33:36.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.505+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b819f5b0 con 0x7fa9b8103a80 2024-07-29T00:33:36.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.505+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8035030 con 0x7fa9b8103a80 2024-07-29T00:33:36.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.505+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81a6ef0 con 0x7fa9b8103a80 2024-07-29T00:33:36.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.506+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b81051c0 con 0x7fa9b8103a80 2024-07-29T00:33:36.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.508+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa9a8042070 con 0x7fa9b8103a80 2024-07-29T00:33:36.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.509+0000 7fa9bca00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa99c0a2b80 0x7fa99c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:36.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.510+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fa9a80ef240 con 0x7fa9b8103a80 2024-07-29T00:33:36.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.510+0000 7fa9afe00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa99c0a2b80 0x7fa99c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:36.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.510+0000 7fa9afe00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa99c0a2b80 0x7fa99c0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8101d80 tx=0x7fa9a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:36.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.513+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa9a80bbf50 con 0x7fa9b8103a80 2024-07-29T00:33:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.751+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9b810bff0 con 0x7fa9b8103a80 2024-07-29T00:33:36.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:36.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:36.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:36.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:36.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:36.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:36.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:36 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:36.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.752+0000 7fa9bca00640 1 -- 172.21.15.175:0/3184226302 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7fa9a80c0250 con 0x7fa9b8103a80 2024-07-29T00:33:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:36.754 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":11,"btime":"2024-07-29T00:33:36:586329+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-07-29T00:33:36.754 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 11 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.755+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa99c0a2b80 msgr2=0x7fa99c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.755+0000 7fa9bde00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa99c0a2b80 0x7fa99c0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8101d80 tx=0x7fa9a0005f90 comp rx=0 tx=0).stop 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.755+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 msgr2=0x7fa9b819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.755+0000 7fa9bde00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 0x7fa9b819a370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9a802a940 tx=0x7fa9a8003b20 comp rx=0 tx=0).stop 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.755+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 shutdown_connections 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.755+0000 7fa9bde00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b8109a80 0x7fa9b819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.756+0000 7fa9bde00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fa99c0a2b80 0x7fa99c0a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.756+0000 7fa9bde00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b8104420 0x7fa9b819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.756+0000 7fa9bde00640 1 --2- 172.21.15.175:0/3184226302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8103a80 0x7fa9b819a370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.756+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 >> 172.21.15.175:0/3184226302 conn(0x7fa9b80ff780 msgr2=0x7fa9b80fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.756+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 shutdown_connections 2024-07-29T00:33:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:36.756+0000 7fa9bde00640 1 -- 172.21.15.175:0/3184226302 wait complete. 2024-07-29T00:33:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:37.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:37.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:36 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:36 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:37 smithi181 ceph-mon[36706]: Deploying daemon mds.a on smithi175 2024-07-29T00:33:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:37 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] up:boot 2024-07-29T00:33:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:37 smithi181 ceph-mon[36706]: fsmap 10 up:standby 2024-07-29T00:33:38.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:37 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:33:38.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:37 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3184226302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:38.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:37 smithi175 ceph-mon[32138]: Deploying daemon mds.a on smithi175 2024-07-29T00:33:38.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:37 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] up:boot 2024-07-29T00:33:38.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:37 smithi175 ceph-mon[32138]: fsmap 10 up:standby 2024-07-29T00:33:38.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:37 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:33:38.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:37 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3184226302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:37 smithi204 ceph-mon[37368]: Deploying daemon mds.a on smithi175 2024-07-29T00:33:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:37 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] up:boot 2024-07-29T00:33:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:37 smithi204 ceph-mon[37368]: fsmap 10 up:standby 2024-07-29T00:33:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:37 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:33:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:37 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3184226302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:38 smithi175 ceph-mon[32138]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:38 smithi181 ceph-mon[36706]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:38 smithi204 ceph-mon[37368]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:39.285 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:38 smithi175 systemd[1]: Starting Ceph mds.a for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:39.611 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 podman[72903]: 2024-07-29 00:33:39.199163228 +0000 UTC m=+0.039564115 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:39.611 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 podman[72903]: 2024-07-29 00:33:39.329531188 +0000 UTC m=+0.169932087 container create c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-ba82afe, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:33:39.611 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 podman[72903]: 2024-07-29 00:33:39.610569082 +0000 UTC m=+0.450969988 container init c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, ceph=True, GIT_BRANCH=HEAD) 2024-07-29T00:33:40.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 podman[72903]: 2024-07-29 00:33:39.620704106 +0000 UTC m=+0.461104987 container start c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-07-29T00:33:40.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 ceph-mds[72921]: -- 172.21.15.175:0/3035760699 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e36845e680 con 0x55e368433400 2024-07-29T00:33:40.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 ceph-mds[72921]: -- 172.21.15.175:0/3035760699 <== mon.0 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e368455860 con 0x55e368433400 2024-07-29T00:33:40.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a[72917]: starting mds.a at 2024-07-29T00:33:40.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 bash[72903]: c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 2024-07-29T00:33:40.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:33:39 smithi175 systemd[1]: Started Ceph mds.a for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:40.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:40.927 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:40 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:41.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:41.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:40 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:41.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-29T00:33:41.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-29T00:33:41.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:40 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:41.333 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:41.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:41 smithi175 ceph-mon[32138]: Deploying daemon mds.b on smithi181 2024-07-29T00:33:41.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:41 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6840/1735377936,v1:172.21.15.175:6841/1735377936] up:boot 2024-07-29T00:33:41.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:41 smithi175 ceph-mon[32138]: fsmap 11 up:standby 2024-07-29T00:33:41.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:41 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-29T00:33:41.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:41 smithi181 ceph-mon[36706]: Deploying daemon mds.b on smithi181 2024-07-29T00:33:42.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:41 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6840/1735377936,v1:172.21.15.175:6841/1735377936] up:boot 2024-07-29T00:33:42.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:41 smithi181 ceph-mon[36706]: fsmap 11 up:standby 2024-07-29T00:33:42.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:41 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-29T00:33:42.000 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:41 smithi181 systemd[1]: Starting Ceph mds.b for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 -- 172.21.15.175:0/4187545505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 msgr2=0x7fc1e4108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/4187545505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e4108130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009900 tx=0x7fc1d002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 -- 172.21.15.175:0/4187545505 shutdown_connections 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/4187545505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 0x7fc1e4114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/4187545505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e4108130 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/4187545505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e410dd50 0x7fc1e410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 -- 172.21.15.175:0/4187545505 >> 172.21.15.175:0/4187545505 conn(0x7fc1e4077670 msgr2=0x7fc1e4079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 -- 172.21.15.175:0/4187545505 shutdown_connections 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:41.999+0000 7fc1e8a00640 1 -- 172.21.15.175:0/4187545505 wait complete. 2024-07-29T00:33:42.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 Processor -- start 2024-07-29T00:33:42.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 -- start start 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 0x7fc1e419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e410dd50 0x7fc1e41a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4118070 con 0x7fc1e4108670 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4117ef0 con 0x7fc1e4107d10 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.001+0000 7fc1e8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41181f0 con 0x7fc1e410dd50 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59038/0 (socket says 172.21.15.175:59038) 2024-07-29T00:33:42.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 -- 172.21.15.175:0/2529324050 learned_addr learned my addr 172.21.15.175:0/2529324050 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e2a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 0x7fc1e419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3e00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e410dd50 0x7fc1e41a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 -- 172.21.15.175:0/2529324050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e410dd50 msgr2=0x7fc1e41a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e410dd50 0x7fc1e41a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 -- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 msgr2=0x7fc1e419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 0x7fc1e419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 -- 172.21.15.175:0/2529324050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e41a3540 con 0x7fc1e4107d10 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e2a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 0x7fc1e419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.002+0000 7fc1e3400640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e419e590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400e9f0 tx=0x7fc1d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.003+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400cd80 con 0x7fc1e4107d10 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.003+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1d4005500 con 0x7fc1e4107d10 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.003+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41a3830 con 0x7fc1e4107d10 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.003+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4010690 con 0x7fc1e4107d10 2024-07-29T00:33:42.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.003+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41ab110 con 0x7fc1e4107d10 2024-07-29T00:33:42.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.004+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e4108130 con 0x7fc1e4107d10 2024-07-29T00:33:42.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.006+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc1d40046c0 con 0x7fc1e4107d10 2024-07-29T00:33:42.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.008+0000 7fc1dbe00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc1c80a2c00 0x7fc1c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:42.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.008+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fc1d40cefe0 con 0x7fc1e4107d10 2024-07-29T00:33:42.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.008+0000 7fc1e2a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc1c80a2c00 0x7fc1c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:42.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.009+0000 7fc1e2a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc1c80a2c00 0x7fc1c80a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1e407a0a0 tx=0x7fc1d0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:42.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.011+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc1d409c3a0 con 0x7fc1e4107d10 2024-07-29T00:33:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:41 smithi204 ceph-mon[37368]: Deploying daemon mds.b on smithi181 2024-07-29T00:33:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:41 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6840/1735377936,v1:172.21.15.175:6841/1735377936] up:boot 2024-07-29T00:33:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:41 smithi204 ceph-mon[37368]: fsmap 11 up:standby 2024-07-29T00:33:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:41 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-29T00:33:42.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.241+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1e4110240 con 0x7fc1e4107d10 2024-07-29T00:33:42.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.242+0000 7fc1dbe00640 1 -- 172.21.15.175:0/2529324050 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7fc1d409bca0 con 0x7fc1e4107d10 2024-07-29T00:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":12,"btime":"2024-07-29T00:33:40:614371+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-07-29T00:33:42.243 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 12 2024-07-29T00:33:42.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc1c80a2c00 msgr2=0x7fc1c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:42.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc1c80a2c00 0x7fc1c80a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1e407a0a0 tx=0x7fc1d0005a20 comp rx=0 tx=0).stop 2024-07-29T00:33:42.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 msgr2=0x7fc1e419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:42.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e419e590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400e9f0 tx=0x7fc1d400edf0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 shutdown_connections 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc1c80a2c00 0x7fc1c80a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1e410dd50 0x7fc1e41a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e4108670 0x7fc1e419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 --2- 172.21.15.175:0/2529324050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e4107d10 0x7fc1e419e590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 >> 172.21.15.175:0/2529324050 conn(0x7fc1e4077670 msgr2=0x7fc1e4078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 shutdown_connections 2024-07-29T00:33:42.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:42.245+0000 7fc1e8a00640 1 -- 172.21.15.175:0/2529324050 wait complete. 2024-07-29T00:33:42.394 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 podman[66431]: 2024-07-29 00:33:41.979565151 +0000 UTC m=+0.016907053 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:33:42.394 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 podman[66431]: 2024-07-29 00:33:42.116902913 +0000 UTC m=+0.154244848 container create 84ff2af4a35aaf2c3377bd63ee11baf0384f043ce71b5c4399e7cff90ba30949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe) 2024-07-29T00:33:42.394 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 podman[66431]: 2024-07-29 00:33:42.394656201 +0000 UTC m=+0.431998095 container init 84ff2af4a35aaf2c3377bd63ee11baf0384f043ce71b5c4399e7cff90ba30949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:33:42.646 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 podman[66431]: 2024-07-29 00:33:42.397315657 +0000 UTC m=+0.434657554 container start 84ff2af4a35aaf2c3377bd63ee11baf0384f043ce71b5c4399e7cff90ba30949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-29T00:33:42.646 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mds[66450]: -- 172.21.15.181:0/645032282 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcfedab860 con 0x55fcfed88800 2024-07-29T00:33:42.646 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: starting mds.b at 2024-07-29T00:33:42.647 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 bash[66431]: 84ff2af4a35aaf2c3377bd63ee11baf0384f043ce71b5c4399e7cff90ba30949 2024-07-29T00:33:42.647 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:33:42 smithi181 systemd[1]: Started Ceph mds.b for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:33:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2529324050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:42 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2529324050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:42 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:43.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2529324050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:43.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:42 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:43.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:43 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] up:boot 2024-07-29T00:33:43.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:43 smithi181 ceph-mon[36706]: fsmap 12 up:standby 2024-07-29T00:33:43.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:33:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:43 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:43.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:43 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] up:boot 2024-07-29T00:33:43.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:43 smithi204 ceph-mon[37368]: fsmap 12 up:standby 2024-07-29T00:33:43.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:33:43.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:43 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:43.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:43 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] up:boot 2024-07-29T00:33:43.918 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:43 smithi175 ceph-mon[32138]: fsmap 12 up:standby 2024-07-29T00:33:43.919 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:33:43.919 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:43 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:44 smithi181 ceph-mon[36706]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:44 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:44 smithi204 ceph-mon[37368]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:45.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.007 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:44 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:44 smithi175 ceph-mon[32138]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:44 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:45.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:33:45.616 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 -- 172.21.15.175:0/661281318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1076c0 msgr2=0x7ff38c107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 --2- 172.21.15.175:0/661281318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1076c0 0x7ff38c107aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009890 tx=0x7ff37c02a360 comp rx=0 tx=0).stop 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 -- 172.21.15.175:0/661281318 shutdown_connections 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 --2- 172.21.15.175:0/661281318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff38c102060 0x7ff38c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 --2- 172.21.15.175:0/661281318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1016c0 0x7ff38c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 --2- 172.21.15.175:0/661281318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1076c0 0x7ff38c107aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 -- 172.21.15.175:0/661281318 >> 172.21.15.175:0/661281318 conn(0x7ff38c0fd400 msgr2=0x7ff38c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:46.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 -- 172.21.15.175:0/661281318 shutdown_connections 2024-07-29T00:33:46.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.299+0000 7ff38b200640 1 -- 172.21.15.175:0/661281318 wait complete. 2024-07-29T00:33:46.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.300+0000 7ff38b200640 1 Processor -- start 2024-07-29T00:33:46.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 -- start start 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1016c0 0x7ff38c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff38c102060 0x7ff38c074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1076c0 0x7ff38c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c111ac0 con 0x7ff38c102060 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c111940 con 0x7ff38c1076c0 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.301+0000 7ff38b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c111c40 con 0x7ff38c1016c0 2024-07-29T00:33:46.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff389e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1016c0 0x7ff38c0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:46.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff389e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1016c0 0x7ff38c0741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42852/0 (socket says 172.21.15.175:42852) 2024-07-29T00:33:46.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff389e00640 1 -- 172.21.15.175:0/3802059066 learned_addr learned my addr 172.21.15.175:0/3802059066 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:46.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff38a800640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1076c0 0x7ff38c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:46.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff389400640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff38c102060 0x7ff38c074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:46.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff38a800640 1 -- 172.21.15.175:0/3802059066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1016c0 msgr2=0x7ff38c0741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff38a800640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1016c0 0x7ff38c0741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff38a800640 1 -- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff38c102060 msgr2=0x7ff38c074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff38a800640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff38c102060 0x7ff38c074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.302+0000 7ff38a800640 1 -- 172.21.15.175:0/3802059066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c1a6480 con 0x7ff38c1076c0 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.303+0000 7ff38a800640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1076c0 0x7ff38c074c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff38400ef00 tx=0x7ff38400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.303+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384004b20 con 0x7ff38c1076c0 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.303+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384003e50 con 0x7ff38c1076c0 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.303+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c1a6770 con 0x7ff38c1076c0 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.303+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400b570 con 0x7ff38c1076c0 2024-07-29T00:33:46.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.303+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1a6cb0 con 0x7ff38c1076c0 2024-07-29T00:33:46.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.304+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c101b20 con 0x7ff38c1076c0 2024-07-29T00:33:46.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.308+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff38401d020 con 0x7ff38c1076c0 2024-07-29T00:33:46.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.309+0000 7ff382a00640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff3700a2c00 0x7ff3700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:46.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.309+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7ff3840ce9a0 con 0x7ff38c1076c0 2024-07-29T00:33:46.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.309+0000 7ff389e00640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff3700a2c00 0x7ff3700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:46.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.310+0000 7ff389e00640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff3700a2c00 0x7ff3700a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005e00 tx=0x7ff37c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:46.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.313+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff38409bd00 con 0x7ff38c1076c0 2024-07-29T00:33:46.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:46.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:46 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:46.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:46 smithi175 ceph-mon[32138]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:46.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.554+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff38c02d5a0 con 0x7ff38c1076c0 2024-07-29T00:33:46.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.555+0000 7ff382a00640 1 -- 172.21.15.175:0/3802059066 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7ff38409b600 con 0x7ff38c1076c0 2024-07-29T00:33:46.554 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:46.554 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":13,"btime":"2024-07-29T00:33:42:640094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-07-29T00:33:46.555 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 13 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.557+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff3700a2c00 msgr2=0x7ff3700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.557+0000 7ff38b200640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff3700a2c00 0x7ff3700a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005e00 tx=0x7ff37c0056d0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.557+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1076c0 msgr2=0x7ff38c074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.557+0000 7ff38b200640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1076c0 0x7ff38c074c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff38400ef00 tx=0x7ff38400c740 comp rx=0 tx=0).stop 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 shutdown_connections 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c1076c0 0x7ff38c074c60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff38c102060 0x7ff38c074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff3700a2c00 0x7ff3700a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 --2- 172.21.15.175:0/3802059066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff38c1016c0 0x7ff38c0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:46.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 >> 172.21.15.175:0/3802059066 conn(0x7ff38c0fd400 msgr2=0x7ff38c0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:46.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 shutdown_connections 2024-07-29T00:33:46.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:46.558+0000 7ff38b200640 1 -- 172.21.15.175:0/3802059066 wait complete. 2024-07-29T00:33:46.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:46.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:46 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:46.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:46 smithi204 ceph-mon[37368]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:46.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:46.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:46 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:46.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:46 smithi181 ceph-mon[36706]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:47.395 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:47.395 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:47.395 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:47.395 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:47 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:47.395 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:47 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3802059066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:47.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-07-29T00:33:47.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:47.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:47.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:47.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:47 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:47.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:47 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3802059066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:47.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:47.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:47.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:47.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:47 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:33:47.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:47 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3802059066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:33:48.561 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:48.712 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:48 smithi175 ceph-mon[32138]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:48.713 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:48 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.713 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:48 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.713 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:48 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.713 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:48 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:48 smithi181 ceph-mon[36706]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:48 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:48 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:48 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:48 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:48 smithi204 ceph-mon[37368]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:48 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:48 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:48 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:48.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:48 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 -- 172.21.15.175:0/251049313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8810dd50 msgr2=0x7f6a8810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 --2- 172.21.15.175:0/251049313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8810dd50 0x7f6a8810e130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009930 tx=0x7f6a7802a420 comp rx=0 tx=0).stop 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 -- 172.21.15.175:0/251049313 shutdown_connections 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 --2- 172.21.15.175:0/251049313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a88108670 0x7f6a88114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 --2- 172.21.15.175:0/251049313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88107d10 0x7f6a88108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 --2- 172.21.15.175:0/251049313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8810dd50 0x7f6a8810e130 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.224+0000 7f6a87200640 1 -- 172.21.15.175:0/251049313 >> 172.21.15.175:0/251049313 conn(0x7f6a88077670 msgr2=0x7f6a88079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.225+0000 7f6a87200640 1 -- 172.21.15.175:0/251049313 shutdown_connections 2024-07-29T00:33:49.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.225+0000 7f6a87200640 1 -- 172.21.15.175:0/251049313 wait complete. 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.225+0000 7f6a87200640 1 Processor -- start 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 -- start start 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a88107d10 0x7f6a8819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88108670 0x7f6a8819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a8810dd50 0x7f6a881a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881180b0 con 0x7f6a88107d10 2024-07-29T00:33:49.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88117f30 con 0x7f6a8810dd50 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a87200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88118230 con 0x7f6a88108670 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a85e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a88107d10 0x7f6a8819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a7d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88108670 0x7f6a8819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a85e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a88107d10 0x7f6a8819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35378/0 (socket says 172.21.15.175:35378) 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.226+0000 7f6a85e00640 1 -- 172.21.15.175:0/32768231 learned_addr learned my addr 172.21.15.175:0/32768231 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a86800640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a8810dd50 0x7f6a881a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7d400640 1 -- 172.21.15.175:0/32768231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a8810dd50 msgr2=0x7f6a881a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7d400640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a8810dd50 0x7f6a881a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7d400640 1 -- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a88107d10 msgr2=0x7f6a8819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7d400640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a88107d10 0x7f6a8819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7d400640 1 -- 172.21.15.175:0/32768231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a881a3580 con 0x7f6a88108670 2024-07-29T00:33:49.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7d400640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88108670 0x7f6a8819eb10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000e9f0 tx=0x7f6a7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:49.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.227+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000ccc0 con 0x7f6a88108670 2024-07-29T00:33:49.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.228+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a881a3870 con 0x7f6a88108670 2024-07-29T00:33:49.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.228+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a70010910 con 0x7f6a88108670 2024-07-29T00:33:49.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.228+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70018960 con 0x7f6a88108670 2024-07-29T00:33:49.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.228+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a881ab150 con 0x7f6a88108670 2024-07-29T00:33:49.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.229+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a88108130 con 0x7f6a88108670 2024-07-29T00:33:49.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.231+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6a70010ac0 con 0x7f6a88108670 2024-07-29T00:33:49.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.232+0000 7f6a7f400640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6a6c0a2b80 0x7f6a6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:49.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.232+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f6a700cf490 con 0x7f6a88108670 2024-07-29T00:33:49.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.233+0000 7f6a85e00640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6a6c0a2b80 0x7f6a6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:49.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.233+0000 7f6a85e00640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6a6c0a2b80 0x7f6a6c0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a780096c0 tx=0x7f6a78005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.234+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6a7009c1a0 con 0x7f6a88108670 2024-07-29T00:33:49.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.446+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6a8802d5a0 con 0x7f6a88108670 2024-07-29T00:33:49.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.447+0000 7f6a7f400640 1 -- 172.21.15.175:0/32768231 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7f6a700a04a0 con 0x7f6a88108670 2024-07-29T00:33:49.446 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:33:49.446 INFO:teuthology.orchestra.run.smithi175.stdout:[ 2024-07-29T00:33:49.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:33:49.447 INFO:teuthology.orchestra.run.smithi175.stdout: "poolnum": 1, 2024-07-29T00:33:49.447 INFO:teuthology.orchestra.run.smithi175.stdout: "poolname": ".mgr" 2024-07-29T00:33:49.447 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:33:49.447 INFO:teuthology.orchestra.run.smithi175.stdout:] 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.449+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6a6c0a2b80 msgr2=0x7f6a6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.449+0000 7f6a87200640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6a6c0a2b80 0x7f6a6c0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a780096c0 tx=0x7f6a78005b50 comp rx=0 tx=0).stop 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.449+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88108670 msgr2=0x7f6a8819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.449+0000 7f6a87200640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88108670 0x7f6a8819eb10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000e9f0 tx=0x7f6a7000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.449+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 shutdown_connections 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a8810dd50 0x7f6a881a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6a6c0a2b80 0x7f6a6c0a5040 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a780096c0 tx=0x7f6a78005b50 comp rx=0 tx=0).stop 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88108670 0x7f6a8819eb10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 --2- 172.21.15.175:0/32768231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a88107d10 0x7f6a8819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:49.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 >> 172.21.15.175:0/32768231 conn(0x7f6a88077670 msgr2=0x7f6a88078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:49.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 shutdown_connections 2024-07-29T00:33:49.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:49.450+0000 7f6a87200640 1 -- 172.21.15.175:0/32768231 wait complete. 2024-07-29T00:33:49.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:49 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/32768231' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-29T00:33:49.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:49 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/32768231' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-29T00:33:49.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:49 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/32768231' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-29T00:33:50.308 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-29T00:33:50.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-29T00:33:50.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:50 smithi175 ceph-mon[32138]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:50.794 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:50 smithi204 ceph-mon[37368]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:50 smithi181 ceph-mon[36706]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 -- 172.21.15.175:0/3326195633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 msgr2=0x7fe424105f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 --2- 172.21.15.175:0/3326195633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe424105f10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009be0 tx=0x7fe41c02a740 comp rx=0 tx=0).stop 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 -- 172.21.15.175:0/3326195633 shutdown_connections 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 --2- 172.21.15.175:0/3326195633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe424105f10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 --2- 172.21.15.175:0/3326195633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe424105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 --2- 172.21.15.175:0/3326195633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe424100670 0x7fe424100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:51.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 -- 172.21.15.175:0/3326195633 >> 172.21.15.175:0/3326195633 conn(0x7fe4240fc300 msgr2=0x7fe4240fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:51.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 -- 172.21.15.175:0/3326195633 shutdown_connections 2024-07-29T00:33:51.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.452+0000 7fe42b000640 1 -- 172.21.15.175:0/3326195633 wait complete. 2024-07-29T00:33:51.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.453+0000 7fe42b000640 1 Processor -- start 2024-07-29T00:33:51.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 -- start start 2024-07-29T00:33:51.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe424100670 0x7fe42419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:51.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe42419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:51.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe4241a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241180a0 con 0x7fe424100f90 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe424117f20 con 0x7fe424100670 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe424118220 con 0x7fe424105a80 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe429200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe42419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe4241a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe4241a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:39468/0 (socket says 172.21.15.175:39468) 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe429c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe424100670 0x7fe42419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe429200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe42419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35398/0 (socket says 172.21.15.175:35398) 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.454+0000 7fe42a600640 1 -- 172.21.15.175:0/2006299938 learned_addr learned my addr 172.21.15.175:0/2006299938 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe42a600640 1 -- 172.21.15.175:0/2006299938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe424100670 msgr2=0x7fe42419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe42a600640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe424100670 0x7fe42419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe42a600640 1 -- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 msgr2=0x7fe42419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:51.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe42a600640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe42419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:51.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe42a600640 1 -- 172.21.15.175:0/2006299938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4241a34b0 con 0x7fe424105a80 2024-07-29T00:33:51.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe429200640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe42419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:51.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe42a600640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe4241a2dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe41c034040 tx=0x7fe41c0305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.455+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c040020 con 0x7fe424105a80 2024-07-29T00:33:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.456+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c003e50 con 0x7fe424105a80 2024-07-29T00:33:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.456+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c036040 con 0x7fe424105a80 2024-07-29T00:33:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.456+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241a3740 con 0x7fe424105a80 2024-07-29T00:33:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.456+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241a3bd0 con 0x7fe424105a80 2024-07-29T00:33:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.457+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424105460 con 0x7fe424105a80 2024-07-29T00:33:51.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.459+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe41c043020 con 0x7fe424105a80 2024-07-29T00:33:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.460+0000 7fe422000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe40c0a2c00 0x7fe40c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.460+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fe41c0ef8a0 con 0x7fe424105a80 2024-07-29T00:33:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.460+0000 7fe429c00640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe40c0a2c00 0x7fe40c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.461+0000 7fe429c00640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe40c0a2c00 0x7fe40c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe418005fd0 tx=0x7fe418009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.463+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe41c04a020 con 0x7fe424105a80 2024-07-29T00:33:51.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:51.655+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe4241ab280 con 0x7fe424105a80 2024-07-29T00:33:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:52.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:52.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:33:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:33:52.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:33:52.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:52 smithi175 ceph-mon[32138]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:52.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:52 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2006299938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-29T00:33:52.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:52 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-29T00:33:52.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.660+0000 7fe422000640 1 -- 172.21.15.175:0/2006299938 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7fe41c0bcc00 con 0x7fe424105a80 2024-07-29T00:33:52.663 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'cephfs_metadata' created 2024-07-29T00:33:52.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe40c0a2c00 msgr2=0x7fe40c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:52.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe40c0a2c00 0x7fe40c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe418005fd0 tx=0x7fe418009260 comp rx=0 tx=0).stop 2024-07-29T00:33:52.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 msgr2=0x7fe4241a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:52.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe4241a2dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe41c034040 tx=0x7fe41c0305d0 comp rx=0 tx=0).stop 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 shutdown_connections 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe40c0a2c00 0x7fe40c0a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe424105a80 0x7fe4241a2dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424100f90 0x7fe42419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 --2- 172.21.15.175:0/2006299938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe424100670 0x7fe42419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.666+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 >> 172.21.15.175:0/2006299938 conn(0x7fe4240fc300 msgr2=0x7fe424103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.667+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 shutdown_connections 2024-07-29T00:33:52.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:52.667+0000 7fe42b000640 1 -- 172.21.15.175:0/2006299938 wait complete. 2024-07-29T00:33:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:52 smithi181 ceph-mon[36706]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:52 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2006299938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-29T00:33:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:52 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-29T00:33:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:52 smithi204 ceph-mon[37368]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:52 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2006299938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-29T00:33:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:52 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-29T00:33:53.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-29T00:33:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:53 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-29T00:33:53.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:53 smithi175 ceph-mon[32138]: osdmap e81: 12 total, 12 up, 12 in 2024-07-29T00:33:53.956 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:53 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-29T00:33:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:53 smithi181 ceph-mon[36706]: osdmap e81: 12 total, 12 up, 12 in 2024-07-29T00:33:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:53 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-29T00:33:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:53 smithi204 ceph-mon[37368]: osdmap e81: 12 total, 12 up, 12 in 2024-07-29T00:33:54.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 -- 172.21.15.175:0/2390812845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 msgr2=0x7f263c101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:54.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 --2- 172.21.15.175:0/2390812845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c101b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2624009860 tx=0x7f262402a340 comp rx=0 tx=0).stop 2024-07-29T00:33:54.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 -- 172.21.15.175:0/2390812845 shutdown_connections 2024-07-29T00:33:54.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 --2- 172.21.15.175:0/2390812845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 0x7f263c10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:54.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 --2- 172.21.15.175:0/2390812845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c101b10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:54.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 --2- 172.21.15.175:0/2390812845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 0x7f263c107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:54.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 -- 172.21.15.175:0/2390812845 >> 172.21.15.175:0/2390812845 conn(0x7f263c0fd430 msgr2=0x7f263c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:54.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 -- 172.21.15.175:0/2390812845 shutdown_connections 2024-07-29T00:33:54.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.620+0000 7f2642200640 1 -- 172.21.15.175:0/2390812845 wait complete. 2024-07-29T00:33:54.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.622+0000 7f2642200640 1 Processor -- start 2024-07-29T00:33:54.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.622+0000 7f2642200640 1 -- start start 2024-07-29T00:33:54.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2642200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:54.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2642200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 0x7f263c074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:54.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2642200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 0x7f263c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:54.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2642200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f263c111ab0 con 0x7f263c1076b0 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2642200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f263c111930 con 0x7f263c1016b0 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2642200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f263c111c30 con 0x7f263c102050 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2641800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 0x7f263c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f263be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 0x7f263c074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2640e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2641800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 0x7f263c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35412/0 (socket says 172.21.15.175:35412) 2024-07-29T00:33:54.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.623+0000 7f2641800640 1 -- 172.21.15.175:0/3694035247 learned_addr learned my addr 172.21.15.175:0/3694035247 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:54.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2640e00640 1 -- 172.21.15.175:0/3694035247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 msgr2=0x7f263c074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2640e00640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 0x7f263c074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2640e00640 1 -- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 msgr2=0x7f263c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2640e00640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 0x7f263c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2640e00640 1 -- 172.21.15.175:0/3694035247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c1a63f0 con 0x7f263c1016b0 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f263be00640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 0x7f263c074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2640e00640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c0741e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f262c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c003970 con 0x7f263c1016b0 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c004020 con 0x7f263c1016b0 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c1a66e0 con 0x7f263c1016b0 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c00ba60 con 0x7f263c1016b0 2024-07-29T00:33:54.626 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.624+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c1a6c40 con 0x7f263c1016b0 2024-07-29T00:33:54.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.626+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c101b10 con 0x7f263c1016b0 2024-07-29T00:33:54.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.627+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f262c0041d0 con 0x7f263c1016b0 2024-07-29T00:33:54.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.629+0000 7f2639600640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26200a2c00 0x7f26200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:54.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.629+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f262c0cee20 con 0x7f263c1016b0 2024-07-29T00:33:54.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.629+0000 7f263be00640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26200a2c00 0x7f26200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:54.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.630+0000 7f263be00640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26200a2c00 0x7f26200a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f262402a930 tx=0x7f26240056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:54.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.633+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f262c09c010 con 0x7f263c1016b0 2024-07-29T00:33:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:54 smithi175 ceph-mon[32138]: pgmap v283: 33 pgs: 5 creating+peering, 27 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:54 smithi175 ceph-mon[32138]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-29T00:33:54.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:54 smithi175 ceph-mon[32138]: osdmap e82: 12 total, 12 up, 12 in 2024-07-29T00:33:54.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:54.829+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f263c1a6f00 con 0x7f263c1016b0 2024-07-29T00:33:54.901 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:54 smithi181 ceph-mon[36706]: pgmap v283: 33 pgs: 5 creating+peering, 27 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:54.901 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:54 smithi181 ceph-mon[36706]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-29T00:33:54.901 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:54 smithi181 ceph-mon[36706]: osdmap e82: 12 total, 12 up, 12 in 2024-07-29T00:33:55.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:54 smithi204 ceph-mon[37368]: pgmap v283: 33 pgs: 5 creating+peering, 27 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:55.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:54 smithi204 ceph-mon[37368]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-29T00:33:55.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:54 smithi204 ceph-mon[37368]: osdmap e82: 12 total, 12 up, 12 in 2024-07-29T00:33:55.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.692+0000 7f2639600640 1 -- 172.21.15.175:0/3694035247 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f262c09b910 con 0x7f263c1016b0 2024-07-29T00:33:55.694 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'cephfs_data' created 2024-07-29T00:33:55.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26200a2c00 msgr2=0x7f26200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:55.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26200a2c00 0x7f26200a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f262402a930 tx=0x7f26240056d0 comp rx=0 tx=0).stop 2024-07-29T00:33:55.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 msgr2=0x7f263c0741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:55.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c0741e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f262c00c880 comp rx=0 tx=0).stop 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 shutdown_connections 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f263c1076b0 0x7f263c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f26200a2c00 0x7f26200a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f263c102050 0x7f263c074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 --2- 172.21.15.175:0/3694035247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f263c1016b0 0x7f263c0741e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.699+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 >> 172.21.15.175:0/3694035247 conn(0x7f263c0fd430 msgr2=0x7f263c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:55.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.700+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 shutdown_connections 2024-07-29T00:33:55.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:55.700+0000 7f2642200640 1 -- 172.21.15.175:0/3694035247 wait complete. 2024-07-29T00:33:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:55 smithi204 ceph-mon[37368]: osdmap e83: 12 total, 12 up, 12 in 2024-07-29T00:33:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:55 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3694035247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-29T00:33:55.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:55 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-29T00:33:55.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:55 smithi181 ceph-mon[36706]: osdmap e83: 12 total, 12 up, 12 in 2024-07-29T00:33:55.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:55 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3694035247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-29T00:33:56.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:55 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-29T00:33:56.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:55 smithi175 ceph-mon[32138]: osdmap e83: 12 total, 12 up, 12 in 2024-07-29T00:33:56.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:55 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3694035247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-29T00:33:56.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:55 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-29T00:33:56.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-29T00:33:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:56 smithi181 ceph-mon[36706]: pgmap v286: 33 pgs: 2 creating+activating, 5 creating+peering, 12 unknown, 14 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:56 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-29T00:33:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:56 smithi181 ceph-mon[36706]: osdmap e84: 12 total, 12 up, 12 in 2024-07-29T00:33:57.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:56 smithi175 ceph-mon[32138]: pgmap v286: 33 pgs: 2 creating+activating, 5 creating+peering, 12 unknown, 14 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:57.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:56 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-29T00:33:57.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:56 smithi175 ceph-mon[32138]: osdmap e84: 12 total, 12 up, 12 in 2024-07-29T00:33:57.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:56 smithi204 ceph-mon[37368]: pgmap v286: 33 pgs: 2 creating+activating, 5 creating+peering, 12 unknown, 14 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:57.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:56 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-29T00:33:57.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:56 smithi204 ceph-mon[37368]: osdmap e84: 12 total, 12 up, 12 in 2024-07-29T00:33:57.112 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:33:57.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:57 smithi175 ceph-mon[32138]: osdmap e85: 12 total, 12 up, 12 in 2024-07-29T00:33:57.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:57 smithi175 ceph-mon[32138]: pgmap v289: 97 pgs: 2 creating+activating, 13 creating+peering, 44 unknown, 38 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:57.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:57 smithi175 ceph-mon[32138]: osdmap e86: 12 total, 12 up, 12 in 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 -- 172.21.15.175:0/2102593731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108140 msgr2=0x7fc998108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 --2- 172.21.15.175:0/2102593731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108140 0x7fc998108520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc988009900 tx=0x7fc98802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 -- 172.21.15.175:0/2102593731 shutdown_connections 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 --2- 172.21.15.175:0/2102593731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc99810d220 0x7fc998114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 --2- 172.21.15.175:0/2102593731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc998108af0 0x7fc99810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 --2- 172.21.15.175:0/2102593731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108140 0x7fc998108520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 -- 172.21.15.175:0/2102593731 >> 172.21.15.175:0/2102593731 conn(0x7fc998077890 msgr2=0x7fc998079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.792+0000 7fc99d000640 1 -- 172.21.15.175:0/2102593731 shutdown_connections 2024-07-29T00:33:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.793+0000 7fc99d000640 1 -- 172.21.15.175:0/2102593731 wait complete. 2024-07-29T00:33:57.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 Processor -- start 2024-07-29T00:33:57.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 -- start start 2024-07-29T00:33:57.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc998108140 0x7fc9981a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:57.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 0x7fc9981a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:57.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99810d220 0x7fc99819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:57.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc998118250 con 0x7fc99810d220 2024-07-29T00:33:57.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.794+0000 7fc99d000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981180d0 con 0x7fc998108af0 2024-07-29T00:33:57.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc99d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981183d0 con 0x7fc998108140 2024-07-29T00:33:57.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 0x7fc9981a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:57.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 0x7fc9981a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55246/0 (socket says 172.21.15.175:55246) 2024-07-29T00:33:57.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 -- 172.21.15.175:0/624265042 learned_addr learned my addr 172.21.15.175:0/624265042 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:33:57.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc995e00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc998108140 0x7fc9981a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:57.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc996800640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99810d220 0x7fc99819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 -- 172.21.15.175:0/624265042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc998108140 msgr2=0x7fc9981a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc998108140 0x7fc9981a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 -- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99810d220 msgr2=0x7fc99819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99810d220 0x7fc99819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc98fe00640 1 -- 172.21.15.175:0/624265042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99819ecd0 con 0x7fc998108af0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.795+0000 7fc996800640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99810d220 0x7fc99819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.796+0000 7fc98fe00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 0x7fc9981a4920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc98000ea60 tx=0x7fc98000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.796+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000cde0 con 0x7fc998108af0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.796+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9800035f0 con 0x7fc998108af0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.796+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99819efc0 con 0x7fc998108af0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.796+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980010670 con 0x7fc998108af0 2024-07-29T00:33:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.796+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981ab2b0 con 0x7fc998108af0 2024-07-29T00:33:57.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.797+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc998108520 con 0x7fc998108af0 2024-07-29T00:33:57.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.799+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc9800037a0 con 0x7fc998108af0 2024-07-29T00:33:57.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.800+0000 7fc98ea00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9780a2c00 0x7fc9780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:33:57.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.800+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7fc9800cf220 con 0x7fc998108af0 2024-07-29T00:33:57.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.801+0000 7fc995e00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9780a2c00 0x7fc9780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:33:57.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.801+0000 7fc995e00640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9780a2c00 0x7fc9780a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc988009230 tx=0x7fc9880058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:33:57.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:57.805+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc98009c290 con 0x7fc998108af0 2024-07-29T00:33:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:57 smithi181 ceph-mon[36706]: osdmap e85: 12 total, 12 up, 12 in 2024-07-29T00:33:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:57 smithi181 ceph-mon[36706]: pgmap v289: 97 pgs: 2 creating+activating, 13 creating+peering, 44 unknown, 38 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:57 smithi181 ceph-mon[36706]: osdmap e86: 12 total, 12 up, 12 in 2024-07-29T00:33:58.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.038+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc99819fad0 con 0x7fc998108af0 2024-07-29T00:33:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:57 smithi204 ceph-mon[37368]: osdmap e85: 12 total, 12 up, 12 in 2024-07-29T00:33:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:57 smithi204 ceph-mon[37368]: pgmap v289: 97 pgs: 2 creating+activating, 13 creating+peering, 44 unknown, 38 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:33:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:57 smithi204 ceph-mon[37368]: osdmap e86: 12 total, 12 up, 12 in 2024-07-29T00:33:58.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.739+0000 7fc98ea00640 1 -- 172.21.15.175:0/624265042 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc98009bb90 con 0x7fc998108af0 2024-07-29T00:33:58.741 INFO:teuthology.orchestra.run.smithi175.stderr:new fs with metadata pool 2 and data pool 3 2024-07-29T00:33:58.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.745+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9780a2c00 msgr2=0x7fc9780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:58.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.745+0000 7fc99d000640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9780a2c00 0x7fc9780a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc988009230 tx=0x7fc9880058d0 comp rx=0 tx=0).stop 2024-07-29T00:33:58.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.745+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 msgr2=0x7fc9981a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:33:58.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.745+0000 7fc99d000640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 0x7fc9981a4920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc98000ea60 tx=0x7fc98000ef20 comp rx=0 tx=0).stop 2024-07-29T00:33:58.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.747+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 shutdown_connections 2024-07-29T00:33:58.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.747+0000 7fc99d000640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99810d220 0x7fc99819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:58.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.747+0000 7fc99d000640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998108af0 0x7fc9981a4920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:58.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.747+0000 7fc99d000640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9780a2c00 0x7fc9780a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:58.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.747+0000 7fc99d000640 1 --2- 172.21.15.175:0/624265042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc998108140 0x7fc9981a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:33:58.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.747+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 >> 172.21.15.175:0/624265042 conn(0x7fc998077890 msgr2=0x7fc998078b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:33:58.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.748+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 shutdown_connections 2024-07-29T00:33:58.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:33:58.748+0000 7fc99d000640 1 -- 172.21.15.175:0/624265042 wait complete. 2024-07-29T00:33:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:58 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/624265042' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-29T00:33:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:58 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-29T00:33:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:58 smithi181 ceph-mon[36706]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-29T00:33:59.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:58 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/624265042' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-29T00:33:59.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:58 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-29T00:33:59.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:58 smithi175 ceph-mon[32138]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-29T00:33:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:58 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/624265042' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-29T00:33:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:58 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-29T00:33:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:58 smithi204 ceph-mon[37368]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-29T00:33:59.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:33:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:59 smithi181 ceph-mon[36706]: osdmap e87: 12 total, 12 up, 12 in 2024-07-29T00:33:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:59 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-29T00:33:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:59 smithi181 ceph-mon[36706]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-29T00:33:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:59 smithi181 ceph-mon[36706]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-29T00:33:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:33:59 smithi181 ceph-mon[36706]: pgmap v292: 97 pgs: 8 creating+peering, 12 unknown, 77 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-29T00:34:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:59 smithi175 ceph-mon[32138]: osdmap e87: 12 total, 12 up, 12 in 2024-07-29T00:34:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:59 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-29T00:34:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:59 smithi175 ceph-mon[32138]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-29T00:34:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:59 smithi175 ceph-mon[32138]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-29T00:34:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:33:59 smithi175 ceph-mon[32138]: pgmap v292: 97 pgs: 8 creating+peering, 12 unknown, 77 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-29T00:34:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:59 smithi204 ceph-mon[37368]: osdmap e87: 12 total, 12 up, 12 in 2024-07-29T00:34:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:59 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-29T00:34:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:59 smithi204 ceph-mon[37368]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-29T00:34:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:59 smithi204 ceph-mon[37368]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-29T00:34:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:33:59 smithi204 ceph-mon[37368]: pgmap v292: 97 pgs: 8 creating+peering, 12 unknown, 77 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-29T00:34:00.174 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 -- 172.21.15.175:0/4051665833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 msgr2=0x7f5290110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 --2- 172.21.15.175:0/4051665833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f5290110770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5288009be0 tx=0x7f528802a6d0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 -- 172.21.15.175:0/4051665833 shutdown_connections 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 --2- 172.21.15.175:0/4051665833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f5290110770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 --2- 172.21.15.175:0/4051665833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52901038a0 0x7f5290103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 --2- 172.21.15.175:0/4051665833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901098a0 0x7f5290109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 -- 172.21.15.175:0/4051665833 >> 172.21.15.175:0/4051665833 conn(0x7f52900ff560 msgr2=0x7f5290101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:00.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 -- 172.21.15.175:0/4051665833 shutdown_connections 2024-07-29T00:34:00.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.775+0000 7f528f200640 1 -- 172.21.15.175:0/4051665833 wait complete. 2024-07-29T00:34:00.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.776+0000 7f528f200640 1 Processor -- start 2024-07-29T00:34:00.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.776+0000 7f528f200640 1 -- start start 2024-07-29T00:34:00.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 0x7f529019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:00.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f529019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:00.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52901098a0 0x7f529019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5290113ca0 con 0x7f52901038a0 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5290113b20 con 0x7f5290104240 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5290113e20 con 0x7f52901098a0 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 0x7f529019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 0x7f529019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34006/0 (socket says 172.21.15.175:34006) 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528de00640 1 -- 172.21.15.175:0/221854021 learned_addr learned my addr 172.21.15.175:0/221854021 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528d400640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f529019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.777+0000 7f528e800640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52901098a0 0x7f529019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528de00640 1 -- 172.21.15.175:0/221854021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52901098a0 msgr2=0x7f529019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528de00640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52901098a0 0x7f529019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528de00640 1 -- 172.21.15.175:0/221854021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 msgr2=0x7f529019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528de00640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f529019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528de00640 1 -- 172.21.15.175:0/221854021 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529019f070 con 0x7f52901038a0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528d400640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f529019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528de00640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 0x7f529019a0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f528000e9c0 tx=0x7f528000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528000ccc0 con 0x7f52901038a0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5280005500 con 0x7f52901038a0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280010430 con 0x7f52901038a0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529019f360 con 0x7f52901038a0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.778+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a6c40 con 0x7f52901038a0 2024-07-29T00:34:00.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.780+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5290104fe0 con 0x7f52901038a0 2024-07-29T00:34:00.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.783+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f52800046c0 con 0x7f52901038a0 2024-07-29T00:34:00.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.784+0000 7f5286a00640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f52740a2b80 0x7f52740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:00.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.784+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f52800cf3f0 con 0x7f52901038a0 2024-07-29T00:34:00.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.785+0000 7f528d400640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f52740a2b80 0x7f52740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:00.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.785+0000 7f528d400640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f52740a2b80 0x7f52740a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5290108e70 tx=0x7f527c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:00.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.787+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5280053df0 con 0x7f52901038a0 2024-07-29T00:34:00.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.977+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5290103d00 con 0x7f52901038a0 2024-07-29T00:34:00.977 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:00 smithi175 ceph-mon[32138]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-29T00:34:00.977 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:00 smithi175 ceph-mon[32138]: Cluster is now healthy 2024-07-29T00:34:00.978 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:00 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] up:active 2024-07-29T00:34:00.978 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:00 smithi175 ceph-mon[32138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:00.978 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:00 smithi175 ceph-mon[32138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:00.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.981+0000 7f5286a00640 1 -- 172.21.15.175:0/221854021 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22263 (secure 0 0 0) 0x7f528009c400 con 0x7f52901038a0 2024-07-29T00:34:00.981 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:00.982 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":87,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:33:58.041543+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f52740a2b80 msgr2=0x7f52740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f52740a2b80 0x7f52740a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5290108e70 tx=0x7f527c005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 msgr2=0x7f529019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 0x7f529019a0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f528000e9c0 tx=0x7f528000ee80 comp rx=0 tx=0).stop 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 shutdown_connections 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f52740a2b80 0x7f52740a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52901098a0 0x7f529019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5290104240 0x7f529019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 --2- 172.21.15.175:0/221854021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52901038a0 0x7f529019a0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 >> 172.21.15.175:0/221854021 conn(0x7f52900ff560 msgr2=0x7f5290100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:00.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 shutdown_connections 2024-07-29T00:34:00.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:00.984+0000 7f528f200640 1 -- 172.21.15.175:0/221854021 wait complete. 2024-07-29T00:34:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:00 smithi181 ceph-mon[36706]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-29T00:34:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:00 smithi181 ceph-mon[36706]: Cluster is now healthy 2024-07-29T00:34:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:00 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] up:active 2024-07-29T00:34:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:00 smithi181 ceph-mon[36706]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:00 smithi181 ceph-mon[36706]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:00 smithi204 ceph-mon[37368]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-29T00:34:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:00 smithi204 ceph-mon[37368]: Cluster is now healthy 2024-07-29T00:34:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:00 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] up:active 2024-07-29T00:34:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:00 smithi204 ceph-mon[37368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:01.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:00 smithi204 ceph-mon[37368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:01 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/221854021' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:01 smithi181 ceph-mon[36706]: pgmap v293: 97 pgs: 97 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-07-29T00:34:02.006 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:01 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/221854021' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:02.006 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:01 smithi175 ceph-mon[32138]: pgmap v293: 97 pgs: 97 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-07-29T00:34:02.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:34:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:01 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/221854021' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:01 smithi204 ceph-mon[37368]: pgmap v293: 97 pgs: 97 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-07-29T00:34:02.510 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:02.999 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:34:02 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: 2024-07-29T00:34:02.647+0000 7f135b65e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 -- 172.21.15.175:0/1433802754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 msgr2=0x7fd450109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 --2- 172.21.15.175:0/1433802754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd450109e70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd444009930 tx=0x7fd44402a420 comp rx=0 tx=0).stop 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 -- 172.21.15.175:0/1433802754 shutdown_connections 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 --2- 172.21.15.175:0/1433802754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450104430 0x7fd450110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 --2- 172.21.15.175:0/1433802754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450103a90 0x7fd450103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 --2- 172.21.15.175:0/1433802754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd450109e70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.185+0000 7fd458800640 1 -- 172.21.15.175:0/1433802754 >> 172.21.15.175:0/1433802754 conn(0x7fd4500ff710 msgr2=0x7fd450101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:03.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.186+0000 7fd458800640 1 -- 172.21.15.175:0/1433802754 shutdown_connections 2024-07-29T00:34:03.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.186+0000 7fd458800640 1 -- 172.21.15.175:0/1433802754 wait complete. 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 Processor -- start 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 -- start start 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450103a90 0x7fd45019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 0x7fd45019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd45019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd450113ee0 con 0x7fd450109a90 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd450113d60 con 0x7fd450103a90 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.187+0000 7fd458800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd450114060 con 0x7fd450104430 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 0x7fd45019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd456800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd45019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450103a90 0x7fd45019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:03.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 0x7fd45019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57398/0 (socket says 172.21.15.175:57398) 2024-07-29T00:34:03.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450103a90 0x7fd45019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51746/0 (socket says 172.21.15.175:51746) 2024-07-29T00:34:03.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 -- 172.21.15.175:0/166272741 learned_addr learned my addr 172.21.15.175:0/166272741 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:03.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd456800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd45019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34012/0 (socket says 172.21.15.175:34012) 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 -- 172.21.15.175:0/166272741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450103a90 msgr2=0x7fd45019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450103a90 0x7fd45019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 -- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 msgr2=0x7fd45019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd45019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 -- 172.21.15.175:0/166272741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45019f330 con 0x7fd450104430 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.188+0000 7fd455400640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 0x7fd45019a8c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd44000ea60 tx=0x7fd44000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.189+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44000cde0 con 0x7fd450104430 2024-07-29T00:34:03.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.189+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4400035f0 con 0x7fd450104430 2024-07-29T00:34:03.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.189+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45019f620 con 0x7fd450104430 2024-07-29T00:34:03.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.189+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440010670 con 0x7fd450104430 2024-07-29T00:34:03.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.189+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501a6f00 con 0x7fd450104430 2024-07-29T00:34:03.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.190+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd450103ef0 con 0x7fd450104430 2024-07-29T00:34:03.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.192+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd4400037a0 con 0x7fd450104430 2024-07-29T00:34:03.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.194+0000 7fd44aa00640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd4380a2b80 0x7fd4380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:03.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.194+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd4400cf210 con 0x7fd450104430 2024-07-29T00:34:03.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.194+0000 7fd455e00640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd4380a2b80 0x7fd4380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:03.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.195+0000 7fd455e00640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd4380a2b80 0x7fd4380a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd444006010 tx=0x7fd444000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:03.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.197+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd44009c220 con 0x7fd450104430 2024-07-29T00:34:03.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.386+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd45019b6e0 con 0x7fd450104430 2024-07-29T00:34:03.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.388+0000 7fd44aa00640 1 -- 172.21.15.175:0/166272741 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22263 (secure 0 0 0) 0x7fd45019b6e0 con 0x7fd450104430 2024-07-29T00:34:03.389 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:03.389 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":87,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:33:58.041543+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:34:03.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd4380a2b80 msgr2=0x7fd4380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:03.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd4380a2b80 0x7fd4380a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd444006010 tx=0x7fd444000bc0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 msgr2=0x7fd45019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:03.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 0x7fd45019a8c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd44000ea60 tx=0x7fd44000ef20 comp rx=0 tx=0).stop 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 shutdown_connections 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd450109a90 0x7fd45019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd4380a2b80 0x7fd4380a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd450104430 0x7fd45019a8c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 --2- 172.21.15.175:0/166272741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd450103a90 0x7fd45019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 >> 172.21.15.175:0/166272741 conn(0x7fd4500ff710 msgr2=0x7fd450100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 shutdown_connections 2024-07-29T00:34:03.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:03.392+0000 7fd458800640 1 -- 172.21.15.175:0/166272741 wait complete. 2024-07-29T00:34:03.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:03 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/166272741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:03 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/166272741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:03.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:03 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/166272741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:04.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-07-29T00:34:04.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:04 smithi175 ceph-mon[32138]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-29T00:34:04.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:04 smithi181 ceph-mon[36706]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-29T00:34:04.796 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:04.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:04 smithi204 ceph-mon[37368]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-29T00:34:05.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 -- 172.21.15.175:0/2571241871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 msgr2=0x7fafc4110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:05.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 --2- 172.21.15.175:0/2571241871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc4110950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009cd0 tx=0x7fafbc02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 -- 172.21.15.175:0/2571241871 shutdown_connections 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 --2- 172.21.15.175:0/2571241871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc4110950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 --2- 172.21.15.175:0/2571241871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 0x7fafc4103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 --2- 172.21.15.175:0/2571241871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc4109a80 0x7fafc4109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 -- 172.21.15.175:0/2571241871 >> 172.21.15.175:0/2571241871 conn(0x7fafc40ff760 msgr2=0x7fafc4101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 -- 172.21.15.175:0/2571241871 shutdown_connections 2024-07-29T00:34:05.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.455+0000 7fafc9800640 1 -- 172.21.15.175:0/2571241871 wait complete. 2024-07-29T00:34:05.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.456+0000 7fafc9800640 1 Processor -- start 2024-07-29T00:34:05.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.456+0000 7fafc9800640 1 -- start start 2024-07-29T00:34:05.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 0x7fafc419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:05.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:05.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc9800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc4109a80 0x7fafc419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:05.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4113ed0 con 0x7fafc4104420 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc9800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4113d50 con 0x7fafc4103a80 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc9800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4114050 con 0x7fafc4109a80 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafba000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 0x7fafc419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafba000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34038/0 (socket says 172.21.15.175:34038) 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafba000640 1 -- 172.21.15.175:0/2242478093 learned_addr learned my addr 172.21.15.175:0/2242478093 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:05.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc3400640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc4109a80 0x7fafc419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:05.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc2a00640 1 -- 172.21.15.175:0/2242478093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc4109a80 msgr2=0x7fafc419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:05.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc2a00640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc4109a80 0x7fafc419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:05.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.457+0000 7fafc2a00640 1 -- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 msgr2=0x7fafc419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:05.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafc2a00640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafc2a00640 1 -- 172.21.15.175:0/2242478093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc419f300 con 0x7fafc4103a80 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafba000640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafc2a00640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 0x7fafc419a350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb400ccc0 con 0x7fafc4103a80 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb4005020 con 0x7fafc4103a80 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc419f5f0 con 0x7fafc4103a80 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40049e0 con 0x7fafc4103a80 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.458+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc41a6ed0 con 0x7fafc4103a80 2024-07-29T00:34:05.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.460+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc4103ee0 con 0x7fafc4103a80 2024-07-29T00:34:05.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.462+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fafb4005560 con 0x7fafc4103a80 2024-07-29T00:34:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.463+0000 7fafbbe00640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fafa80a2b80 0x7fafa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.463+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fafb40cee60 con 0x7fafc4103a80 2024-07-29T00:34:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.463+0000 7fafba000640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fafa80a2b80 0x7fafa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.464+0000 7fafba000640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fafa80a2b80 0x7fafa80a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafc4101d40 tx=0x7fafac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:05.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.465+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fafb40051d0 con 0x7fafc4103a80 2024-07-29T00:34:05.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:05.692+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fafc410c050 con 0x7fafc4103a80 2024-07-29T00:34:06.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.607+0000 7fafbbe00640 1 -- 172.21.15.175:0/2242478093 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fafb409be70 con 0x7fafc4103a80 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fafa80a2b80 msgr2=0x7fafa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fafa80a2b80 0x7fafa80a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafc4101d40 tx=0x7fafac005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 msgr2=0x7fafc419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 0x7fafc419a350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 shutdown_connections 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fafa80a2b80 0x7fafa80a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc4109a80 0x7fafc419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:06.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc4104420 0x7fafc419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 --2- 172.21.15.175:0/2242478093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc4103a80 0x7fafc419a350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.610+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 >> 172.21.15.175:0/2242478093 conn(0x7fafc40ff760 msgr2=0x7fafc40ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.611+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 shutdown_connections 2024-07-29T00:34:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:06.611+0000 7fafc9800640 1 -- 172.21.15.175:0/2242478093 wait complete. 2024-07-29T00:34:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:06 smithi181 ceph-mon[36706]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-29T00:34:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:06 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2242478093' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-29T00:34:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:06 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-29T00:34:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:06 smithi175 ceph-mon[32138]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-29T00:34:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:06 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2242478093' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-29T00:34:07.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:06 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-29T00:34:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:06 smithi204 ceph-mon[37368]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-29T00:34:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:06 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2242478093' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-29T00:34:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:06 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-29T00:34:07.603 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-07-29T00:34:07.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-07-29T00:34:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:07 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-29T00:34:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:07 smithi181 ceph-mon[36706]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:08.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:07 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-29T00:34:08.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:07 smithi175 ceph-mon[32138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:08.036 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:07 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-29T00:34:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:07 smithi204 ceph-mon[37368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 -- 172.21.15.175:0/3842738411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 msgr2=0x7f248c109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 --2- 172.21.15.175:0/3842738411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 0x7f248c109c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f247c009930 tx=0x7f247c02a420 comp rx=0 tx=0).stop 2024-07-29T00:34:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 -- 172.21.15.175:0/3842738411 shutdown_connections 2024-07-29T00:34:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 --2- 172.21.15.175:0/3842738411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f248c104240 0x7f248c110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 --2- 172.21.15.175:0/3842738411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 --2- 172.21.15.175:0/3842738411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 0x7f248c109c80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 -- 172.21.15.175:0/3842738411 >> 172.21.15.175:0/3842738411 conn(0x7f248c0ff560 msgr2=0x7f248c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 -- 172.21.15.175:0/3842738411 shutdown_connections 2024-07-29T00:34:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.691+0000 7f2490a00640 1 -- 172.21.15.175:0/3842738411 wait complete. 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.692+0000 7f2490a00640 1 Processor -- start 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.692+0000 7f2490a00640 1 -- start start 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2490a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2490a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f248c104240 0x7f248c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2490a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 0x7f248c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2490a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f248c113c60 con 0x7f248c1098a0 2024-07-29T00:34:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2490a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f248c113ae0 con 0x7f248c1038a0 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2490a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f248c113de0 con 0x7f248c104240 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f248b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 0x7f248c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f248a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f2481e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f248c104240 0x7f248c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f248a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51786/0 (socket says 172.21.15.175:51786) 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.693+0000 7f248a800640 1 -- 172.21.15.175:0/2967397903 learned_addr learned my addr 172.21.15.175:0/2967397903 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f248a800640 1 -- 172.21.15.175:0/2967397903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f248c104240 msgr2=0x7f248c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f248a800640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f248c104240 0x7f248c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f248a800640 1 -- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 msgr2=0x7f248c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:08.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f248a800640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 0x7f248c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f248a800640 1 -- 172.21.15.175:0/2967397903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c19f150 con 0x7f248c1038a0 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f248a800640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c19a1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f247c009230 tx=0x7f247c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c03f030 con 0x7f248c1038a0 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.694+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c19f3e0 con 0x7f248c1038a0 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.695+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1a6d20 con 0x7f248c1038a0 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.695+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f247c004440 con 0x7f248c1038a0 2024-07-29T00:34:08.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.695+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c049060 con 0x7f248c1038a0 2024-07-29T00:34:08.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.696+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c103d00 con 0x7f248c1038a0 2024-07-29T00:34:08.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.697+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f247c042030 con 0x7f248c1038a0 2024-07-29T00:34:08.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.699+0000 7f2483e00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24700a2c00 0x7f24700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:08.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.699+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f247c0ef210 con 0x7f248c1038a0 2024-07-29T00:34:08.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.699+0000 7f2481e00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24700a2c00 0x7f24700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:08.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.700+0000 7f2481e00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24700a2c00 0x7f24700a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f248c101bd0 tx=0x7f2474005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:08.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.703+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f247c0bbbd0 con 0x7f248c1038a0 2024-07-29T00:34:08.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:08.932+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f248c10be10 con 0x7f248c1038a0 2024-07-29T00:34:08.932 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:08 smithi175 ceph-mon[32138]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-29T00:34:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:08 smithi181 ceph-mon[36706]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-29T00:34:09.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:08 smithi204 ceph-mon[37368]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-29T00:34:09.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.629+0000 7f2483e00640 1 -- 172.21.15.175:0/2967397903 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f247c0bfed0 con 0x7f248c1038a0 2024-07-29T00:34:09.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.632+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24700a2c00 msgr2=0x7f24700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:09.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.632+0000 7f2490a00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24700a2c00 0x7f24700a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f248c101bd0 tx=0x7f2474005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:09.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.632+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 msgr2=0x7f248c19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:09.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.632+0000 7f2490a00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c19a1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f247c009230 tx=0x7f247c004040 comp rx=0 tx=0).stop 2024-07-29T00:34:09.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 shutdown_connections 2024-07-29T00:34:09.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248c1098a0 0x7f248c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:09.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24700a2c00 0x7f24700a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:09.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f248c104240 0x7f248c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:09.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 --2- 172.21.15.175:0/2967397903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f248c1038a0 0x7f248c19a1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:09.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 >> 172.21.15.175:0/2967397903 conn(0x7f248c0ff560 msgr2=0x7f248c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:09.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 shutdown_connections 2024-07-29T00:34:09.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:09.633+0000 7f2490a00640 1 -- 172.21.15.175:0/2967397903 wait complete. 2024-07-29T00:34:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:09 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2967397903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-29T00:34:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:09 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-29T00:34:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:09 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2967397903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-29T00:34:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:09 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-29T00:34:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:09 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2967397903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-29T00:34:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:09 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-29T00:34:10.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-07-29T00:34:10.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-29T00:34:10.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-29T00:34:10.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-29T00:34:10.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-29T00:34:10.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:10.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-07-29T00:34:10.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:10 smithi175 ceph-mon[32138]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-07-29T00:34:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-29T00:34:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-29T00:34:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-29T00:34:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-29T00:34:11.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:11.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-07-29T00:34:11.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:10 smithi181 ceph-mon[36706]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-07-29T00:34:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-29T00:34:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-29T00:34:11.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-29T00:34:11.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-29T00:34:11.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-29T00:34:11.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-07-29T00:34:11.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:10 smithi204 ceph-mon[37368]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-07-29T00:34:11.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 -- 172.21.15.175:0/2013172528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 msgr2=0x7f1450103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:11.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 --2- 172.21.15.175:0/2013172528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f1450103d00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f143c009930 tx=0x7f143c02a420 comp rx=0 tx=0).stop 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 -- 172.21.15.175:0/2013172528 shutdown_connections 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 --2- 172.21.15.175:0/2013172528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1450104240 0x7f1450110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 --2- 172.21.15.175:0/2013172528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f1450103d00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 --2- 172.21.15.175:0/2013172528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14501098a0 0x7f1450109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 -- 172.21.15.175:0/2013172528 >> 172.21.15.175:0/2013172528 conn(0x7f14500ff560 msgr2=0x7f1450101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.869+0000 7f1456e00640 1 -- 172.21.15.175:0/2013172528 shutdown_connections 2024-07-29T00:34:11.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.870+0000 7f1456e00640 1 -- 172.21.15.175:0/2013172528 wait complete. 2024-07-29T00:34:11.871 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:11 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] up:active 2024-07-29T00:34:11.871 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:11 smithi175 ceph-mon[32138]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-29T00:34:11.871 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:11 smithi175 ceph-mon[32138]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-07-29T00:34:11.872 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:11 smithi175 ceph-mon[32138]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-07-29T00:34:11.872 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:11 smithi175 ceph-mon[32138]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-07-29T00:34:11.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.871+0000 7f1456e00640 1 Processor -- start 2024-07-29T00:34:11.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.871+0000 7f1456e00640 1 -- start start 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.871+0000 7f1456e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f145019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1456e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1450104240 0x7f145019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1456e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14501098a0 0x7f145019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1456e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113cf0 con 0x7f14501038a0 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1456e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113b70 con 0x7f1450104240 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1456e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113e70 con 0x7f14501098a0 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1455a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f145019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1455000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1450104240 0x7f145019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1455a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f145019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45734/0 (socket says 172.21.15.175:45734) 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1455a00640 1 -- 172.21.15.175:0/257692937 learned_addr learned my addr 172.21.15.175:0/257692937 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:11.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.872+0000 7f1456400640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14501098a0 0x7f145019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455a00640 1 -- 172.21.15.175:0/257692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14501098a0 msgr2=0x7f145019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455a00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14501098a0 0x7f145019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455a00640 1 -- 172.21.15.175:0/257692937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1450104240 msgr2=0x7f145019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455a00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1450104240 0x7f145019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455a00640 1 -- 172.21.15.175:0/257692937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145019f160 con 0x7f14501038a0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455000640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1450104240 0x7f145019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1455a00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f145019a1b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f144400e9f0 tx=0x7f144400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144400cd80 con 0x7f14501038a0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1444010910 con 0x7f14501038a0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145019f450 con 0x7f14501038a0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444018990 con 0x7f14501038a0 2024-07-29T00:34:11.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.873+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14501a6d30 con 0x7f14501038a0 2024-07-29T00:34:11.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.875+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1450104fe0 con 0x7f14501038a0 2024-07-29T00:34:11.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.877+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1444010ac0 con 0x7f14501038a0 2024-07-29T00:34:11.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.878+0000 7f144e000640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f14380a2c00 0x7f14380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:11.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.878+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f14440cf160 con 0x7f14501038a0 2024-07-29T00:34:11.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.879+0000 7f1455000640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f14380a2c00 0x7f14380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:11.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.879+0000 7f1455000640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f14380a2c00 0x7f14380a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f143c006010 tx=0x7f143c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:11.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:11.883+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f144409bb20 con 0x7f14501038a0 2024-07-29T00:34:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:11 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] up:active 2024-07-29T00:34:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:11 smithi181 ceph-mon[36706]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-29T00:34:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:11 smithi181 ceph-mon[36706]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-07-29T00:34:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:11 smithi181 ceph-mon[36706]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-07-29T00:34:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:11 smithi181 ceph-mon[36706]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-07-29T00:34:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:11 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] up:active 2024-07-29T00:34:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:11 smithi204 ceph-mon[37368]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-29T00:34:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:11 smithi204 ceph-mon[37368]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-07-29T00:34:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:11 smithi204 ceph-mon[37368]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-07-29T00:34:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:11 smithi204 ceph-mon[37368]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-07-29T00:34:12.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.111+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f145010be90 con 0x7f14501038a0 2024-07-29T00:34:12.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.663+0000 7f144e000640 1 -- 172.21.15.175:0/257692937 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f144409fe20 con 0x7f14501038a0 2024-07-29T00:34:12.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f14380a2c00 msgr2=0x7f14380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:12.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f14380a2c00 0x7f14380a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f143c006010 tx=0x7f143c031510 comp rx=0 tx=0).stop 2024-07-29T00:34:12.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 msgr2=0x7f145019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:12.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f145019a1b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f144400e9f0 tx=0x7f144400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:34:12.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 shutdown_connections 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f14380a2c00 0x7f14380a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14501098a0 0x7f145019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1450104240 0x7f145019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.667+0000 7f1456e00640 1 --2- 172.21.15.175:0/257692937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14501038a0 0x7f145019a1b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.668+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 >> 172.21.15.175:0/257692937 conn(0x7f14500ff560 msgr2=0x7f14500ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.668+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 shutdown_connections 2024-07-29T00:34:12.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:12.668+0000 7f1456e00640 1 -- 172.21.15.175:0/257692937 wait complete. 2024-07-29T00:34:12.999 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: 2024-07-29T00:34:12.648+0000 7f135b65e640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-07-29T00:34:12.999 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: 2024-07-29T00:34:12.648+0000 7f135b65e640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] up:active 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-07-29T00:34:13.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:12 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/257692937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-29T00:34:13.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-07-29T00:34:13.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] up:active 2024-07-29T00:34:13.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-29T00:34:13.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-07-29T00:34:13.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-07-29T00:34:13.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-07-29T00:34:13.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:12 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/257692937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-29T00:34:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-07-29T00:34:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] up:active 2024-07-29T00:34:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-29T00:34:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-07-29T00:34:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-07-29T00:34:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-07-29T00:34:13.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:12 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/257692937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-29T00:34:13.560 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-07-29T00:34:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/257692937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-29T00:34:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] up:active 2024-07-29T00:34:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-29T00:34:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-29T00:34:13.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: Cluster is now healthy 2024-07-29T00:34:13.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-07-29T00:34:13.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-29T00:34:13.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:13 smithi175 ceph-mon[32138]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-07-29T00:34:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/257692937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-29T00:34:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] up:active 2024-07-29T00:34:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-29T00:34:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-29T00:34:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: Cluster is now healthy 2024-07-29T00:34:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-07-29T00:34:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-29T00:34:14.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:13 smithi181 ceph-mon[36706]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-07-29T00:34:14.016 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/257692937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-29T00:34:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] up:active 2024-07-29T00:34:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-29T00:34:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-29T00:34:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: Cluster is now healthy 2024-07-29T00:34:14.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-07-29T00:34:14.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-29T00:34:14.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:13 smithi204 ceph-mon[37368]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 -- 172.21.15.175:0/4062082081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1098a0 msgr2=0x7f5dcc109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 --2- 172.21.15.175:0/4062082081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc109c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009930 tx=0x7f5dc002a420 comp rx=0 tx=0).stop 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 -- 172.21.15.175:0/4062082081 shutdown_connections 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 --2- 172.21.15.175:0/4062082081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dcc104240 0x7f5dcc110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 --2- 172.21.15.175:0/4062082081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 --2- 172.21.15.175:0/4062082081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc109c80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 -- 172.21.15.175:0/4062082081 >> 172.21.15.175:0/4062082081 conn(0x7f5dcc0ff560 msgr2=0x7f5dcc101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:14.709 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:14 smithi175 ceph-mon[32138]: pgmap v299: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-07-29T00:34:14.709 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:14 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] up:active 2024-07-29T00:34:14.709 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:14 smithi175 ceph-mon[32138]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-29T00:34:14.709 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:14 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:34:14.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.707+0000 7f5dd2600640 1 -- 172.21.15.175:0/4062082081 shutdown_connections 2024-07-29T00:34:14.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.708+0000 7f5dd2600640 1 -- 172.21.15.175:0/4062082081 wait complete. 2024-07-29T00:34:14.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 Processor -- start 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 -- start start 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dcc104240 0x7f5dcc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113cb0 con 0x7f5dcc1038a0 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113b30 con 0x7f5dcc1098a0 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd2600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113e30 con 0x7f5dcc104240 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:14.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45754/0 (socket says 172.21.15.175:45754) 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.709+0000 7f5dd1200640 1 -- 172.21.15.175:0/87236885 learned_addr learned my addr 172.21.15.175:0/87236885 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dcbe00640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dcc104240 0x7f5dcc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1c00640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1200640 1 -- 172.21.15.175:0/87236885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dcc104240 msgr2=0x7f5dcc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1200640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dcc104240 0x7f5dcc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1200640 1 -- 172.21.15.175:0/87236885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1098a0 msgr2=0x7f5dcc19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1200640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1200640 1 -- 172.21.15.175:0/87236885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc19f1c0 con 0x7f5dcc1038a0 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1c00640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:34:14.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.710+0000 7f5dd1200640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc19a210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5dc002a940 tx=0x7f5dc0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:14.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.711+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc003f070 con 0x7f5dcc1038a0 2024-07-29T00:34:14.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.711+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc0031c90 con 0x7f5dcc1038a0 2024-07-29T00:34:14.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.711+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0035030 con 0x7f5dcc1038a0 2024-07-29T00:34:14.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.711+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc19f450 con 0x7f5dcc1038a0 2024-07-29T00:34:14.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.711+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc19f8e0 con 0x7f5dcc1038a0 2024-07-29T00:34:14.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.712+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc104fe0 con 0x7f5dcc1038a0 2024-07-29T00:34:14.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.715+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5dc0042070 con 0x7f5dcc1038a0 2024-07-29T00:34:14.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.717+0000 7f5dc9600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5db40a2c00 0x7f5db40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:14.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.717+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f5dc00ef9a0 con 0x7f5dcc1038a0 2024-07-29T00:34:14.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.717+0000 7f5dcbe00640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5db40a2c00 0x7f5db40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:14.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.718+0000 7f5dcbe00640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5db40a2c00 0x7f5db40a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc101c40 tx=0x7f5db8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:14.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.719+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5dc00bc9b0 con 0x7f5dcc1038a0 2024-07-29T00:34:14.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:14.950+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5dcc10be40 con 0x7f5dcc1038a0 2024-07-29T00:34:14.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:14 smithi204 ceph-mon[37368]: pgmap v299: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-07-29T00:34:14.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:14 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] up:active 2024-07-29T00:34:14.987 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:14 smithi204 ceph-mon[37368]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-29T00:34:14.987 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:14 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:34:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:14 smithi181 ceph-mon[36706]: pgmap v299: 97 pgs: 97 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-07-29T00:34:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:14 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] up:active 2024-07-29T00:34:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:14 smithi181 ceph-mon[36706]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-29T00:34:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:14 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:34:15.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.691+0000 7f5dc9600640 1 -- 172.21.15.175:0/87236885 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f5dc00b6e20 con 0x7f5dcc1038a0 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5db40a2c00 msgr2=0x7f5db40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5db40a2c00 0x7f5db40a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc101c40 tx=0x7f5db8005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 msgr2=0x7f5dcc19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc19a210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5dc002a940 tx=0x7f5dc0003b90 comp rx=0 tx=0).stop 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 shutdown_connections 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dcc1098a0 0x7f5dcc19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5db40a2c00 0x7f5db40a50c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dcc104240 0x7f5dcc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 --2- 172.21.15.175:0/87236885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dcc1038a0 0x7f5dcc19a210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:15.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.695+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 >> 172.21.15.175:0/87236885 conn(0x7f5dcc0ff560 msgr2=0x7f5dcc0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:15.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.696+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 shutdown_connections 2024-07-29T00:34:15.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:15.696+0000 7f5dd2600640 1 -- 172.21.15.175:0/87236885 wait complete. 2024-07-29T00:34:15.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:15 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/87236885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-29T00:34:16.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:15 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/87236885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-29T00:34:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:15 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/87236885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-29T00:34:16.613 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-07-29T00:34:16.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-07-29T00:34:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:16 smithi181 ceph-mon[36706]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-07-29T00:34:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:16 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/87236885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-29T00:34:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:16 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:34:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:16 smithi175 ceph-mon[32138]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-07-29T00:34:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:16 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/87236885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-29T00:34:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:16 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:34:17.082 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:16 smithi204 ceph-mon[37368]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-07-29T00:34:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:16 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/87236885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-29T00:34:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:16 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:34:17.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 -- 172.21.15.175:0/4021284907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4109a80 msgr2=0x7f68d4109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:17.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 --2- 172.21.15.175:0/4021284907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4109a80 0x7f68d4109e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009cd0 tx=0x7f68cc02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:34:17.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 -- 172.21.15.175:0/4021284907 shutdown_connections 2024-07-29T00:34:17.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 --2- 172.21.15.175:0/4021284907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 0x7f68d4110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:17.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 --2- 172.21.15.175:0/4021284907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d4103a80 0x7f68d4103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 --2- 172.21.15.175:0/4021284907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4109a80 0x7f68d4109e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.895+0000 7f68d9200640 1 -- 172.21.15.175:0/4021284907 >> 172.21.15.175:0/4021284907 conn(0x7f68d40ff780 msgr2=0x7f68d4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.896+0000 7f68d9200640 1 -- 172.21.15.175:0/4021284907 shutdown_connections 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.896+0000 7f68d9200640 1 -- 172.21.15.175:0/4021284907 wait complete. 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 Processor -- start 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 -- start start 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 0x7f68d419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 0x7f68d419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d4109a80 0x7f68d419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4113e80 con 0x7f68d4103a80 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4113d00 con 0x7f68d4104420 2024-07-29T00:34:17.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d9200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4114000 con 0x7f68d4109a80 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 0x7f68d419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.897+0000 7f68d1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 0x7f68d419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45776/0 (socket says 172.21.15.175:45776) 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 -- 172.21.15.175:0/1507692589 learned_addr learned my addr 172.21.15.175:0/1507692589 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1400640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 0x7f68d419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d2800640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d4109a80 0x7f68d419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 -- 172.21.15.175:0/1507692589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d4109a80 msgr2=0x7f68d419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d4109a80 0x7f68d419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 -- 172.21.15.175:0/1507692589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 msgr2=0x7f68d419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 0x7f68d419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 -- 172.21.15.175:0/1507692589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d419f2f0 con 0x7f68d4103a80 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1400640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 0x7f68d419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.898+0000 7f68d1e00640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 0x7f68d419a340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02ae00 tx=0x7f68cc031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.899+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc003e00 con 0x7f68d4103a80 2024-07-29T00:34:17.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.899+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68cc004520 con 0x7f68d4103a80 2024-07-29T00:34:17.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.899+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d419f580 con 0x7f68d4103a80 2024-07-29T00:34:17.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.899+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc030de0 con 0x7f68d4103a80 2024-07-29T00:34:17.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.899+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d41a6ec0 con 0x7f68d4103a80 2024-07-29T00:34:17.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.900+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d4103ee0 con 0x7f68d4103a80 2024-07-29T00:34:17.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.902+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f68cc0046d0 con 0x7f68d4103a80 2024-07-29T00:34:17.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.903+0000 7f68caa00640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f68b80a2c00 0x7f68b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:17.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.904+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f68cc02e030 con 0x7f68d4103a80 2024-07-29T00:34:17.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.904+0000 7f68d1400640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f68b80a2c00 0x7f68b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:17.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.904+0000 7f68d1400640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f68b80a2c00 0x7f68b80a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68d41090d0 tx=0x7f68c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:17.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:17.907+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f68cc04b060 con 0x7f68d4103a80 2024-07-29T00:34:18.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.124+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f68d4062320 con 0x7f68b80a2c00 2024-07-29T00:34:18.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.155+0000 7f68caa00640 1 -- 172.21.15.175:0/1507692589 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68d4062320 con 0x7f68b80a2c00 2024-07-29T00:34:18.156 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:34:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:34:18.132+0000 7f1d33c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:34:18.156 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:34:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:34:18.132+0000 7f1d33c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:34:18.156 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:34:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:34:18.132+0000 7f1d33c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:34:18.156 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:34:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:34:18.132+0000 7f1d33c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:34:18.156 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:34:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:34:18.132+0000 7f1d33c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:34:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f68b80a2c00 msgr2=0x7f68b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f68b80a2c00 0x7f68b80a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68d41090d0 tx=0x7f68c0005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 msgr2=0x7f68d419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 0x7f68d419a340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02ae00 tx=0x7f68cc031e60 comp rx=0 tx=0).stop 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 shutdown_connections 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f68b80a2c00 0x7f68b80a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d4109a80 0x7f68d419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68d4104420 0x7f68d419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 --2- 172.21.15.175:0/1507692589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d4103a80 0x7f68d419a340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 >> 172.21.15.175:0/1507692589 conn(0x7f68d40ff780 msgr2=0x7f68d4101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 shutdown_connections 2024-07-29T00:34:18.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:18.158+0000 7f68d9200640 1 -- 172.21.15.175:0/1507692589 wait complete. 2024-07-29T00:34:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:18 smithi181 ceph-mon[36706]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-07-29T00:34:19.005 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:18 smithi175 ceph-mon[32138]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-07-29T00:34:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:18 smithi204 ceph-mon[37368]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-07-29T00:34:19.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-07-29T00:34:19.557 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:19 smithi181 ceph-mon[36706]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:19 smithi175 ceph-mon[32138]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:19 smithi204 ceph-mon[37368]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 -- 172.21.15.175:0/106826231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94074bd0 msgr2=0x7f9e9410da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 --2- 172.21.15.175:0/106826231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94074bd0 0x7f9e9410da00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009900 tx=0x7f9e8002a480 comp rx=0 tx=0).stop 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 -- 172.21.15.175:0/106826231 shutdown_connections 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 --2- 172.21.15.175:0/106826231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9410df40 0x7f9e94110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 --2- 172.21.15.175:0/106826231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e94074bd0 0x7f9e9410da00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 --2- 172.21.15.175:0/106826231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e940742b0 0x7f9e94074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 -- 172.21.15.175:0/106826231 >> 172.21.15.175:0/106826231 conn(0x7f9e940fd3a0 msgr2=0x7f9e940ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 -- 172.21.15.175:0/106826231 shutdown_connections 2024-07-29T00:34:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.181+0000 7f9e9b000640 1 -- 172.21.15.175:0/106826231 wait complete. 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.182+0000 7f9e9b000640 1 Processor -- start 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.182+0000 7f9e9b000640 1 -- start start 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 0x7f9e9419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9b000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94074bd0 0x7f9e9419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9410df40 0x7f9e9419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94113ce0 con 0x7f9e940742b0 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9b000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94113b60 con 0x7f9e94074bd0 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94113e60 con 0x7f9e9410df40 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 0x7f9e9419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 0x7f9e9419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40758/0 (socket says 172.21.15.175:40758) 2024-07-29T00:34:20.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 -- 172.21.15.175:0/391755579 learned_addr learned my addr 172.21.15.175:0/391755579 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99200640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94074bd0 0x7f9e9419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e9a600640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9410df40 0x7f9e9419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 -- 172.21.15.175:0/391755579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9410df40 msgr2=0x7f9e9419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9410df40 0x7f9e9419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 -- 172.21.15.175:0/391755579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94074bd0 msgr2=0x7f9e9419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94074bd0 0x7f9e9419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.183+0000 7f9e99c00640 1 -- 172.21.15.175:0/391755579 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9419f170 con 0x7f9e940742b0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.184+0000 7f9e99c00640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 0x7f9e9419a1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800e9f0 tx=0x7f9e8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.184+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800ccc0 con 0x7f9e940742b0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.184+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88010910 con 0x7f9e940742b0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.184+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9419f460 con 0x7f9e940742b0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.184+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880188a0 con 0x7f9e940742b0 2024-07-29T00:34:20.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.184+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e941a6d40 con 0x7f9e940742b0 2024-07-29T00:34:20.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.185+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e94074690 con 0x7f9e940742b0 2024-07-29T00:34:20.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.187+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9e88010ac0 con 0x7f9e940742b0 2024-07-29T00:34:20.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.189+0000 7f9e92000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9e7c0a2c10 0x7f9e7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:20.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.189+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f9e880cf490 con 0x7f9e940742b0 2024-07-29T00:34:20.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.189+0000 7f9e99200640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9e7c0a2c10 0x7f9e7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:20.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.190+0000 7f9e99200640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9e7c0a2c10 0x7f9e7c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e80000c00 tx=0x7f9e80005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:20.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.193+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9e8809c3c0 con 0x7f9e940742b0 2024-07-29T00:34:20.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.411+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9e94062320 con 0x7f9e7c0a2c10 2024-07-29T00:34:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.450+0000 7f9e92000640 1 -- 172.21.15.175:0/391755579 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9e94062320 con 0x7f9e7c0a2c10 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.452+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9e7c0a2c10 msgr2=0x7f9e7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.452+0000 7f9e9b000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9e7c0a2c10 0x7f9e7c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e80000c00 tx=0x7f9e80005b50 comp rx=0 tx=0).stop 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.452+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 msgr2=0x7f9e9419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.452+0000 7f9e9b000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 0x7f9e9419a1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800e9f0 tx=0x7f9e8800edf0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 shutdown_connections 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9e7c0a2c10 0x7f9e7c0a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9410df40 0x7f9e9419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94074bd0 0x7f9e9419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 --2- 172.21.15.175:0/391755579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e940742b0 0x7f9e9419a1c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:20.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 >> 172.21.15.175:0/391755579 conn(0x7f9e940fd3a0 msgr2=0x7f9e940fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:20.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 shutdown_connections 2024-07-29T00:34:20.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:20.453+0000 7f9e9b000640 1 -- 172.21.15.175:0/391755579 wait complete. 2024-07-29T00:34:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:20 smithi181 ceph-mon[36706]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-07-29T00:34:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:20 smithi181 ceph-mon[36706]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-07-29T00:34:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:20 smithi181 ceph-mon[36706]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:21.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:20 smithi175 ceph-mon[32138]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-07-29T00:34:21.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:20 smithi175 ceph-mon[32138]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-07-29T00:34:21.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:20 smithi175 ceph-mon[32138]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:20 smithi204 ceph-mon[37368]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-07-29T00:34:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:20 smithi204 ceph-mon[37368]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-07-29T00:34:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:20 smithi204 ceph-mon[37368]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:21.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-07-29T00:34:21.851 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:21 smithi181 ceph-mon[36706]: pgmap v303: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2024-07-29T00:34:22.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:21 smithi175 ceph-mon[32138]: pgmap v303: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2024-07-29T00:34:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:21 smithi204 ceph-mon[37368]: pgmap v303: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 -- 172.21.15.175:0/3147633605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510102a20 msgr2=0x7f0510102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 --2- 172.21.15.175:0/3147633605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510102a20 0x7f0510102e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0504009900 tx=0x7f050402a480 comp rx=0 tx=0).stop 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 -- 172.21.15.175:0/3147633605 shutdown_connections 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 --2- 172.21.15.175:0/3147633605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051010be50 0x7f051010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 --2- 172.21.15.175:0/3147633605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0510103340 0x7f051010b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 --2- 172.21.15.175:0/3147633605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510102a20 0x7f0510102e00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.544+0000 7f0517e00640 1 -- 172.21.15.175:0/3147633605 >> 172.21.15.175:0/3147633605 conn(0x7f05100fc480 msgr2=0x7f05100fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.545+0000 7f0517e00640 1 -- 172.21.15.175:0/3147633605 shutdown_connections 2024-07-29T00:34:22.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.545+0000 7f0517e00640 1 -- 172.21.15.175:0/3147633605 wait complete. 2024-07-29T00:34:22.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 Processor -- start 2024-07-29T00:34:22.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 -- start start 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0510102a20 0x7f0510198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 0x7f05101986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f051010be50 0x7f051019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0510111ca0 con 0x7f0510102a20 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0510111b20 con 0x7f0510103340 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.546+0000 7f0517e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0510111e20 con 0x7f051010be50 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 0x7f05101986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 0x7f05101986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:44824/0 (socket says 172.21.15.175:44824) 2024-07-29T00:34:22.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 -- 172.21.15.175:0/3430135431 learned_addr learned my addr 172.21.15.175:0/3430135431 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0515400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0510102a20 0x7f0510198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0515e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f051010be50 0x7f051019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 -- 172.21.15.175:0/3430135431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f051010be50 msgr2=0x7f051019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f051010be50 0x7f051019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 -- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0510102a20 msgr2=0x7f0510198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0510102a20 0x7f0510198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 -- 172.21.15.175:0/3430135431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051019d110 con 0x7f0510103340 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.547+0000 7f0514a00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 0x7f05101986a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00e9f0 tx=0x7f04fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.548+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00ccc0 con 0x7f0510103340 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.548+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04fc00ce60 con 0x7f0510103340 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.548+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc010690 con 0x7f0510103340 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.548+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051019d400 con 0x7f0510103340 2024-07-29T00:34:22.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.548+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a4ce0 con 0x7f0510103340 2024-07-29T00:34:22.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.549+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0510102e00 con 0x7f0510103340 2024-07-29T00:34:22.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.551+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f04fc0046c0 con 0x7f0510103340 2024-07-29T00:34:22.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.552+0000 7f050e000640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f04f80a2c90 0x7f04f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:22.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.552+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f04fc0cf400 con 0x7f0510103340 2024-07-29T00:34:22.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.553+0000 7f0515400640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f04f80a2c90 0x7f04f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:22.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.553+0000 7f0515400640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f04f80a2c90 0x7f04f80a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0504000c00 tx=0x7f0504005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:22.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.556+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f04fc09c2b0 con 0x7f0510103340 2024-07-29T00:34:22.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.781+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0510062320 con 0x7f04f80a2c90 2024-07-29T00:34:22.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.792+0000 7f050e000640 1 -- 172.21.15.175:0/3430135431 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0510062320 con 0x7f04f80a2c90 2024-07-29T00:34:22.793 INFO:teuthology.orchestra.run.smithi175.stdout:/volumes/qa/sv_0/3b6b254c-69ec-4601-924a-5f93706d653a 2024-07-29T00:34:22.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f04f80a2c90 msgr2=0x7f04f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:22.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f04f80a2c90 0x7f04f80a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0504000c00 tx=0x7f0504005b50 comp rx=0 tx=0).stop 2024-07-29T00:34:22.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 msgr2=0x7f05101986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:22.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 0x7f05101986a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00e9f0 tx=0x7f04fc00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 shutdown_connections 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f04f80a2c90 0x7f04f80a5150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f051010be50 0x7f051019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510103340 0x7f05101986a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 --2- 172.21.15.175:0/3430135431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0510102a20 0x7f0510198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 >> 172.21.15.175:0/3430135431 conn(0x7f05100fc480 msgr2=0x7f05100fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 shutdown_connections 2024-07-29T00:34:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:22.796+0000 7f0517e00640 1 -- 172.21.15.175:0/3430135431 wait complete. 2024-07-29T00:34:23.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-07-29T00:34:24.189 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:24.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 -- 172.21.15.175:0/2570703943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500104430 msgr2=0x7f9500110960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 --2- 172.21.15.175:0/2570703943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500104430 0x7f9500110960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009c10 tx=0x7f94f802a6e0 comp rx=0 tx=0).stop 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 -- 172.21.15.175:0/2570703943 shutdown_connections 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 --2- 172.21.15.175:0/2570703943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500104430 0x7f9500110960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 --2- 172.21.15.175:0/2570703943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500103a90 0x7f9500103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 --2- 172.21.15.175:0/2570703943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9500109a90 0x7f9500109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 -- 172.21.15.175:0/2570703943 >> 172.21.15.175:0/2570703943 conn(0x7f95000ff710 msgr2=0x7f9500101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.796+0000 7f9504e00640 1 -- 172.21.15.175:0/2570703943 shutdown_connections 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.797+0000 7f9504e00640 1 -- 172.21.15.175:0/2570703943 wait complete. 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.797+0000 7f9504e00640 1 Processor -- start 2024-07-29T00:34:24.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 -- start start 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9500103a90 0x7f950019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500104430 0x7f950019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500109a90 0x7f950019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9500113e40 con 0x7f9500109a90 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9500113cc0 con 0x7f9500104430 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f9504e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9500113fc0 con 0x7f9500103a90 2024-07-29T00:34:24.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f94fe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500109a90 0x7f950019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f94fe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500109a90 0x7f950019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40794/0 (socket says 172.21.15.175:40794) 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f94fe800640 1 -- 172.21.15.175:0/3928434962 learned_addr learned my addr 172.21.15.175:0/3928434962 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f94fd400640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500104430 0x7f950019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.798+0000 7f94fde00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9500103a90 0x7f950019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94fd400640 1 -- 172.21.15.175:0/3928434962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9500103a90 msgr2=0x7f950019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94fd400640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9500103a90 0x7f950019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94fd400640 1 -- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500109a90 msgr2=0x7f950019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94fd400640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500109a90 0x7f950019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94fd400640 1 -- 172.21.15.175:0/3928434962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950019f2a0 con 0x7f9500104430 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94fd400640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500104430 0x7f950019a830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e9f0 tx=0x7f94ec00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00ccc0 con 0x7f9500104430 2024-07-29T00:34:24.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec0035f0 con 0x7f9500104430 2024-07-29T00:34:24.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.799+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00ce60 con 0x7f9500104430 2024-07-29T00:34:24.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.800+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950019f590 con 0x7f9500104430 2024-07-29T00:34:24.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.800+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001a6e70 con 0x7f9500104430 2024-07-29T00:34:24.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.801+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9500103ef0 con 0x7f9500104430 2024-07-29T00:34:24.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.804+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f94ec0037a0 con 0x7f9500104430 2024-07-29T00:34:24.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.805+0000 7f94f6a00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f94e40a2c90 0x7f94e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:24.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.805+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f94ec0cf2d0 con 0x7f9500104430 2024-07-29T00:34:24.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.805+0000 7f94fde00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f94e40a2c90 0x7f94e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:24.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.806+0000 7f94fde00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f94e40a2c90 0x7f94e40a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f94f0005fd0 tx=0x7f94f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:24.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:24.806+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f94ec09c180 con 0x7f9500104430 2024-07-29T00:34:24.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:24 smithi181 ceph-mon[36706]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:24.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:24 smithi181 ceph-mon[36706]: pgmap v304: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-07-29T00:34:25.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.027+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9500062320 con 0x7f94e40a2c90 2024-07-29T00:34:25.028 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:24 smithi175 ceph-mon[32138]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:25.028 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:24 smithi175 ceph-mon[32138]: pgmap v304: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-07-29T00:34:25.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.064+0000 7f94f6a00640 1 -- 172.21.15.175:0/3928434962 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9500062320 con 0x7f94e40a2c90 2024-07-29T00:34:25.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.067+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f94e40a2c90 msgr2=0x7f94e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:25.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.067+0000 7f9504e00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f94e40a2c90 0x7f94e40a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f94f0005fd0 tx=0x7f94f0009260 comp rx=0 tx=0).stop 2024-07-29T00:34:25.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.067+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500104430 msgr2=0x7f950019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.067+0000 7f9504e00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500104430 0x7f950019a830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e9f0 tx=0x7f94ec00edf0 comp rx=0 tx=0).stop 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 shutdown_connections 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9500109a90 0x7f950019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9500104430 0x7f950019a830 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f94e40a2c90 0x7f94e40a5150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 --2- 172.21.15.175:0/3928434962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9500103a90 0x7f950019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 >> 172.21.15.175:0/3928434962 conn(0x7f95000ff710 msgr2=0x7f9500101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 shutdown_connections 2024-07-29T00:34:25.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:25.068+0000 7f9504e00640 1 -- 172.21.15.175:0/3928434962 wait complete. 2024-07-29T00:34:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:24 smithi204 ceph-mon[37368]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:24 smithi204 ceph-mon[37368]: pgmap v304: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-07-29T00:34:26.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-07-29T00:34:26.525 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:26.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:26 smithi175 ceph-mon[32138]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:26.888 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:26 smithi175 ceph-mon[32138]: pgmap v305: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-07-29T00:34:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:26 smithi181 ceph-mon[36706]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:26 smithi181 ceph-mon[36706]: pgmap v305: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-07-29T00:34:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:26 smithi204 ceph-mon[37368]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:26 smithi204 ceph-mon[37368]: pgmap v305: 97 pgs: 97 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 -- 172.21.15.175:0/842089751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b0100e40 msgr2=0x7fc4b010db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 --2- 172.21.15.175:0/842089751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b0100e40 0x7fc4b010db00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009900 tx=0x7fc49c02a3f0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 -- 172.21.15.175:0/842089751 shutdown_connections 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 --2- 172.21.15.175:0/842089751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b010e040 0x7fc4b0110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 --2- 172.21.15.175:0/842089751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b0100e40 0x7fc4b010db00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 --2- 172.21.15.175:0/842089751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b0100490 0x7fc4b0100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.160+0000 7fc4af200640 1 -- 172.21.15.175:0/842089751 >> 172.21.15.175:0/842089751 conn(0x7fc4b00fc300 msgr2=0x7fc4b00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.161+0000 7fc4af200640 1 -- 172.21.15.175:0/842089751 shutdown_connections 2024-07-29T00:34:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.161+0000 7fc4af200640 1 -- 172.21.15.175:0/842089751 wait complete. 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 Processor -- start 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 -- start start 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 0x7fc4b019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b0100e40 0x7fc4b019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b010e040 0x7fc4b019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0113d30 con 0x7fc4b010e040 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0113bb0 con 0x7fc4b0100e40 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.162+0000 7fc4af200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0113eb0 con 0x7fc4b0100490 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 0x7fc4b019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 0x7fc4b019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57050/0 (socket says 172.21.15.175:57050) 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ad400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b0100e40 0x7fc4b019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:27.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 -- 172.21.15.175:0/3958808141 learned_addr learned my addr 172.21.15.175:0/3958808141 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:27.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ae800640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b010e040 0x7fc4b019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:27.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 -- 172.21.15.175:0/3958808141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b0100e40 msgr2=0x7fc4b019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b0100e40 0x7fc4b019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 -- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b010e040 msgr2=0x7fc4b019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b010e040 0x7fc4b019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 -- 172.21.15.175:0/3958808141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b019f1e0 con 0x7fc4b0100490 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.163+0000 7fc4ade00640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 0x7fc4b019a230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000e9f0 tx=0x7fc4a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.164+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000cd80 con 0x7fc4b0100490 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.164+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a0010910 con 0x7fc4b0100490 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.164+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0018990 con 0x7fc4b0100490 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.164+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b019f4d0 con 0x7fc4b0100490 2024-07-29T00:34:27.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.164+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01a6db0 con 0x7fc4b0100490 2024-07-29T00:34:27.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.165+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b0100870 con 0x7fc4b0100490 2024-07-29T00:34:27.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.167+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc4a0010ac0 con 0x7fc4b0100490 2024-07-29T00:34:27.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.168+0000 7fc4a6a00640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4940a2c90 0x7fc4940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:27.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.168+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc4a00cf9c0 con 0x7fc4b0100490 2024-07-29T00:34:27.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.168+0000 7fc4ad400640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4940a2c90 0x7fc4940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:27.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.169+0000 7fc4ad400640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4940a2c90 0x7fc4940a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000c00 tx=0x7fc49c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:27.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.170+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc4a009c870 con 0x7fc4b0100490 2024-07-29T00:34:27.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.386+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc4b010f2c0 con 0x7fc4940a2c90 2024-07-29T00:34:27.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.395+0000 7fc4a6a00640 1 -- 172.21.15.175:0/3958808141 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc4b010f2c0 con 0x7fc4940a2c90 2024-07-29T00:34:27.396 INFO:teuthology.orchestra.run.smithi175.stdout:/volumes/qa/sv_1/790caf24-b417-4f46-948e-eb25ca9090b1 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4940a2c90 msgr2=0x7fc4940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4940a2c90 0x7fc4940a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000c00 tx=0x7fc49c005b50 comp rx=0 tx=0).stop 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 msgr2=0x7fc4b019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 0x7fc4b019a230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000e9f0 tx=0x7fc4a000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 shutdown_connections 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4b010e040 0x7fc4b019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b0100e40 0x7fc4b019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.398+0000 7fc4af200640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4940a2c90 0x7fc4940a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.399+0000 7fc4af200640 1 --2- 172.21.15.175:0/3958808141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0100490 0x7fc4b019a230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:27.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.399+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 >> 172.21.15.175:0/3958808141 conn(0x7fc4b00fc300 msgr2=0x7fc4b00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:27.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.399+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 shutdown_connections 2024-07-29T00:34:27.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:27.399+0000 7fc4af200640 1 -- 172.21.15.175:0/3958808141 wait complete. 2024-07-29T00:34:28.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:34:28.870 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:28 smithi181 ceph-mon[36706]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:28 smithi181 ceph-mon[36706]: pgmap v306: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-07-29T00:34:29.014 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:28 smithi175 ceph-mon[32138]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:29.015 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:28 smithi175 ceph-mon[32138]: pgmap v306: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-07-29T00:34:29.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:28 smithi204 ceph-mon[37368]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:34:29.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:28 smithi204 ceph-mon[37368]: pgmap v306: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-07-29T00:34:29.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 -- 172.21.15.175:0/3191620637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 msgr2=0x7f87e8114ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 --2- 172.21.15.175:0/3191620637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e8114ab0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009ca0 tx=0x7f87e002a950 comp rx=0 tx=0).stop 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 -- 172.21.15.175:0/3191620637 shutdown_connections 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 --2- 172.21.15.175:0/3191620637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e8114ab0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 --2- 172.21.15.175:0/3191620637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e8112180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 --2- 172.21.15.175:0/3191620637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e8074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 -- 172.21.15.175:0/3191620637 >> 172.21.15.175:0/3191620637 conn(0x7f87e80fd3d0 msgr2=0x7f87e80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 -- 172.21.15.175:0/3191620637 shutdown_connections 2024-07-29T00:34:29.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.560+0000 7f87ef000640 1 -- 172.21.15.175:0/3191620637 wait complete. 2024-07-29T00:34:29.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.561+0000 7f87ef000640 1 Processor -- start 2024-07-29T00:34:29.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.561+0000 7f87ef000640 1 -- start start 2024-07-29T00:34:29.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.561+0000 7f87ef000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:29.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.561+0000 7f87ef000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ef000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e81a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ef000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81180c0 con 0x7f87e8074280 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ef000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8117f40 con 0x7f87e81126c0 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ef000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8118240 con 0x7f87e8074ba0 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87edc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87edc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34882/0 (socket says 172.21.15.175:34882) 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:40084/0 (socket says 172.21.15.175:40084) 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87edc00640 1 -- 172.21.15.175:0/2241663805 learned_addr learned my addr 172.21.15.175:0/2241663805 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ee600640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e81a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 -- 172.21.15.175:0/2241663805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 msgr2=0x7f87e81a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:29.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e81a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 -- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 msgr2=0x7f87e819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ed200640 1 -- 172.21.15.175:0/2241663805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e81a34f0 con 0x7f87e8074ba0 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87ee600640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e81a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.562+0000 7f87edc00640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.563+0000 7f87ed200640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e819ea80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005fd0 tx=0x7f87d400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.563+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4009c30 con 0x7f87e8074ba0 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.563+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e81a37e0 con 0x7f87e8074ba0 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.563+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d4009dd0 con 0x7f87e8074ba0 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.563+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4010460 con 0x7f87e8074ba0 2024-07-29T00:34:29.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.563+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81ab0c0 con 0x7f87e8074ba0 2024-07-29T00:34:29.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.564+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8074660 con 0x7f87e8074ba0 2024-07-29T00:34:29.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.566+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f87d4003f50 con 0x7f87e8074ba0 2024-07-29T00:34:29.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.568+0000 7f87e6000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f87d00a2c10 0x7f87d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:29.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.568+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f87d40cefa0 con 0x7f87e8074ba0 2024-07-29T00:34:29.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.568+0000 7f87edc00640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f87d00a2c10 0x7f87d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:29.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.569+0000 7f87edc00640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f87d00a2c10 0x7f87d00a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87dc011550 tx=0x7f87dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:29.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.570+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f87d4017020 con 0x7f87e8074ba0 2024-07-29T00:34:29.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.799+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87e819f8a0 con 0x7f87e8074ba0 2024-07-29T00:34:29.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.800+0000 7f87e6000640 1 -- 172.21.15.175:0/2241663805 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f87e819f8a0 con 0x7f87e8074ba0 2024-07-29T00:34:29.801 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:29.801 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":28,"btime":"2024-07-29T00:34:15:679894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:34:14.951643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:34:29.801 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:34:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f87d00a2c10 msgr2=0x7f87d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f87d00a2c10 0x7f87d00a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87dc011550 tx=0x7f87dc005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 msgr2=0x7f87e819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e819ea80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005fd0 tx=0x7f87d400dde0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 shutdown_connections 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e81126c0 0x7f87e81a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f87d00a2c10 0x7f87d00a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8074ba0 0x7f87e819ea80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 --2- 172.21.15.175:0/2241663805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87e8074280 0x7f87e819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 >> 172.21.15.175:0/2241663805 conn(0x7f87e80fd3d0 msgr2=0x7f87e80fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 shutdown_connections 2024-07-29T00:34:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:29.804+0000 7f87ef000640 1 -- 172.21.15.175:0/2241663805 wait complete. 2024-07-29T00:34:30.821 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:34:30.893 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:30 smithi175 ceph-mon[32138]: pgmap v307: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-07-29T00:34:30.893 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:30 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2241663805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:34:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:30 smithi181 ceph-mon[36706]: pgmap v307: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-07-29T00:34:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:30 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2241663805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:34:31.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:30 smithi204 ceph-mon[37368]: pgmap v307: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-07-29T00:34:31.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:30 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2241663805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:34:31.280 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 -- 172.21.15.175:0/1344701238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f9810bce0 msgr2=0x7f1f9810e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1344701238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f9810bce0 0x7f1f9810e150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009900 tx=0x7f1f9002a480 comp rx=0 tx=0).stop 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 -- 172.21.15.175:0/1344701238 shutdown_connections 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1344701238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f9810bce0 0x7f1f9810e150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1344701238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 0x7f1f9810b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1344701238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98102940 0x7f1f98102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 -- 172.21.15.175:0/1344701238 >> 172.21.15.175:0/1344701238 conn(0x7f1f980fc300 msgr2=0x7f1f980fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 -- 172.21.15.175:0/1344701238 shutdown_connections 2024-07-29T00:34:31.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.943+0000 7f1f9e400640 1 -- 172.21.15.175:0/1344701238 wait complete. 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.944+0000 7f1f9e400640 1 Processor -- start 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.944+0000 7f1f9e400640 1 -- start start 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 0x7f1f981005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 0x7f1f98100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9810bce0 0x7f1f98101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98111ac0 con 0x7f1f98102940 2024-07-29T00:34:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98111940 con 0x7f1f9810bce0 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98111c40 con 0x7f1f98103260 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 0x7f1f981005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 0x7f1f981005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34900/0 (socket says 172.21.15.175:34900) 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9d000640 1 -- 172.21.15.175:0/1993367539 learned_addr learned my addr 172.21.15.175:0/1993367539 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f97e00640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 0x7f1f98100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.945+0000 7f1f9da00640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9810bce0 0x7f1f98101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f97e00640 1 -- 172.21.15.175:0/1993367539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9810bce0 msgr2=0x7f1f98101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f97e00640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9810bce0 0x7f1f98101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f97e00640 1 -- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 msgr2=0x7f1f981005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f97e00640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 0x7f1f981005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f97e00640 1 -- 172.21.15.175:0/1993367539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f981a63e0 con 0x7f1f98103260 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f9d000640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 0x7f1f981005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:31.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f97e00640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 0x7f1f98100ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e9f0 tx=0x7f1f8000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:31.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000ccc0 con 0x7f1f98103260 2024-07-29T00:34:31.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f80010910 con 0x7f1f98103260 2024-07-29T00:34:31.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f981a66d0 con 0x7f1f98103260 2024-07-29T00:34:31.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.946+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80018960 con 0x7f1f98103260 2024-07-29T00:34:31.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.947+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981a6c10 con 0x7f1f98103260 2024-07-29T00:34:31.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.948+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f98102d20 con 0x7f1f98103260 2024-07-29T00:34:31.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.949+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1f80010ac0 con 0x7f1f98103260 2024-07-29T00:34:31.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.951+0000 7f1f95600640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:31.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.951+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f1f800cf440 con 0x7f1f98103260 2024-07-29T00:34:31.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.951+0000 7f1f9d000640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:31.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.952+0000 7f1f9d000640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800f7d0 tx=0x7f1f88009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:31.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:31.955+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1f8009bca0 con 0x7f1f98103260 2024-07-29T00:34:32.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.145+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f98106dc0 con 0x7f1f98103260 2024-07-29T00:34:32.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.147+0000 7f1f95600640 1 -- 172.21.15.175:0/1993367539 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22263 (secure 0 0 0) 0x7f1f8009ffa0 con 0x7f1f98103260 2024-07-29T00:34:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:32.148 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":87,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:33:58.041543+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:34:32.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f1f7c0a2c10 msgr2=0x7f1f7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:32.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800f7d0 tx=0x7f1f88009500 comp rx=0 tx=0).stop 2024-07-29T00:34:32.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 msgr2=0x7f1f98100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:32.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 0x7f1f98100ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e9f0 tx=0x7f1f8000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:34:32.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 shutdown_connections 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9810bce0 0x7f1f98101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f98103260 0x7f1f98100ae0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 --2- 172.21.15.175:0/1993367539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f98102940 0x7f1f981005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 >> 172.21.15.175:0/1993367539 conn(0x7f1f980fc300 msgr2=0x7f1f980fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 shutdown_connections 2024-07-29T00:34:32.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:32.150+0000 7f1f9e400640 1 -- 172.21.15.175:0/1993367539 wait complete. 2024-07-29T00:34:32.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:32 smithi175 ceph-mon[32138]: pgmap v308: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-07-29T00:34:32.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1993367539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:32 smithi181 ceph-mon[36706]: pgmap v308: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-07-29T00:34:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1993367539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:32 smithi204 ceph-mon[37368]: pgmap v308: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-07-29T00:34:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1993367539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:33.201 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-29T00:34:33.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:34:33.659 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:34.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 -- 172.21.15.175:0/2562490941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6981016a0 msgr2=0x7fc698101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:34.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 --2- 172.21.15.175:0/2562490941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6981016a0 0x7fc698101b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009900 tx=0x7fc68c02a410 comp rx=0 tx=0).stop 2024-07-29T00:34:34.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 -- 172.21.15.175:0/2562490941 shutdown_connections 2024-07-29T00:34:34.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 --2- 172.21.15.175:0/2562490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc698102040 0x7fc69810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 --2- 172.21.15.175:0/2562490941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6981016a0 0x7fc698101b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 --2- 172.21.15.175:0/2562490941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6981076a0 0x7fc698107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.282+0000 7fc69f200640 1 -- 172.21.15.175:0/2562490941 >> 172.21.15.175:0/2562490941 conn(0x7fc6980fd3a0 msgr2=0x7fc6980ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:34.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.283+0000 7fc69f200640 1 -- 172.21.15.175:0/2562490941 shutdown_connections 2024-07-29T00:34:34.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.283+0000 7fc69f200640 1 -- 172.21.15.175:0/2562490941 wait complete. 2024-07-29T00:34:34.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.284+0000 7fc69f200640 1 Processor -- start 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.284+0000 7fc69f200640 1 -- start start 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6981016a0 0x7fc69819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 0x7fc69819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6981076a0 0x7fc69819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc698111af0 con 0x7fc6981016a0 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc698111970 con 0x7fc698102040 2024-07-29T00:34:34.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc698111c70 con 0x7fc6981076a0 2024-07-29T00:34:34.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 0x7fc69819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:34.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6981016a0 0x7fc69819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:34.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 0x7fc69819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42028/0 (socket says 172.21.15.175:42028) 2024-07-29T00:34:34.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 -- 172.21.15.175:0/3928483999 learned_addr learned my addr 172.21.15.175:0/3928483999 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69e800640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6981076a0 0x7fc69819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 -- 172.21.15.175:0/3928483999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6981076a0 msgr2=0x7fc69819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6981076a0 0x7fc69819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 -- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6981016a0 msgr2=0x7fc69819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6981016a0 0x7fc69819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.285+0000 7fc69d400640 1 -- 172.21.15.175:0/3928483999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69819f100 con 0x7fc698102040 2024-07-29T00:34:34.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.286+0000 7fc69d400640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 0x7fc69819a690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02aa00 tx=0x7fc68c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:34.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.286+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c03f070 con 0x7fc698102040 2024-07-29T00:34:34.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.286+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c031c30 con 0x7fc698102040 2024-07-29T00:34:34.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.286+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c035030 con 0x7fc698102040 2024-07-29T00:34:34.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.286+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69819f390 con 0x7fc698102040 2024-07-29T00:34:34.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.286+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69819f820 con 0x7fc698102040 2024-07-29T00:34:34.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.287+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc698101b00 con 0x7fc698102040 2024-07-29T00:34:34.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.289+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc68c042070 con 0x7fc698102040 2024-07-29T00:34:34.291 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.291+0000 7fc692a00640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc6800a2c10 0x7fc6800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:34.291 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.291+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc68c0efde0 con 0x7fc698102040 2024-07-29T00:34:34.291 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.291+0000 7fc69de00640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc6800a2c10 0x7fc6800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:34.292 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.291+0000 7fc69de00640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc6800a2c10 0x7fc6800a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc684009b50 tx=0x7fc684005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:34.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.293+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc68c031de0 con 0x7fc698102040 2024-07-29T00:34:34.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.478+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6980fe980 con 0x7fc6800a2c10 2024-07-29T00:34:34.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.491+0000 7fc692a00640 1 -- 172.21.15.175:0/3928483999 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7fc6980fe980 con 0x7fc6800a2c10 2024-07-29T00:34:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:34.498 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:34:34.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc6800a2c10 msgr2=0x7fc6800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:34.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc6800a2c10 0x7fc6800a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc684009b50 tx=0x7fc684005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:34.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 msgr2=0x7fc69819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 0x7fc69819a690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02aa00 tx=0x7fc68c003c10 comp rx=0 tx=0).stop 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 shutdown_connections 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc6800a2c10 0x7fc6800a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6981076a0 0x7fc69819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc698102040 0x7fc69819a690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 --2- 172.21.15.175:0/3928483999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6981016a0 0x7fc69819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:34.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 >> 172.21.15.175:0/3928483999 conn(0x7fc6980fd3a0 msgr2=0x7fc69810ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:34.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 shutdown_connections 2024-07-29T00:34:34.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:34.496+0000 7fc69f200640 1 -- 172.21.15.175:0/3928483999 wait complete. 2024-07-29T00:34:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:34 smithi175 ceph-mon[32138]: pgmap v309: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-29T00:34:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:34 smithi181 ceph-mon[36706]: pgmap v309: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-29T00:34:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:34 smithi204 ceph-mon[37368]: pgmap v309: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-29T00:34:35.538 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-07-29T00:34:33.549384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647523,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":351,"num_read_kb":287,"num_write":422,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356680,"kb_used_data":11688,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585144,"statfs":{"total":1151940427776,"available":1151575187456,"internally_reserved":0,"allocated":11968512,"data_stored":7421865,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28287,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":125,"num_read_kb":98,"num_write":27,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004465"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746126+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:33:58.746126+0000","last_peered":"2024-07-29T00:33:58.746126+0000","last_clean":"2024-07-29T00:33:58.746126+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:33:58.746126+0000","last_undegraded":"2024-07-29T00:33:58.746126+0000","last_fullsized":"2024-07-29T00:33:58.746126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748738+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:33:58.748738+0000","last_peered":"2024-07-29T00:33:58.748738+0000","last_clean":"2024-07-29T00:33:58.748738+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:33:58.748738+0000","last_undegraded":"2024-07-29T00:33:58.748738+0000","last_fullsized":"2024-07-29T00:33:58.748738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747308+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:33:58.747308+0000","last_peered":"2024-07-29T00:33:58.747308+0000","last_clean":"2024-07-29T00:33:58.747308+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:33:58.747308+0000","last_undegraded":"2024-07-29T00:33:58.747308+0000","last_fullsized":"2024-07-29T00:33:58.747308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.728482+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:33:58.728482+0000","last_peered":"2024-07-29T00:33:58.728482+0000","last_clean":"2024-07-29T00:33:58.728482+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:33:58.728482+0000","last_undegraded":"2024-07-29T00:33:58.728482+0000","last_fullsized":"2024-07-29T00:33:58.728482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729962+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:33:58.729962+0000","last_peered":"2024-07-29T00:33:58.729962+0000","last_clean":"2024-07-29T00:33:58.729962+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:33:58.729962+0000","last_undegraded":"2024-07-29T00:33:58.729962+0000","last_fullsized":"2024-07-29T00:33:58.729962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.744465+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:33:58.744465+0000","last_peered":"2024-07-29T00:33:58.744465+0000","last_clean":"2024-07-29T00:33:58.744465+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:33:58.744465+0000","last_undegraded":"2024-07-29T00:33:58.744465+0000","last_fullsized":"2024-07-29T00:33:58.744465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749281+0000","last_change":"2024-07-29T00:33:56.710011+0000","last_active":"2024-07-29T00:33:58.749281+0000","last_peered":"2024-07-29T00:33:58.749281+0000","last_clean":"2024-07-29T00:33:58.749281+0000","last_became_active":"2024-07-29T00:33:56.709664+0000","last_became_peered":"2024-07-29T00:33:56.709664+0000","last_unstale":"2024-07-29T00:33:58.749281+0000","last_undegraded":"2024-07-29T00:33:58.749281+0000","last_fullsized":"2024-07-29T00:33:58.749281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:53:29.795817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.751202+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:33:58.751202+0000","last_peered":"2024-07-29T00:33:58.751202+0000","last_clean":"2024-07-29T00:33:58.751202+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:33:58.751202+0000","last_undegraded":"2024-07-29T00:33:58.751202+0000","last_fullsized":"2024-07-29T00:33:58.751202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748602+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:33:58.748602+0000","last_peered":"2024-07-29T00:33:58.748602+0000","last_clean":"2024-07-29T00:33:58.748602+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:33:58.748602+0000","last_undegraded":"2024-07-29T00:33:58.748602+0000","last_fullsized":"2024-07-29T00:33:58.748602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.744872+0000","last_change":"2024-07-29T00:33:56.713521+0000","last_active":"2024-07-29T00:33:58.744872+0000","last_peered":"2024-07-29T00:33:58.744872+0000","last_clean":"2024-07-29T00:33:58.744872+0000","last_became_active":"2024-07-29T00:33:56.713170+0000","last_became_peered":"2024-07-29T00:33:56.713170+0000","last_unstale":"2024-07-29T00:33:58.744872+0000","last_undegraded":"2024-07-29T00:33:58.744872+0000","last_fullsized":"2024-07-29T00:33:58.744872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:27:11.771315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749390+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:33:58.749390+0000","last_peered":"2024-07-29T00:33:58.749390+0000","last_clean":"2024-07-29T00:33:58.749390+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:33:58.749390+0000","last_undegraded":"2024-07-29T00:33:58.749390+0000","last_fullsized":"2024-07-29T00:33:58.749390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.728493+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:33:58.728493+0000","last_peered":"2024-07-29T00:33:58.728493+0000","last_clean":"2024-07-29T00:33:58.728493+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:33:58.728493+0000","last_undegraded":"2024-07-29T00:33:58.728493+0000","last_fullsized":"2024-07-29T00:33:58.728493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749423+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:33:58.749423+0000","last_peered":"2024-07-29T00:33:58.749423+0000","last_clean":"2024-07-29T00:33:58.749423+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:33:58.749423+0000","last_undegraded":"2024-07-29T00:33:58.749423+0000","last_fullsized":"2024-07-29T00:33:58.749423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747292+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:33:58.747292+0000","last_peered":"2024-07-29T00:33:58.747292+0000","last_clean":"2024-07-29T00:33:58.747292+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:33:58.747292+0000","last_undegraded":"2024-07-29T00:33:58.747292+0000","last_fullsized":"2024-07-29T00:33:58.747292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746983+0000","last_change":"2024-07-29T00:33:56.724404+0000","last_active":"2024-07-29T00:33:58.746983+0000","last_peered":"2024-07-29T00:33:58.746983+0000","last_clean":"2024-07-29T00:33:58.746983+0000","last_became_active":"2024-07-29T00:33:56.723044+0000","last_became_peered":"2024-07-29T00:33:56.723044+0000","last_unstale":"2024-07-29T00:33:58.746983+0000","last_undegraded":"2024-07-29T00:33:58.746983+0000","last_fullsized":"2024-07-29T00:33:58.746983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:54:28.870102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747345+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:33:58.747345+0000","last_peered":"2024-07-29T00:33:58.747345+0000","last_clean":"2024-07-29T00:33:58.747345+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:33:58.747345+0000","last_undegraded":"2024-07-29T00:33:58.747345+0000","last_fullsized":"2024-07-29T00:33:58.747345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746284+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:33:58.746284+0000","last_peered":"2024-07-29T00:33:58.746284+0000","last_clean":"2024-07-29T00:33:58.746284+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:33:58.746284+0000","last_undegraded":"2024-07-29T00:33:58.746284+0000","last_fullsized":"2024-07-29T00:33:58.746284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729763+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:33:58.729763+0000","last_peered":"2024-07-29T00:33:58.729763+0000","last_clean":"2024-07-29T00:33:58.729763+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:33:58.729763+0000","last_undegraded":"2024-07-29T00:33:58.729763+0000","last_fullsized":"2024-07-29T00:33:58.729763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748415+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:33:58.748415+0000","last_peered":"2024-07-29T00:33:58.748415+0000","last_clean":"2024-07-29T00:33:58.748415+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:33:58.748415+0000","last_undegraded":"2024-07-29T00:33:58.748415+0000","last_fullsized":"2024-07-29T00:33:58.748415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747191+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:33:58.747191+0000","last_peered":"2024-07-29T00:33:58.747191+0000","last_clean":"2024-07-29T00:33:58.747191+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:33:58.747191+0000","last_undegraded":"2024-07-29T00:33:58.747191+0000","last_fullsized":"2024-07-29T00:33:58.747191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749659+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:33:58.749659+0000","last_peered":"2024-07-29T00:33:58.749659+0000","last_clean":"2024-07-29T00:33:58.749659+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:33:58.749659+0000","last_undegraded":"2024-07-29T00:33:58.749659+0000","last_fullsized":"2024-07-29T00:33:58.749659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747268+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:33:58.747268+0000","last_peered":"2024-07-29T00:33:58.747268+0000","last_clean":"2024-07-29T00:33:58.747268+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:33:58.747268+0000","last_undegraded":"2024-07-29T00:33:58.747268+0000","last_fullsized":"2024-07-29T00:33:58.747268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.752998+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:33:58.752998+0000","last_peered":"2024-07-29T00:33:58.752998+0000","last_clean":"2024-07-29T00:33:58.752998+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:33:58.752998+0000","last_undegraded":"2024-07-29T00:33:58.752998+0000","last_fullsized":"2024-07-29T00:33:58.752998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749703+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:33:58.749703+0000","last_peered":"2024-07-29T00:33:58.749703+0000","last_clean":"2024-07-29T00:33:58.749703+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:33:58.749703+0000","last_undegraded":"2024-07-29T00:33:58.749703+0000","last_fullsized":"2024-07-29T00:33:58.749703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.752717+0000","last_change":"2024-07-29T00:33:56.722913+0000","last_active":"2024-07-29T00:33:58.752717+0000","last_peered":"2024-07-29T00:33:58.752717+0000","last_clean":"2024-07-29T00:33:58.752717+0000","last_became_active":"2024-07-29T00:33:56.722561+0000","last_became_peered":"2024-07-29T00:33:56.722561+0000","last_unstale":"2024-07-29T00:33:58.752717+0000","last_undegraded":"2024-07-29T00:33:58.752717+0000","last_fullsized":"2024-07-29T00:33:58.752717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:44.298057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748764+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:33:58.748764+0000","last_peered":"2024-07-29T00:33:58.748764+0000","last_clean":"2024-07-29T00:33:58.748764+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:33:58.748764+0000","last_undegraded":"2024-07-29T00:33:58.748764+0000","last_fullsized":"2024-07-29T00:33:58.748764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745321+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:33:58.745321+0000","last_peered":"2024-07-29T00:33:58.745321+0000","last_clean":"2024-07-29T00:33:58.745321+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:33:58.745321+0000","last_undegraded":"2024-07-29T00:33:58.745321+0000","last_fullsized":"2024-07-29T00:33:58.745321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748610+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:33:58.748610+0000","last_peered":"2024-07-29T00:33:58.748610+0000","last_clean":"2024-07-29T00:33:58.748610+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:33:58.748610+0000","last_undegraded":"2024-07-29T00:33:58.748610+0000","last_fullsized":"2024-07-29T00:33:58.748610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748417+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:33:58.748417+0000","last_peered":"2024-07-29T00:33:58.748417+0000","last_clean":"2024-07-29T00:33:58.748417+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:33:58.748417+0000","last_undegraded":"2024-07-29T00:33:58.748417+0000","last_fullsized":"2024-07-29T00:33:58.748417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746169+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:33:58.746169+0000","last_peered":"2024-07-29T00:33:58.746169+0000","last_clean":"2024-07-29T00:33:58.746169+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:33:58.746169+0000","last_undegraded":"2024-07-29T00:33:58.746169+0000","last_fullsized":"2024-07-29T00:33:58.746169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748336+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:33:58.748336+0000","last_peered":"2024-07-29T00:33:58.748336+0000","last_clean":"2024-07-29T00:33:58.748336+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:33:58.748336+0000","last_undegraded":"2024-07-29T00:33:58.748336+0000","last_fullsized":"2024-07-29T00:33:58.748336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745569+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:33:58.745569+0000","last_peered":"2024-07-29T00:33:58.745569+0000","last_clean":"2024-07-29T00:33:58.745569+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:33:58.745569+0000","last_undegraded":"2024-07-29T00:33:58.745569+0000","last_fullsized":"2024-07-29T00:33:58.745569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729947+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:33:58.729947+0000","last_peered":"2024-07-29T00:33:58.729947+0000","last_clean":"2024-07-29T00:33:58.729947+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:33:58.729947+0000","last_undegraded":"2024-07-29T00:33:58.729947+0000","last_fullsized":"2024-07-29T00:33:58.729947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:31.722007+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:34:31.722007+0000","last_peered":"2024-07-29T00:34:31.722007+0000","last_clean":"2024-07-29T00:34:31.722007+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:34:31.722007+0000","last_undegraded":"2024-07-29T00:34:31.722007+0000","last_fullsized":"2024-07-29T00:34:31.722007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747714+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:33:58.747714+0000","last_peered":"2024-07-29T00:33:58.747714+0000","last_clean":"2024-07-29T00:33:58.747714+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:33:58.747714+0000","last_undegraded":"2024-07-29T00:33:58.747714+0000","last_fullsized":"2024-07-29T00:33:58.747714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:11.673139+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:34:11.673139+0000","last_peered":"2024-07-29T00:34:11.673139+0000","last_clean":"2024-07-29T00:34:11.673139+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:34:11.673139+0000","last_undegraded":"2024-07-29T00:34:11.673139+0000","last_fullsized":"2024-07-29T00:34:11.673139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:10.660065+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:34:10.660065+0000","last_peered":"2024-07-29T00:34:10.660065+0000","last_clean":"2024-07-29T00:34:10.660065+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:34:10.660065+0000","last_undegraded":"2024-07-29T00:34:10.660065+0000","last_fullsized":"2024-07-29T00:34:10.660065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.751460+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:33:58.751460+0000","last_peered":"2024-07-29T00:33:58.751460+0000","last_clean":"2024-07-29T00:33:58.751460+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:33:58.751460+0000","last_undegraded":"2024-07-29T00:33:58.751460+0000","last_fullsized":"2024-07-29T00:33:58.751460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.682491+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:34:12.682491+0000","last_peered":"2024-07-29T00:34:12.682491+0000","last_clean":"2024-07-29T00:34:12.682491+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:34:12.682491+0000","last_undegraded":"2024-07-29T00:34:12.682491+0000","last_fullsized":"2024-07-29T00:34:12.682491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729461+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:33:58.729461+0000","last_peered":"2024-07-29T00:33:58.729461+0000","last_clean":"2024-07-29T00:33:58.729461+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:33:58.729461+0000","last_undegraded":"2024-07-29T00:33:58.729461+0000","last_fullsized":"2024-07-29T00:33:58.729461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:29.718589+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:34:29.718589+0000","last_peered":"2024-07-29T00:34:29.718589+0000","last_clean":"2024-07-29T00:34:29.718589+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:34:29.718589+0000","last_undegraded":"2024-07-29T00:34:29.718589+0000","last_fullsized":"2024-07-29T00:34:29.718589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748387+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:33:58.748387+0000","last_peered":"2024-07-29T00:33:58.748387+0000","last_clean":"2024-07-29T00:33:58.748387+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:33:58.748387+0000","last_undegraded":"2024-07-29T00:33:58.748387+0000","last_fullsized":"2024-07-29T00:33:58.748387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.682792+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:34:12.682792+0000","last_peered":"2024-07-29T00:34:12.682792+0000","last_clean":"2024-07-29T00:34:12.682792+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:34:12.682792+0000","last_undegraded":"2024-07-29T00:34:12.682792+0000","last_fullsized":"2024-07-29T00:34:12.682792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748086+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:33:58.748086+0000","last_peered":"2024-07-29T00:33:58.748086+0000","last_clean":"2024-07-29T00:33:58.748086+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:33:58.748086+0000","last_undegraded":"2024-07-29T00:33:58.748086+0000","last_fullsized":"2024-07-29T00:33:58.748086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:11.674501+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:34:11.674501+0000","last_peered":"2024-07-29T00:34:11.674501+0000","last_clean":"2024-07-29T00:34:11.674501+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:34:11.674501+0000","last_undegraded":"2024-07-29T00:34:11.674501+0000","last_fullsized":"2024-07-29T00:34:11.674501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747129+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:33:58.747129+0000","last_peered":"2024-07-29T00:33:58.747129+0000","last_clean":"2024-07-29T00:33:58.747129+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:33:58.747129+0000","last_undegraded":"2024-07-29T00:33:58.747129+0000","last_fullsized":"2024-07-29T00:33:58.747129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:29.719561+0000","last_change":"2024-07-29T00:33:53.682871+0000","last_active":"2024-07-29T00:34:29.719561+0000","last_peered":"2024-07-29T00:34:29.719561+0000","last_clean":"2024-07-29T00:34:29.719561+0000","last_became_active":"2024-07-29T00:33:53.682354+0000","last_became_peered":"2024-07-29T00:33:53.682354+0000","last_unstale":"2024-07-29T00:34:29.719561+0000","last_undegraded":"2024-07-29T00:34:29.719561+0000","last_fullsized":"2024-07-29T00:34:29.719561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:27:48.417334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50109,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":68,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.730406+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:33:58.730406+0000","last_peered":"2024-07-29T00:33:58.730406+0000","last_clean":"2024-07-29T00:33:58.730406+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:33:58.730406+0000","last_undegraded":"2024-07-29T00:33:58.730406+0000","last_fullsized":"2024-07-29T00:33:58.730406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:13.692396+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:34:13.692396+0000","last_peered":"2024-07-29T00:34:13.692396+0000","last_clean":"2024-07-29T00:34:13.692396+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:34:13.692396+0000","last_undegraded":"2024-07-29T00:34:13.692396+0000","last_fullsized":"2024-07-29T00:34:13.692396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748084+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:33:58.748084+0000","last_peered":"2024-07-29T00:33:58.748084+0000","last_clean":"2024-07-29T00:33:58.748084+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:33:58.748084+0000","last_undegraded":"2024-07-29T00:33:58.748084+0000","last_fullsized":"2024-07-29T00:33:58.748084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.683512+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:34:12.683512+0000","last_peered":"2024-07-29T00:34:12.683512+0000","last_clean":"2024-07-29T00:34:12.683512+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:34:12.683512+0000","last_undegraded":"2024-07-29T00:34:12.683512+0000","last_fullsized":"2024-07-29T00:34:12.683512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749118+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:33:58.749118+0000","last_peered":"2024-07-29T00:33:58.749118+0000","last_clean":"2024-07-29T00:33:58.749118+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:33:58.749118+0000","last_undegraded":"2024-07-29T00:33:58.749118+0000","last_fullsized":"2024-07-29T00:33:58.749118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:30.739277+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:34:30.739277+0000","last_peered":"2024-07-29T00:34:30.739277+0000","last_clean":"2024-07-29T00:34:30.739277+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:34:30.739277+0000","last_undegraded":"2024-07-29T00:34:30.739277+0000","last_fullsized":"2024-07-29T00:34:30.739277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749995+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:33:58.749995+0000","last_peered":"2024-07-29T00:33:58.749995+0000","last_clean":"2024-07-29T00:33:58.749995+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:33:58.749995+0000","last_undegraded":"2024-07-29T00:33:58.749995+0000","last_fullsized":"2024-07-29T00:33:58.749995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.683496+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:34:12.683496+0000","last_peered":"2024-07-29T00:34:12.683496+0000","last_clean":"2024-07-29T00:34:12.683496+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:34:12.683496+0000","last_undegraded":"2024-07-29T00:34:12.683496+0000","last_fullsized":"2024-07-29T00:34:12.683496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747111+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:33:58.747111+0000","last_peered":"2024-07-29T00:33:58.747111+0000","last_clean":"2024-07-29T00:33:58.747111+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:33:58.747111+0000","last_undegraded":"2024-07-29T00:33:58.747111+0000","last_fullsized":"2024-07-29T00:33:58.747111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.652881+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:34:12.652881+0000","last_peered":"2024-07-29T00:34:12.652881+0000","last_clean":"2024-07-29T00:34:12.652881+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:34:12.652881+0000","last_undegraded":"2024-07-29T00:34:12.652881+0000","last_fullsized":"2024-07-29T00:34:12.652881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745038+0000","last_change":"2024-07-29T00:33:56.713862+0000","last_active":"2024-07-29T00:33:58.745038+0000","last_peered":"2024-07-29T00:33:58.745038+0000","last_clean":"2024-07-29T00:33:58.745038+0000","last_became_active":"2024-07-29T00:33:56.713282+0000","last_became_peered":"2024-07-29T00:33:56.713282+0000","last_unstale":"2024-07-29T00:33:58.745038+0000","last_undegraded":"2024-07-29T00:33:58.745038+0000","last_fullsized":"2024-07-29T00:33:58.745038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:27:40.038147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.686697+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:34:12.686697+0000","last_peered":"2024-07-29T00:34:12.686697+0000","last_clean":"2024-07-29T00:34:12.686697+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:34:12.686697+0000","last_undegraded":"2024-07-29T00:34:12.686697+0000","last_fullsized":"2024-07-29T00:34:12.686697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.750172+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:33:58.750172+0000","last_peered":"2024-07-29T00:33:58.750172+0000","last_clean":"2024-07-29T00:33:58.750172+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:33:58.750172+0000","last_undegraded":"2024-07-29T00:33:58.750172+0000","last_fullsized":"2024-07-29T00:33:58.750172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.682612+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:34:12.682612+0000","last_peered":"2024-07-29T00:34:12.682612+0000","last_clean":"2024-07-29T00:34:12.682612+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:34:12.682612+0000","last_undegraded":"2024-07-29T00:34:12.682612+0000","last_fullsized":"2024-07-29T00:34:12.682612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749331+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:33:58.749331+0000","last_peered":"2024-07-29T00:33:58.749331+0000","last_clean":"2024-07-29T00:33:58.749331+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:33:58.749331+0000","last_undegraded":"2024-07-29T00:33:58.749331+0000","last_fullsized":"2024-07-29T00:33:58.749331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:27.716671+0000","last_change":"2024-07-29T00:33:54.215904+0000","last_active":"2024-07-29T00:34:27.716671+0000","last_peered":"2024-07-29T00:34:27.716671+0000","last_clean":"2024-07-29T00:34:27.716671+0000","last_became_active":"2024-07-29T00:33:54.215431+0000","last_became_peered":"2024-07-29T00:33:54.215431+0000","last_unstale":"2024-07-29T00:34:27.716671+0000","last_undegraded":"2024-07-29T00:34:27.716671+0000","last_fullsized":"2024-07-29T00:34:27.716671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:44:33.563700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749432+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:33:58.749432+0000","last_peered":"2024-07-29T00:33:58.749432+0000","last_clean":"2024-07-29T00:33:58.749432+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:33:58.749432+0000","last_undegraded":"2024-07-29T00:33:58.749432+0000","last_fullsized":"2024-07-29T00:33:58.749432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.750107+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:33:58.750107+0000","last_peered":"2024-07-29T00:33:58.750107+0000","last_clean":"2024-07-29T00:33:58.750107+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:33:58.750107+0000","last_undegraded":"2024-07-29T00:33:58.750107+0000","last_fullsized":"2024-07-29T00:33:58.750107+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:13.692914+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:34:13.692914+0000","last_peered":"2024-07-29T00:34:13.692914+0000","last_clean":"2024-07-29T00:34:13.692914+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:34:13.692914+0000","last_undegraded":"2024-07-29T00:34:13.692914+0000","last_fullsized":"2024-07-29T00:34:13.692914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.753867+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:33:58.753867+0000","last_peered":"2024-07-29T00:33:58.753867+0000","last_clean":"2024-07-29T00:33:58.753867+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:33:58.753867+0000","last_undegraded":"2024-07-29T00:33:58.753867+0000","last_fullsized":"2024-07-29T00:33:58.753867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:10.656631+0000","last_change":"2024-07-29T00:33:53.680861+0000","last_active":"2024-07-29T00:34:10.656631+0000","last_peered":"2024-07-29T00:34:10.656631+0000","last_clean":"2024-07-29T00:34:10.656631+0000","last_became_active":"2024-07-29T00:33:53.680260+0000","last_became_peered":"2024-07-29T00:33:53.680260+0000","last_unstale":"2024-07-29T00:34:10.656631+0000","last_undegraded":"2024-07-29T00:34:10.656631+0000","last_fullsized":"2024-07-29T00:34:10.656631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:50:13.931513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747409+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:33:58.747409+0000","last_peered":"2024-07-29T00:33:58.747409+0000","last_clean":"2024-07-29T00:33:58.747409+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:33:58.747409+0000","last_undegraded":"2024-07-29T00:33:58.747409+0000","last_fullsized":"2024-07-29T00:33:58.747409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:27.732687+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:34:27.732687+0000","last_peered":"2024-07-29T00:34:27.732687+0000","last_clean":"2024-07-29T00:34:27.732687+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:34:27.732687+0000","last_undegraded":"2024-07-29T00:34:27.732687+0000","last_fullsized":"2024-07-29T00:34:27.732687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749292+0000","last_change":"2024-07-29T00:33:56.705254+0000","last_active":"2024-07-29T00:33:58.749292+0000","last_peered":"2024-07-29T00:33:58.749292+0000","last_clean":"2024-07-29T00:33:58.749292+0000","last_became_active":"2024-07-29T00:33:56.704900+0000","last_became_peered":"2024-07-29T00:33:56.704900+0000","last_unstale":"2024-07-29T00:33:58.749292+0000","last_undegraded":"2024-07-29T00:33:58.749292+0000","last_fullsized":"2024-07-29T00:33:58.749292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:38:00.364399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:30.729049+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:34:30.729049+0000","last_peered":"2024-07-29T00:34:30.729049+0000","last_clean":"2024-07-29T00:34:30.729049+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:34:30.729049+0000","last_undegraded":"2024-07-29T00:34:30.729049+0000","last_fullsized":"2024-07-29T00:34:30.729049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.744522+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:33:58.744522+0000","last_peered":"2024-07-29T00:33:58.744522+0000","last_clean":"2024-07-29T00:33:58.744522+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:33:58.744522+0000","last_undegraded":"2024-07-29T00:33:58.744522+0000","last_fullsized":"2024-07-29T00:33:58.744522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729646+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:33:58.729646+0000","last_peered":"2024-07-29T00:33:58.729646+0000","last_clean":"2024-07-29T00:33:58.729646+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:33:58.729646+0000","last_undegraded":"2024-07-29T00:33:58.729646+0000","last_fullsized":"2024-07-29T00:33:58.729646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:00.622853+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:34:00.622853+0000","last_peered":"2024-07-29T00:34:00.622853+0000","last_clean":"2024-07-29T00:34:00.622853+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:34:00.622853+0000","last_undegraded":"2024-07-29T00:34:00.622853+0000","last_fullsized":"2024-07-29T00:34:00.622853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.687334+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:34:12.687334+0000","last_peered":"2024-07-29T00:34:12.687334+0000","last_clean":"2024-07-29T00:34:12.687334+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:34:12.687334+0000","last_undegraded":"2024-07-29T00:34:12.687334+0000","last_fullsized":"2024-07-29T00:34:12.687334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748776+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:33:58.748776+0000","last_peered":"2024-07-29T00:33:58.748776+0000","last_clean":"2024-07-29T00:33:58.748776+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:33:58.748776+0000","last_undegraded":"2024-07-29T00:33:58.748776+0000","last_fullsized":"2024-07-29T00:33:58.748776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:28.748230+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:34:28.748230+0000","last_peered":"2024-07-29T00:34:28.748230+0000","last_clean":"2024-07-29T00:34:28.748230+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:34:28.748230+0000","last_undegraded":"2024-07-29T00:34:28.748230+0000","last_fullsized":"2024-07-29T00:34:28.748230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.752872+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:33:58.752872+0000","last_peered":"2024-07-29T00:33:58.752872+0000","last_clean":"2024-07-29T00:33:58.752872+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:33:58.752872+0000","last_undegraded":"2024-07-29T00:33:58.752872+0000","last_fullsized":"2024-07-29T00:33:58.752872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.687147+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:34:12.687147+0000","last_peered":"2024-07-29T00:34:12.687147+0000","last_clean":"2024-07-29T00:34:12.687147+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:34:12.687147+0000","last_undegraded":"2024-07-29T00:34:12.687147+0000","last_fullsized":"2024-07-29T00:34:12.687147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748768+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:33:58.748768+0000","last_peered":"2024-07-29T00:33:58.748768+0000","last_clean":"2024-07-29T00:33:58.748768+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:33:58.748768+0000","last_undegraded":"2024-07-29T00:33:58.748768+0000","last_fullsized":"2024-07-29T00:33:58.748768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:09.657248+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:34:09.657248+0000","last_peered":"2024-07-29T00:34:09.657248+0000","last_clean":"2024-07-29T00:34:09.657248+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:34:09.657248+0000","last_undegraded":"2024-07-29T00:34:09.657248+0000","last_fullsized":"2024-07-29T00:34:09.657248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729837+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:33:58.729837+0000","last_peered":"2024-07-29T00:33:58.729837+0000","last_clean":"2024-07-29T00:33:58.729837+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:33:58.729837+0000","last_undegraded":"2024-07-29T00:33:58.729837+0000","last_fullsized":"2024-07-29T00:33:58.729837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.686445+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:34:12.686445+0000","last_peered":"2024-07-29T00:34:12.686445+0000","last_clean":"2024-07-29T00:34:12.686445+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:34:12.686445+0000","last_undegraded":"2024-07-29T00:34:12.686445+0000","last_fullsized":"2024-07-29T00:34:12.686445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745713+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:33:58.745713+0000","last_peered":"2024-07-29T00:33:58.745713+0000","last_clean":"2024-07-29T00:33:58.745713+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:33:58.745713+0000","last_undegraded":"2024-07-29T00:33:58.745713+0000","last_fullsized":"2024-07-29T00:33:58.745713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:11.670857+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:34:11.670857+0000","last_peered":"2024-07-29T00:34:11.670857+0000","last_clean":"2024-07-29T00:34:11.670857+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:34:11.670857+0000","last_undegraded":"2024-07-29T00:34:11.670857+0000","last_fullsized":"2024-07-29T00:34:11.670857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.751310+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:33:58.751310+0000","last_peered":"2024-07-29T00:33:58.751310+0000","last_clean":"2024-07-29T00:33:58.751310+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:33:58.751310+0000","last_undegraded":"2024-07-29T00:33:58.751310+0000","last_fullsized":"2024-07-29T00:33:58.751310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:09.656532+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:34:09.656532+0000","last_peered":"2024-07-29T00:34:09.656532+0000","last_clean":"2024-07-29T00:34:09.656532+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:34:09.656532+0000","last_undegraded":"2024-07-29T00:34:09.656532+0000","last_fullsized":"2024-07-29T00:34:09.656532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749450+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:33:58.749450+0000","last_peered":"2024-07-29T00:33:58.749450+0000","last_clean":"2024-07-29T00:33:58.749450+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:33:58.749450+0000","last_undegraded":"2024-07-29T00:33:58.749450+0000","last_fullsized":"2024-07-29T00:33:58.749450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749392+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:33:58.749392+0000","last_peered":"2024-07-29T00:33:58.749392+0000","last_clean":"2024-07-29T00:33:58.749392+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:33:58.749392+0000","last_undegraded":"2024-07-29T00:33:58.749392+0000","last_fullsized":"2024-07-29T00:33:58.749392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.684466+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:34:12.684466+0000","last_peered":"2024-07-29T00:34:12.684466+0000","last_clean":"2024-07-29T00:34:12.684466+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:34:12.684466+0000","last_undegraded":"2024-07-29T00:34:12.684466+0000","last_fullsized":"2024-07-29T00:34:12.684466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747318+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:33:58.747318+0000","last_peered":"2024-07-29T00:33:58.747318+0000","last_clean":"2024-07-29T00:33:58.747318+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:33:58.747318+0000","last_undegraded":"2024-07-29T00:33:58.747318+0000","last_fullsized":"2024-07-29T00:33:58.747318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:13.693335+0000","last_change":"2024-07-29T00:33:53.679893+0000","last_active":"2024-07-29T00:34:13.693335+0000","last_peered":"2024-07-29T00:34:13.693335+0000","last_clean":"2024-07-29T00:34:13.693335+0000","last_became_active":"2024-07-29T00:33:53.679525+0000","last_became_peered":"2024-07-29T00:33:53.679525+0000","last_unstale":"2024-07-29T00:34:13.693335+0000","last_undegraded":"2024-07-29T00:34:13.693335+0000","last_fullsized":"2024-07-29T00:34:13.693335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:59.138923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747793+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:33:58.747793+0000","last_peered":"2024-07-29T00:33:58.747793+0000","last_clean":"2024-07-29T00:33:58.747793+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:33:58.747793+0000","last_undegraded":"2024-07-29T00:33:58.747793+0000","last_fullsized":"2024-07-29T00:33:58.747793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.755603+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:33:58.755603+0000","last_peered":"2024-07-29T00:33:58.755603+0000","last_clean":"2024-07-29T00:33:58.755603+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:33:58.755603+0000","last_undegraded":"2024-07-29T00:33:58.755603+0000","last_fullsized":"2024-07-29T00:33:58.755603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.753406+0000","last_change":"2024-07-29T00:33:56.716344+0000","last_active":"2024-07-29T00:33:58.753406+0000","last_peered":"2024-07-29T00:33:58.753406+0000","last_clean":"2024-07-29T00:33:58.753406+0000","last_became_active":"2024-07-29T00:33:56.715873+0000","last_became_peered":"2024-07-29T00:33:56.715873+0000","last_unstale":"2024-07-29T00:33:58.753406+0000","last_undegraded":"2024-07-29T00:33:58.753406+0000","last_fullsized":"2024-07-29T00:33:58.753406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:58:08.050061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:30.728194+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:34:30.728194+0000","last_peered":"2024-07-29T00:34:30.728194+0000","last_clean":"2024-07-29T00:34:30.728194+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:34:30.728194+0000","last_undegraded":"2024-07-29T00:34:30.728194+0000","last_fullsized":"2024-07-29T00:34:30.728194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56921,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":215,"num_read_kb":168,"num_write":239,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":170763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247654,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416405,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":457800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1099328,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":458108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1050625,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":457447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757300,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1049913,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":457698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":457272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":508689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":508469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":458786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:34:35.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:34:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:35 smithi175 ceph-mon[32138]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:34:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:35 smithi181 ceph-mon[36706]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:34:36.012 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:35 smithi204 ceph-mon[37368]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 -- 172.21.15.175:0/2177771591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c103a80 msgr2=0x7f514c103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 --2- 172.21.15.175:0/2177771591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c103a80 0x7f514c103ee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5138009930 tx=0x7f513802a440 comp rx=0 tx=0).stop 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 -- 172.21.15.175:0/2177771591 shutdown_connections 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 --2- 172.21.15.175:0/2177771591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514c104420 0x7f514c110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 --2- 172.21.15.175:0/2177771591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c103a80 0x7f514c103ee0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 --2- 172.21.15.175:0/2177771591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c109a80 0x7f514c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 -- 172.21.15.175:0/2177771591 >> 172.21.15.175:0/2177771591 conn(0x7f514c0ff760 msgr2=0x7f514c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 -- 172.21.15.175:0/2177771591 shutdown_connections 2024-07-29T00:34:36.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.658+0000 7f5153e00640 1 -- 172.21.15.175:0/2177771591 wait complete. 2024-07-29T00:34:36.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.659+0000 7f5153e00640 1 Processor -- start 2024-07-29T00:34:36.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 -- start start 2024-07-29T00:34:36.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514c103a80 0x7f514c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:36.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 0x7f514c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:36.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c109a80 0x7f514c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:36.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f514c113e90 con 0x7f514c104420 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f514c113d10 con 0x7f514c109a80 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5153e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f514c114010 con 0x7f514c103a80 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5150a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 0x7f514c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5150a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 0x7f514c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34940/0 (socket says 172.21.15.175:34940) 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5151400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514c103a80 0x7f514c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5150a00640 1 -- 172.21.15.175:0/2375698056 learned_addr learned my addr 172.21.15.175:0/2375698056 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.660+0000 7f5151e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c109a80 0x7f514c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.661+0000 7f5150a00640 1 -- 172.21.15.175:0/2375698056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514c103a80 msgr2=0x7f514c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.661+0000 7f5150a00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514c103a80 0x7f514c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.661+0000 7f5150a00640 1 -- 172.21.15.175:0/2375698056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c109a80 msgr2=0x7f514c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.661+0000 7f5150a00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c109a80 0x7f514c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.661+0000 7f5150a00640 1 -- 172.21.15.175:0/2375698056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c19f3a0 con 0x7f514c104420 2024-07-29T00:34:36.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.661+0000 7f5150a00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 0x7f514c19a930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f513802a960 tx=0x7f5138003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:36.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.662+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513803f070 con 0x7f514c104420 2024-07-29T00:34:36.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.662+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138031c90 con 0x7f514c104420 2024-07-29T00:34:36.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.662+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c19f630 con 0x7f514c104420 2024-07-29T00:34:36.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.662+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138035030 con 0x7f514c104420 2024-07-29T00:34:36.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.662+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1a6f70 con 0x7f514c104420 2024-07-29T00:34:36.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.663+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c1051c0 con 0x7f514c104420 2024-07-29T00:34:36.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.665+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5138042070 con 0x7f514c104420 2024-07-29T00:34:36.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.666+0000 7f514a000640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f51340a2c10 0x7f51340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:36.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.666+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f51380ef4d0 con 0x7f514c104420 2024-07-29T00:34:36.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.666+0000 7f5151400640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f51340a2c10 0x7f51340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:36.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.667+0000 7f5151400640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f51340a2c10 0x7f51340a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f514000bb60 tx=0x7f514000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:36.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.671+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f51380bbd30 con 0x7f514c104420 2024-07-29T00:34:36.671 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:36 smithi175 ceph-mon[32138]: pgmap v310: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-07-29T00:34:36.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.866+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f514c10c040 con 0x7f51340a2c10 2024-07-29T00:34:36.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.878+0000 7f514a000640 1 -- 172.21.15.175:0/2375698056 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f514c10c040 con 0x7f51340a2c10 2024-07-29T00:34:36.882 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:36.883 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f51340a2c10 msgr2=0x7f51340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f51340a2c10 0x7f51340a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f514000bb60 tx=0x7f514000b390 comp rx=0 tx=0).stop 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 msgr2=0x7f514c19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 0x7f514c19a930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f513802a960 tx=0x7f5138003b90 comp rx=0 tx=0).stop 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 shutdown_connections 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c109a80 0x7f514c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514c104420 0x7f514c19a930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f51340a2c10 0x7f51340a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 --2- 172.21.15.175:0/2375698056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514c103a80 0x7f514c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:36.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 >> 172.21.15.175:0/2375698056 conn(0x7f514c0ff760 msgr2=0x7f514c100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:36.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 shutdown_connections 2024-07-29T00:34:36.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:36.884+0000 7f5153e00640 1 -- 172.21.15.175:0/2375698056 wait complete. 2024-07-29T00:34:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:36 smithi181 ceph-mon[36706]: pgmap v310: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-07-29T00:34:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:36 smithi204 ceph-mon[37368]: pgmap v310: 97 pgs: 97 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-07-29T00:34:37.788 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-07-29T00:34:35.549945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647523,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":361,"num_read_kb":292,"num_write":422,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356688,"kb_used_data":11696,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585136,"statfs":{"total":1151940427776,"available":1151575179264,"internally_reserved":0,"allocated":11976704,"data_stored":7430379,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27897,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":97,"num_write":22,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004289"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746126+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:33:58.746126+0000","last_peered":"2024-07-29T00:33:58.746126+0000","last_clean":"2024-07-29T00:33:58.746126+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:33:58.746126+0000","last_undegraded":"2024-07-29T00:33:58.746126+0000","last_fullsized":"2024-07-29T00:33:58.746126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748738+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:33:58.748738+0000","last_peered":"2024-07-29T00:33:58.748738+0000","last_clean":"2024-07-29T00:33:58.748738+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:33:58.748738+0000","last_undegraded":"2024-07-29T00:33:58.748738+0000","last_fullsized":"2024-07-29T00:33:58.748738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747308+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:33:58.747308+0000","last_peered":"2024-07-29T00:33:58.747308+0000","last_clean":"2024-07-29T00:33:58.747308+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:33:58.747308+0000","last_undegraded":"2024-07-29T00:33:58.747308+0000","last_fullsized":"2024-07-29T00:33:58.747308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.728482+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:33:58.728482+0000","last_peered":"2024-07-29T00:33:58.728482+0000","last_clean":"2024-07-29T00:33:58.728482+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:33:58.728482+0000","last_undegraded":"2024-07-29T00:33:58.728482+0000","last_fullsized":"2024-07-29T00:33:58.728482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729962+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:33:58.729962+0000","last_peered":"2024-07-29T00:33:58.729962+0000","last_clean":"2024-07-29T00:33:58.729962+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:33:58.729962+0000","last_undegraded":"2024-07-29T00:33:58.729962+0000","last_fullsized":"2024-07-29T00:33:58.729962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.744465+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:33:58.744465+0000","last_peered":"2024-07-29T00:33:58.744465+0000","last_clean":"2024-07-29T00:33:58.744465+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:33:58.744465+0000","last_undegraded":"2024-07-29T00:33:58.744465+0000","last_fullsized":"2024-07-29T00:33:58.744465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749281+0000","last_change":"2024-07-29T00:33:56.710011+0000","last_active":"2024-07-29T00:33:58.749281+0000","last_peered":"2024-07-29T00:33:58.749281+0000","last_clean":"2024-07-29T00:33:58.749281+0000","last_became_active":"2024-07-29T00:33:56.709664+0000","last_became_peered":"2024-07-29T00:33:56.709664+0000","last_unstale":"2024-07-29T00:33:58.749281+0000","last_undegraded":"2024-07-29T00:33:58.749281+0000","last_fullsized":"2024-07-29T00:33:58.749281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:53:29.795817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.751202+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:33:58.751202+0000","last_peered":"2024-07-29T00:33:58.751202+0000","last_clean":"2024-07-29T00:33:58.751202+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:33:58.751202+0000","last_undegraded":"2024-07-29T00:33:58.751202+0000","last_fullsized":"2024-07-29T00:33:58.751202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748602+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:33:58.748602+0000","last_peered":"2024-07-29T00:33:58.748602+0000","last_clean":"2024-07-29T00:33:58.748602+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:33:58.748602+0000","last_undegraded":"2024-07-29T00:33:58.748602+0000","last_fullsized":"2024-07-29T00:33:58.748602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.744872+0000","last_change":"2024-07-29T00:33:56.713521+0000","last_active":"2024-07-29T00:33:58.744872+0000","last_peered":"2024-07-29T00:33:58.744872+0000","last_clean":"2024-07-29T00:33:58.744872+0000","last_became_active":"2024-07-29T00:33:56.713170+0000","last_became_peered":"2024-07-29T00:33:56.713170+0000","last_unstale":"2024-07-29T00:33:58.744872+0000","last_undegraded":"2024-07-29T00:33:58.744872+0000","last_fullsized":"2024-07-29T00:33:58.744872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:27:11.771315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749390+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:33:58.749390+0000","last_peered":"2024-07-29T00:33:58.749390+0000","last_clean":"2024-07-29T00:33:58.749390+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:33:58.749390+0000","last_undegraded":"2024-07-29T00:33:58.749390+0000","last_fullsized":"2024-07-29T00:33:58.749390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.728493+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:33:58.728493+0000","last_peered":"2024-07-29T00:33:58.728493+0000","last_clean":"2024-07-29T00:33:58.728493+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:33:58.728493+0000","last_undegraded":"2024-07-29T00:33:58.728493+0000","last_fullsized":"2024-07-29T00:33:58.728493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749423+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:33:58.749423+0000","last_peered":"2024-07-29T00:33:58.749423+0000","last_clean":"2024-07-29T00:33:58.749423+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:33:58.749423+0000","last_undegraded":"2024-07-29T00:33:58.749423+0000","last_fullsized":"2024-07-29T00:33:58.749423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747292+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:33:58.747292+0000","last_peered":"2024-07-29T00:33:58.747292+0000","last_clean":"2024-07-29T00:33:58.747292+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:33:58.747292+0000","last_undegraded":"2024-07-29T00:33:58.747292+0000","last_fullsized":"2024-07-29T00:33:58.747292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746983+0000","last_change":"2024-07-29T00:33:56.724404+0000","last_active":"2024-07-29T00:33:58.746983+0000","last_peered":"2024-07-29T00:33:58.746983+0000","last_clean":"2024-07-29T00:33:58.746983+0000","last_became_active":"2024-07-29T00:33:56.723044+0000","last_became_peered":"2024-07-29T00:33:56.723044+0000","last_unstale":"2024-07-29T00:33:58.746983+0000","last_undegraded":"2024-07-29T00:33:58.746983+0000","last_fullsized":"2024-07-29T00:33:58.746983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:54:28.870102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747345+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:33:58.747345+0000","last_peered":"2024-07-29T00:33:58.747345+0000","last_clean":"2024-07-29T00:33:58.747345+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:33:58.747345+0000","last_undegraded":"2024-07-29T00:33:58.747345+0000","last_fullsized":"2024-07-29T00:33:58.747345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746284+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:33:58.746284+0000","last_peered":"2024-07-29T00:33:58.746284+0000","last_clean":"2024-07-29T00:33:58.746284+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:33:58.746284+0000","last_undegraded":"2024-07-29T00:33:58.746284+0000","last_fullsized":"2024-07-29T00:33:58.746284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729763+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:33:58.729763+0000","last_peered":"2024-07-29T00:33:58.729763+0000","last_clean":"2024-07-29T00:33:58.729763+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:33:58.729763+0000","last_undegraded":"2024-07-29T00:33:58.729763+0000","last_fullsized":"2024-07-29T00:33:58.729763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748415+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:33:58.748415+0000","last_peered":"2024-07-29T00:33:58.748415+0000","last_clean":"2024-07-29T00:33:58.748415+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:33:58.748415+0000","last_undegraded":"2024-07-29T00:33:58.748415+0000","last_fullsized":"2024-07-29T00:33:58.748415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747191+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:33:58.747191+0000","last_peered":"2024-07-29T00:33:58.747191+0000","last_clean":"2024-07-29T00:33:58.747191+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:33:58.747191+0000","last_undegraded":"2024-07-29T00:33:58.747191+0000","last_fullsized":"2024-07-29T00:33:58.747191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749659+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:33:58.749659+0000","last_peered":"2024-07-29T00:33:58.749659+0000","last_clean":"2024-07-29T00:33:58.749659+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:33:58.749659+0000","last_undegraded":"2024-07-29T00:33:58.749659+0000","last_fullsized":"2024-07-29T00:33:58.749659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747268+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:33:58.747268+0000","last_peered":"2024-07-29T00:33:58.747268+0000","last_clean":"2024-07-29T00:33:58.747268+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:33:58.747268+0000","last_undegraded":"2024-07-29T00:33:58.747268+0000","last_fullsized":"2024-07-29T00:33:58.747268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.752998+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:33:58.752998+0000","last_peered":"2024-07-29T00:33:58.752998+0000","last_clean":"2024-07-29T00:33:58.752998+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:33:58.752998+0000","last_undegraded":"2024-07-29T00:33:58.752998+0000","last_fullsized":"2024-07-29T00:33:58.752998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749703+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:33:58.749703+0000","last_peered":"2024-07-29T00:33:58.749703+0000","last_clean":"2024-07-29T00:33:58.749703+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:33:58.749703+0000","last_undegraded":"2024-07-29T00:33:58.749703+0000","last_fullsized":"2024-07-29T00:33:58.749703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.752717+0000","last_change":"2024-07-29T00:33:56.722913+0000","last_active":"2024-07-29T00:33:58.752717+0000","last_peered":"2024-07-29T00:33:58.752717+0000","last_clean":"2024-07-29T00:33:58.752717+0000","last_became_active":"2024-07-29T00:33:56.722561+0000","last_became_peered":"2024-07-29T00:33:56.722561+0000","last_unstale":"2024-07-29T00:33:58.752717+0000","last_undegraded":"2024-07-29T00:33:58.752717+0000","last_fullsized":"2024-07-29T00:33:58.752717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:44.298057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748764+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:33:58.748764+0000","last_peered":"2024-07-29T00:33:58.748764+0000","last_clean":"2024-07-29T00:33:58.748764+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:33:58.748764+0000","last_undegraded":"2024-07-29T00:33:58.748764+0000","last_fullsized":"2024-07-29T00:33:58.748764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745321+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:33:58.745321+0000","last_peered":"2024-07-29T00:33:58.745321+0000","last_clean":"2024-07-29T00:33:58.745321+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:33:58.745321+0000","last_undegraded":"2024-07-29T00:33:58.745321+0000","last_fullsized":"2024-07-29T00:33:58.745321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748610+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:33:58.748610+0000","last_peered":"2024-07-29T00:33:58.748610+0000","last_clean":"2024-07-29T00:33:58.748610+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:33:58.748610+0000","last_undegraded":"2024-07-29T00:33:58.748610+0000","last_fullsized":"2024-07-29T00:33:58.748610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748417+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:33:58.748417+0000","last_peered":"2024-07-29T00:33:58.748417+0000","last_clean":"2024-07-29T00:33:58.748417+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:33:58.748417+0000","last_undegraded":"2024-07-29T00:33:58.748417+0000","last_fullsized":"2024-07-29T00:33:58.748417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.746169+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:33:58.746169+0000","last_peered":"2024-07-29T00:33:58.746169+0000","last_clean":"2024-07-29T00:33:58.746169+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:33:58.746169+0000","last_undegraded":"2024-07-29T00:33:58.746169+0000","last_fullsized":"2024-07-29T00:33:58.746169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748336+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:33:58.748336+0000","last_peered":"2024-07-29T00:33:58.748336+0000","last_clean":"2024-07-29T00:33:58.748336+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:33:58.748336+0000","last_undegraded":"2024-07-29T00:33:58.748336+0000","last_fullsized":"2024-07-29T00:33:58.748336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745569+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:33:58.745569+0000","last_peered":"2024-07-29T00:33:58.745569+0000","last_clean":"2024-07-29T00:33:58.745569+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:33:58.745569+0000","last_undegraded":"2024-07-29T00:33:58.745569+0000","last_fullsized":"2024-07-29T00:33:58.745569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729947+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:33:58.729947+0000","last_peered":"2024-07-29T00:33:58.729947+0000","last_clean":"2024-07-29T00:33:58.729947+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:33:58.729947+0000","last_undegraded":"2024-07-29T00:33:58.729947+0000","last_fullsized":"2024-07-29T00:33:58.729947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:31.722007+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:34:31.722007+0000","last_peered":"2024-07-29T00:34:31.722007+0000","last_clean":"2024-07-29T00:34:31.722007+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:34:31.722007+0000","last_undegraded":"2024-07-29T00:34:31.722007+0000","last_fullsized":"2024-07-29T00:34:31.722007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747714+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:33:58.747714+0000","last_peered":"2024-07-29T00:33:58.747714+0000","last_clean":"2024-07-29T00:33:58.747714+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:33:58.747714+0000","last_undegraded":"2024-07-29T00:33:58.747714+0000","last_fullsized":"2024-07-29T00:33:58.747714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:11.673139+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:34:11.673139+0000","last_peered":"2024-07-29T00:34:11.673139+0000","last_clean":"2024-07-29T00:34:11.673139+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:34:11.673139+0000","last_undegraded":"2024-07-29T00:34:11.673139+0000","last_fullsized":"2024-07-29T00:34:11.673139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:10.660065+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:34:10.660065+0000","last_peered":"2024-07-29T00:34:10.660065+0000","last_clean":"2024-07-29T00:34:10.660065+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:34:10.660065+0000","last_undegraded":"2024-07-29T00:34:10.660065+0000","last_fullsized":"2024-07-29T00:34:10.660065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.751460+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:33:58.751460+0000","last_peered":"2024-07-29T00:33:58.751460+0000","last_clean":"2024-07-29T00:33:58.751460+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:33:58.751460+0000","last_undegraded":"2024-07-29T00:33:58.751460+0000","last_fullsized":"2024-07-29T00:33:58.751460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.682491+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:34:12.682491+0000","last_peered":"2024-07-29T00:34:12.682491+0000","last_clean":"2024-07-29T00:34:12.682491+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:34:12.682491+0000","last_undegraded":"2024-07-29T00:34:12.682491+0000","last_fullsized":"2024-07-29T00:34:12.682491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729461+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:33:58.729461+0000","last_peered":"2024-07-29T00:33:58.729461+0000","last_clean":"2024-07-29T00:33:58.729461+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:33:58.729461+0000","last_undegraded":"2024-07-29T00:33:58.729461+0000","last_fullsized":"2024-07-29T00:33:58.729461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:29.718589+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:34:29.718589+0000","last_peered":"2024-07-29T00:34:29.718589+0000","last_clean":"2024-07-29T00:34:29.718589+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:34:29.718589+0000","last_undegraded":"2024-07-29T00:34:29.718589+0000","last_fullsized":"2024-07-29T00:34:29.718589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748387+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:33:58.748387+0000","last_peered":"2024-07-29T00:33:58.748387+0000","last_clean":"2024-07-29T00:33:58.748387+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:33:58.748387+0000","last_undegraded":"2024-07-29T00:33:58.748387+0000","last_fullsized":"2024-07-29T00:33:58.748387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.682792+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:34:12.682792+0000","last_peered":"2024-07-29T00:34:12.682792+0000","last_clean":"2024-07-29T00:34:12.682792+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:34:12.682792+0000","last_undegraded":"2024-07-29T00:34:12.682792+0000","last_fullsized":"2024-07-29T00:34:12.682792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748086+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:33:58.748086+0000","last_peered":"2024-07-29T00:33:58.748086+0000","last_clean":"2024-07-29T00:33:58.748086+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:33:58.748086+0000","last_undegraded":"2024-07-29T00:33:58.748086+0000","last_fullsized":"2024-07-29T00:33:58.748086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:11.674501+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:34:11.674501+0000","last_peered":"2024-07-29T00:34:11.674501+0000","last_clean":"2024-07-29T00:34:11.674501+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:34:11.674501+0000","last_undegraded":"2024-07-29T00:34:11.674501+0000","last_fullsized":"2024-07-29T00:34:11.674501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747129+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:33:58.747129+0000","last_peered":"2024-07-29T00:33:58.747129+0000","last_clean":"2024-07-29T00:33:58.747129+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:33:58.747129+0000","last_undegraded":"2024-07-29T00:33:58.747129+0000","last_fullsized":"2024-07-29T00:33:58.747129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:29.719561+0000","last_change":"2024-07-29T00:33:53.682871+0000","last_active":"2024-07-29T00:34:29.719561+0000","last_peered":"2024-07-29T00:34:29.719561+0000","last_clean":"2024-07-29T00:34:29.719561+0000","last_became_active":"2024-07-29T00:33:53.682354+0000","last_became_peered":"2024-07-29T00:33:53.682354+0000","last_unstale":"2024-07-29T00:34:29.719561+0000","last_undegraded":"2024-07-29T00:34:29.719561+0000","last_fullsized":"2024-07-29T00:34:29.719561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:27:48.417334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50109,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":68,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.730406+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:33:58.730406+0000","last_peered":"2024-07-29T00:33:58.730406+0000","last_clean":"2024-07-29T00:33:58.730406+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:33:58.730406+0000","last_undegraded":"2024-07-29T00:33:58.730406+0000","last_fullsized":"2024-07-29T00:33:58.730406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:13.692396+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:34:13.692396+0000","last_peered":"2024-07-29T00:34:13.692396+0000","last_clean":"2024-07-29T00:34:13.692396+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:34:13.692396+0000","last_undegraded":"2024-07-29T00:34:13.692396+0000","last_fullsized":"2024-07-29T00:34:13.692396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748084+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:33:58.748084+0000","last_peered":"2024-07-29T00:33:58.748084+0000","last_clean":"2024-07-29T00:33:58.748084+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:33:58.748084+0000","last_undegraded":"2024-07-29T00:33:58.748084+0000","last_fullsized":"2024-07-29T00:33:58.748084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.683512+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:34:12.683512+0000","last_peered":"2024-07-29T00:34:12.683512+0000","last_clean":"2024-07-29T00:34:12.683512+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:34:12.683512+0000","last_undegraded":"2024-07-29T00:34:12.683512+0000","last_fullsized":"2024-07-29T00:34:12.683512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749118+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:33:58.749118+0000","last_peered":"2024-07-29T00:33:58.749118+0000","last_clean":"2024-07-29T00:33:58.749118+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:33:58.749118+0000","last_undegraded":"2024-07-29T00:33:58.749118+0000","last_fullsized":"2024-07-29T00:33:58.749118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:30.739277+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:34:30.739277+0000","last_peered":"2024-07-29T00:34:30.739277+0000","last_clean":"2024-07-29T00:34:30.739277+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:34:30.739277+0000","last_undegraded":"2024-07-29T00:34:30.739277+0000","last_fullsized":"2024-07-29T00:34:30.739277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749995+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:33:58.749995+0000","last_peered":"2024-07-29T00:33:58.749995+0000","last_clean":"2024-07-29T00:33:58.749995+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:33:58.749995+0000","last_undegraded":"2024-07-29T00:33:58.749995+0000","last_fullsized":"2024-07-29T00:33:58.749995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.683496+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:34:12.683496+0000","last_peered":"2024-07-29T00:34:12.683496+0000","last_clean":"2024-07-29T00:34:12.683496+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:34:12.683496+0000","last_undegraded":"2024-07-29T00:34:12.683496+0000","last_fullsized":"2024-07-29T00:34:12.683496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747111+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:33:58.747111+0000","last_peered":"2024-07-29T00:33:58.747111+0000","last_clean":"2024-07-29T00:33:58.747111+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:33:58.747111+0000","last_undegraded":"2024-07-29T00:33:58.747111+0000","last_fullsized":"2024-07-29T00:33:58.747111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.652881+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:34:12.652881+0000","last_peered":"2024-07-29T00:34:12.652881+0000","last_clean":"2024-07-29T00:34:12.652881+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:34:12.652881+0000","last_undegraded":"2024-07-29T00:34:12.652881+0000","last_fullsized":"2024-07-29T00:34:12.652881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745038+0000","last_change":"2024-07-29T00:33:56.713862+0000","last_active":"2024-07-29T00:33:58.745038+0000","last_peered":"2024-07-29T00:33:58.745038+0000","last_clean":"2024-07-29T00:33:58.745038+0000","last_became_active":"2024-07-29T00:33:56.713282+0000","last_became_peered":"2024-07-29T00:33:56.713282+0000","last_unstale":"2024-07-29T00:33:58.745038+0000","last_undegraded":"2024-07-29T00:33:58.745038+0000","last_fullsized":"2024-07-29T00:33:58.745038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:27:40.038147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.686697+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:34:12.686697+0000","last_peered":"2024-07-29T00:34:12.686697+0000","last_clean":"2024-07-29T00:34:12.686697+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:34:12.686697+0000","last_undegraded":"2024-07-29T00:34:12.686697+0000","last_fullsized":"2024-07-29T00:34:12.686697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.750172+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:33:58.750172+0000","last_peered":"2024-07-29T00:33:58.750172+0000","last_clean":"2024-07-29T00:33:58.750172+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:33:58.750172+0000","last_undegraded":"2024-07-29T00:33:58.750172+0000","last_fullsized":"2024-07-29T00:33:58.750172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.682612+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:34:12.682612+0000","last_peered":"2024-07-29T00:34:12.682612+0000","last_clean":"2024-07-29T00:34:12.682612+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:34:12.682612+0000","last_undegraded":"2024-07-29T00:34:12.682612+0000","last_fullsized":"2024-07-29T00:34:12.682612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749331+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:33:58.749331+0000","last_peered":"2024-07-29T00:33:58.749331+0000","last_clean":"2024-07-29T00:33:58.749331+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:33:58.749331+0000","last_undegraded":"2024-07-29T00:33:58.749331+0000","last_fullsized":"2024-07-29T00:33:58.749331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:32.723575+0000","last_change":"2024-07-29T00:33:54.215904+0000","last_active":"2024-07-29T00:34:32.723575+0000","last_peered":"2024-07-29T00:34:32.723575+0000","last_clean":"2024-07-29T00:34:32.723575+0000","last_became_active":"2024-07-29T00:33:54.215431+0000","last_became_peered":"2024-07-29T00:33:54.215431+0000","last_unstale":"2024-07-29T00:34:32.723575+0000","last_undegraded":"2024-07-29T00:34:32.723575+0000","last_fullsized":"2024-07-29T00:34:32.723575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:44:33.563700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749432+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:33:58.749432+0000","last_peered":"2024-07-29T00:33:58.749432+0000","last_clean":"2024-07-29T00:33:58.749432+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:33:58.749432+0000","last_undegraded":"2024-07-29T00:33:58.749432+0000","last_fullsized":"2024-07-29T00:33:58.749432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.750107+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:33:58.750107+0000","last_peered":"2024-07-29T00:33:58.750107+0000","last_clean":"2024-07-29T00:33:58.750107+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:33:58.750107+0000","last_undegraded":"2024-07-29T00:33:58.750107+0000","last_fullsized":"2024-07-29T00:33:58.750107+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:13.692914+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:34:13.692914+0000","last_peered":"2024-07-29T00:34:13.692914+0000","last_clean":"2024-07-29T00:34:13.692914+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:34:13.692914+0000","last_undegraded":"2024-07-29T00:34:13.692914+0000","last_fullsized":"2024-07-29T00:34:13.692914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.753867+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:33:58.753867+0000","last_peered":"2024-07-29T00:33:58.753867+0000","last_clean":"2024-07-29T00:33:58.753867+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:33:58.753867+0000","last_undegraded":"2024-07-29T00:33:58.753867+0000","last_fullsized":"2024-07-29T00:33:58.753867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:10.656631+0000","last_change":"2024-07-29T00:33:53.680861+0000","last_active":"2024-07-29T00:34:10.656631+0000","last_peered":"2024-07-29T00:34:10.656631+0000","last_clean":"2024-07-29T00:34:10.656631+0000","last_became_active":"2024-07-29T00:33:53.680260+0000","last_became_peered":"2024-07-29T00:33:53.680260+0000","last_unstale":"2024-07-29T00:34:10.656631+0000","last_undegraded":"2024-07-29T00:34:10.656631+0000","last_fullsized":"2024-07-29T00:34:10.656631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:50:13.931513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747409+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:33:58.747409+0000","last_peered":"2024-07-29T00:33:58.747409+0000","last_clean":"2024-07-29T00:33:58.747409+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:33:58.747409+0000","last_undegraded":"2024-07-29T00:33:58.747409+0000","last_fullsized":"2024-07-29T00:33:58.747409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:32.743251+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:34:32.743251+0000","last_peered":"2024-07-29T00:34:32.743251+0000","last_clean":"2024-07-29T00:34:32.743251+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:34:32.743251+0000","last_undegraded":"2024-07-29T00:34:32.743251+0000","last_fullsized":"2024-07-29T00:34:32.743251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749292+0000","last_change":"2024-07-29T00:33:56.705254+0000","last_active":"2024-07-29T00:33:58.749292+0000","last_peered":"2024-07-29T00:33:58.749292+0000","last_clean":"2024-07-29T00:33:58.749292+0000","last_became_active":"2024-07-29T00:33:56.704900+0000","last_became_peered":"2024-07-29T00:33:56.704900+0000","last_unstale":"2024-07-29T00:33:58.749292+0000","last_undegraded":"2024-07-29T00:33:58.749292+0000","last_fullsized":"2024-07-29T00:33:58.749292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:38:00.364399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:30.729049+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:34:30.729049+0000","last_peered":"2024-07-29T00:34:30.729049+0000","last_clean":"2024-07-29T00:34:30.729049+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:34:30.729049+0000","last_undegraded":"2024-07-29T00:34:30.729049+0000","last_fullsized":"2024-07-29T00:34:30.729049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.744522+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:33:58.744522+0000","last_peered":"2024-07-29T00:33:58.744522+0000","last_clean":"2024-07-29T00:33:58.744522+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:33:58.744522+0000","last_undegraded":"2024-07-29T00:33:58.744522+0000","last_fullsized":"2024-07-29T00:33:58.744522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729646+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:33:58.729646+0000","last_peered":"2024-07-29T00:33:58.729646+0000","last_clean":"2024-07-29T00:33:58.729646+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:33:58.729646+0000","last_undegraded":"2024-07-29T00:33:58.729646+0000","last_fullsized":"2024-07-29T00:33:58.729646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:00.622853+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:34:00.622853+0000","last_peered":"2024-07-29T00:34:00.622853+0000","last_clean":"2024-07-29T00:34:00.622853+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:34:00.622853+0000","last_undegraded":"2024-07-29T00:34:00.622853+0000","last_fullsized":"2024-07-29T00:34:00.622853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.687334+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:34:12.687334+0000","last_peered":"2024-07-29T00:34:12.687334+0000","last_clean":"2024-07-29T00:34:12.687334+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:34:12.687334+0000","last_undegraded":"2024-07-29T00:34:12.687334+0000","last_fullsized":"2024-07-29T00:34:12.687334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748776+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:33:58.748776+0000","last_peered":"2024-07-29T00:33:58.748776+0000","last_clean":"2024-07-29T00:33:58.748776+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:33:58.748776+0000","last_undegraded":"2024-07-29T00:33:58.748776+0000","last_fullsized":"2024-07-29T00:33:58.748776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:28.748230+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:34:28.748230+0000","last_peered":"2024-07-29T00:34:28.748230+0000","last_clean":"2024-07-29T00:34:28.748230+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:34:28.748230+0000","last_undegraded":"2024-07-29T00:34:28.748230+0000","last_fullsized":"2024-07-29T00:34:28.748230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.752872+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:33:58.752872+0000","last_peered":"2024-07-29T00:33:58.752872+0000","last_clean":"2024-07-29T00:33:58.752872+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:33:58.752872+0000","last_undegraded":"2024-07-29T00:33:58.752872+0000","last_fullsized":"2024-07-29T00:33:58.752872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.687147+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:34:12.687147+0000","last_peered":"2024-07-29T00:34:12.687147+0000","last_clean":"2024-07-29T00:34:12.687147+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:34:12.687147+0000","last_undegraded":"2024-07-29T00:34:12.687147+0000","last_fullsized":"2024-07-29T00:34:12.687147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.748768+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:33:58.748768+0000","last_peered":"2024-07-29T00:33:58.748768+0000","last_clean":"2024-07-29T00:33:58.748768+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:33:58.748768+0000","last_undegraded":"2024-07-29T00:33:58.748768+0000","last_fullsized":"2024-07-29T00:33:58.748768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:09.657248+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:34:09.657248+0000","last_peered":"2024-07-29T00:34:09.657248+0000","last_clean":"2024-07-29T00:34:09.657248+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:34:09.657248+0000","last_undegraded":"2024-07-29T00:34:09.657248+0000","last_fullsized":"2024-07-29T00:34:09.657248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.729837+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:33:58.729837+0000","last_peered":"2024-07-29T00:33:58.729837+0000","last_clean":"2024-07-29T00:33:58.729837+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:33:58.729837+0000","last_undegraded":"2024-07-29T00:33:58.729837+0000","last_fullsized":"2024-07-29T00:33:58.729837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.686445+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:34:12.686445+0000","last_peered":"2024-07-29T00:34:12.686445+0000","last_clean":"2024-07-29T00:34:12.686445+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:34:12.686445+0000","last_undegraded":"2024-07-29T00:34:12.686445+0000","last_fullsized":"2024-07-29T00:34:12.686445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.745713+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:33:58.745713+0000","last_peered":"2024-07-29T00:33:58.745713+0000","last_clean":"2024-07-29T00:33:58.745713+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:33:58.745713+0000","last_undegraded":"2024-07-29T00:33:58.745713+0000","last_fullsized":"2024-07-29T00:33:58.745713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:11.670857+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:34:11.670857+0000","last_peered":"2024-07-29T00:34:11.670857+0000","last_clean":"2024-07-29T00:34:11.670857+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:34:11.670857+0000","last_undegraded":"2024-07-29T00:34:11.670857+0000","last_fullsized":"2024-07-29T00:34:11.670857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.751310+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:33:58.751310+0000","last_peered":"2024-07-29T00:33:58.751310+0000","last_clean":"2024-07-29T00:33:58.751310+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:33:58.751310+0000","last_undegraded":"2024-07-29T00:33:58.751310+0000","last_fullsized":"2024-07-29T00:33:58.751310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:09.656532+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:34:09.656532+0000","last_peered":"2024-07-29T00:34:09.656532+0000","last_clean":"2024-07-29T00:34:09.656532+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:34:09.656532+0000","last_undegraded":"2024-07-29T00:34:09.656532+0000","last_fullsized":"2024-07-29T00:34:09.656532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749450+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:33:58.749450+0000","last_peered":"2024-07-29T00:33:58.749450+0000","last_clean":"2024-07-29T00:33:58.749450+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:33:58.749450+0000","last_undegraded":"2024-07-29T00:33:58.749450+0000","last_fullsized":"2024-07-29T00:33:58.749450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.749392+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:33:58.749392+0000","last_peered":"2024-07-29T00:33:58.749392+0000","last_clean":"2024-07-29T00:33:58.749392+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:33:58.749392+0000","last_undegraded":"2024-07-29T00:33:58.749392+0000","last_fullsized":"2024-07-29T00:33:58.749392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:12.684466+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:34:12.684466+0000","last_peered":"2024-07-29T00:34:12.684466+0000","last_clean":"2024-07-29T00:34:12.684466+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:34:12.684466+0000","last_undegraded":"2024-07-29T00:34:12.684466+0000","last_fullsized":"2024-07-29T00:34:12.684466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747318+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:33:58.747318+0000","last_peered":"2024-07-29T00:33:58.747318+0000","last_clean":"2024-07-29T00:33:58.747318+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:33:58.747318+0000","last_undegraded":"2024-07-29T00:33:58.747318+0000","last_fullsized":"2024-07-29T00:33:58.747318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:13.693335+0000","last_change":"2024-07-29T00:33:53.679893+0000","last_active":"2024-07-29T00:34:13.693335+0000","last_peered":"2024-07-29T00:34:13.693335+0000","last_clean":"2024-07-29T00:34:13.693335+0000","last_became_active":"2024-07-29T00:33:53.679525+0000","last_became_peered":"2024-07-29T00:33:53.679525+0000","last_unstale":"2024-07-29T00:34:13.693335+0000","last_undegraded":"2024-07-29T00:34:13.693335+0000","last_fullsized":"2024-07-29T00:34:13.693335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:59.138923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.747793+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:33:58.747793+0000","last_peered":"2024-07-29T00:33:58.747793+0000","last_clean":"2024-07-29T00:33:58.747793+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:33:58.747793+0000","last_undegraded":"2024-07-29T00:33:58.747793+0000","last_fullsized":"2024-07-29T00:33:58.747793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.755603+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:33:58.755603+0000","last_peered":"2024-07-29T00:33:58.755603+0000","last_clean":"2024-07-29T00:33:58.755603+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:33:58.755603+0000","last_undegraded":"2024-07-29T00:33:58.755603+0000","last_fullsized":"2024-07-29T00:33:58.755603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:33:58.753406+0000","last_change":"2024-07-29T00:33:56.716344+0000","last_active":"2024-07-29T00:33:58.753406+0000","last_peered":"2024-07-29T00:33:58.753406+0000","last_clean":"2024-07-29T00:33:58.753406+0000","last_became_active":"2024-07-29T00:33:56.715873+0000","last_became_peered":"2024-07-29T00:33:56.715873+0000","last_unstale":"2024-07-29T00:33:58.753406+0000","last_undegraded":"2024-07-29T00:33:58.753406+0000","last_fullsized":"2024-07-29T00:33:58.753406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:58:08.050061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-29T00:34:30.728194+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:34:30.728194+0000","last_peered":"2024-07-29T00:34:30.728194+0000","last_clean":"2024-07-29T00:34:30.728194+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:34:30.728194+0000","last_undegraded":"2024-07-29T00:34:30.728194+0000","last_fullsized":"2024-07-29T00:34:30.728194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56921,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":225,"num_read_kb":173,"num_write":239,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":179277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247654,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416406,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":457800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1099328,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":458108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1050625,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495662,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":457447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757300,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1049913,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":457698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":457272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":512946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":512726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":458786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:34:37.793 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-29T00:34:37.793 INFO:tasks.cephadm:Setting up client nodes... 2024-07-29T00:34:37.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-29T00:34:38.243 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:38.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 -- 172.21.15.175:0/2475121178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 msgr2=0x7f8f10074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:38.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 --2- 172.21.15.175:0/2475121178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10074d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009900 tx=0x7f8ef802a480 comp rx=0 tx=0).stop 2024-07-29T00:34:38.905 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:38 smithi175 ceph-mon[32138]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:34:38.906 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:38 smithi175 ceph-mon[32138]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-07-29T00:34:38.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 -- 172.21.15.175:0/2475121178 shutdown_connections 2024-07-29T00:34:38.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 --2- 172.21.15.175:0/2475121178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f101095d0 0x7f8f1010ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:38.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 --2- 172.21.15.175:0/2475121178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10074d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 --2- 172.21.15.175:0/2475121178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f10075eb0 0x7f8f10074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 -- 172.21.15.175:0/2475121178 >> 172.21.15.175:0/2475121178 conn(0x7f8f100fcee0 msgr2=0x7f8f100ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.904+0000 7f8f15a00640 1 -- 172.21.15.175:0/2475121178 shutdown_connections 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.905+0000 7f8f15a00640 1 -- 172.21.15.175:0/2475121178 wait complete. 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.906+0000 7f8f15a00640 1 Processor -- start 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.906+0000 7f8f15a00640 1 -- start start 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.906+0000 7f8f15a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.906+0000 7f8f15a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f10075eb0 0x7f8f1019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f15a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f101095d0 0x7f8f101a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f15a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010f5b0 con 0x7f8f100748e0 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f15a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010f430 con 0x7f8f10075eb0 2024-07-29T00:34:38.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f15a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010f730 con 0x7f8f101095d0 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f10075eb0 0x7f8f1019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49280/0 (socket says 172.21.15.175:49280) 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 -- 172.21.15.175:0/3575801313 learned_addr learned my addr 172.21.15.175:0/3575801313 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f15000640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f101095d0 0x7f8f101a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 -- 172.21.15.175:0/3575801313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f101095d0 msgr2=0x7f8f101a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f101095d0 0x7f8f101a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 -- 172.21.15.175:0/3575801313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f10075eb0 msgr2=0x7f8f1019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f10075eb0 0x7f8f1019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:38.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.907+0000 7f8f0fe00640 1 -- 172.21.15.175:0/3575801313 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101a1810 con 0x7f8f100748e0 2024-07-29T00:34:38.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.908+0000 7f8f0fe00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10199be0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d950 tx=0x7f8f0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:38.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.908+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00017030 con 0x7f8f100748e0 2024-07-29T00:34:38.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.908+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00004020 con 0x7f8f100748e0 2024-07-29T00:34:38.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.908+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a1b00 con 0x7f8f100748e0 2024-07-29T00:34:38.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.908+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000047a0 con 0x7f8f100748e0 2024-07-29T00:34:38.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.908+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a2060 con 0x7f8f100748e0 2024-07-29T00:34:38.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.910+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f10108390 con 0x7f8f100748e0 2024-07-29T00:34:38.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.911+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8f000041d0 con 0x7f8f100748e0 2024-07-29T00:34:38.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.914+0000 7f8f0cc00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8ef00a2cc0 0x7f8ef00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:38.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.914+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8f000cea10 con 0x7f8f100748e0 2024-07-29T00:34:38.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.914+0000 7f8f0f400640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8ef00a2cc0 0x7f8ef00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:38.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.915+0000 7f8f0f400640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8ef00a2cc0 0x7f8ef00a5180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80314e0 tx=0x7f8ef8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:38.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:38.918+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8f0009b8c0 con 0x7f8f100748e0 2024-07-29T00:34:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:38 smithi181 ceph-mon[36706]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:34:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:38 smithi181 ceph-mon[36706]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-07-29T00:34:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:38 smithi204 ceph-mon[37368]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:34:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:38 smithi204 ceph-mon[37368]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-07-29T00:34:39.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.175+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8f1002d800 con 0x7f8f100748e0 2024-07-29T00:34:39.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.179+0000 7f8f0cc00640 1 -- 172.21.15.175:0/3575801313 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f8f0009b1c0 con 0x7f8f100748e0 2024-07-29T00:34:39.179 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-07-29T00:34:39.179 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCf46Zmf8aTChAA3IpduMWYLmr6e7Dl8F6Uwg== 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8ef00a2cc0 msgr2=0x7f8ef00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8ef00a2cc0 0x7f8ef00a5180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80314e0 tx=0x7f8ef8000bc0 comp rx=0 tx=0).stop 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 msgr2=0x7f8f10199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10199be0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d950 tx=0x7f8f0000de10 comp rx=0 tx=0).stop 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 shutdown_connections 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f8ef00a2cc0 0x7f8ef00a5180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f101095d0 0x7f8f101a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f10075eb0 0x7f8f1019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 --2- 172.21.15.175:0/3575801313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f100748e0 0x7f8f10199be0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.182+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 >> 172.21.15.175:0/3575801313 conn(0x7f8f100fcee0 msgr2=0x7f8f10105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.183+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 shutdown_connections 2024-07-29T00:34:39.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:39.183+0000 7f8f15a00640 1 -- 172.21.15.175:0/3575801313 wait complete. 2024-07-29T00:34:39.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:39 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3575801313' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-29T00:34:39.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:39 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3575801313' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-29T00:34:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:39 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3575801313' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-29T00:34:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:39 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3575801313' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-29T00:34:40.055 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:34:40.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-29T00:34:40.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-29T00:34:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:39 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3575801313' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-29T00:34:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:39 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3575801313' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-29T00:34:40.189 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-29T00:34:40.189 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-29T00:34:40.189 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph mgr dump --format=json 2024-07-29T00:34:40.693 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:40 smithi181 ceph-mon[36706]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-29T00:34:41.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:40 smithi175 ceph-mon[32138]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-29T00:34:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:40 smithi204 ceph-mon[37368]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-29T00:34:41.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 -- 172.21.15.175:0/1278119212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 msgr2=0x7eff34114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:41.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 --2- 172.21.15.175:0/1278119212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 0x7eff34114b90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009cd0 tx=0x7eff2c02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 -- 172.21.15.175:0/1278119212 shutdown_connections 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 --2- 172.21.15.175:0/1278119212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 0x7eff34114b90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 --2- 172.21.15.175:0/1278119212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff34108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 --2- 172.21.15.175:0/1278119212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3410dd50 0x7eff3410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 -- 172.21.15.175:0/1278119212 >> 172.21.15.175:0/1278119212 conn(0x7eff34077670 msgr2=0x7eff34079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 -- 172.21.15.175:0/1278119212 shutdown_connections 2024-07-29T00:34:41.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.354+0000 7eff33e00640 1 -- 172.21.15.175:0/1278119212 wait complete. 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.355+0000 7eff33e00640 1 Processor -- start 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.355+0000 7eff33e00640 1 -- start start 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.355+0000 7eff33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff3419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.355+0000 7eff33e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 0x7eff3419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff33e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3410dd50 0x7eff341a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff33e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff34118070 con 0x7eff34108670 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff33e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff34117ef0 con 0x7eff34107d10 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff33e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff341181f0 con 0x7eff3410dd50 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff3419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff2a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 0x7eff3419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:41.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff3419e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:40950/0 (socket says 172.21.15.175:40950) 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 -- 172.21.15.175:0/3024617913 learned_addr learned my addr 172.21.15.175:0/3024617913 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff33400640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3410dd50 0x7eff341a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 -- 172.21.15.175:0/3024617913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3410dd50 msgr2=0x7eff341a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3410dd50 0x7eff341a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 -- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 msgr2=0x7eff3419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 0x7eff3419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.356+0000 7eff32a00640 1 -- 172.21.15.175:0/3024617913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff341a34e0 con 0x7eff34107d10 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.357+0000 7eff32a00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff3419e530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e9f0 tx=0x7eff2400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.357+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2400cd80 con 0x7eff34107d10 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.357+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24005500 con 0x7eff34107d10 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.357+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff341a37d0 con 0x7eff34107d10 2024-07-29T00:34:41.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.357+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24010690 con 0x7eff34107d10 2024-07-29T00:34:41.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.357+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341ab0b0 con 0x7eff34107d10 2024-07-29T00:34:41.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.358+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff34108130 con 0x7eff34107d10 2024-07-29T00:34:41.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.360+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7eff240046c0 con 0x7eff34107d10 2024-07-29T00:34:41.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.361+0000 7eff2be00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff180a2c10 0x7eff180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:41.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.362+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7eff240cf4c0 con 0x7eff34107d10 2024-07-29T00:34:41.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.362+0000 7eff2a000640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff180a2c10 0x7eff180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:41.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.362+0000 7eff2a000640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff180a2c10 0x7eff180a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0095f0 tx=0x7eff1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:41.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.365+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7eff2409c370 con 0x7eff34107d10 2024-07-29T00:34:41.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.604+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7eff3419f890 con 0x7eff34107d10 2024-07-29T00:34:41.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.609+0000 7eff2be00640 1 -- 172.21.15.175:0/3024617913 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263209 (secure 0 0 0) 0x7eff2409bc70 con 0x7eff34107d10 2024-07-29T00:34:41.612 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:41.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff180a2c10 msgr2=0x7eff180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:41.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff180a2c10 0x7eff180a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0095f0 tx=0x7eff1c005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:41.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 msgr2=0x7eff3419e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:41.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff3419e530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e9f0 tx=0x7eff2400edf0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 shutdown_connections 2024-07-29T00:34:41.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7eff180a2c10 0x7eff180a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff3410dd50 0x7eff341a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff34108670 0x7eff3419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 --2- 172.21.15.175:0/3024617913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff34107d10 0x7eff3419e530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:41.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 >> 172.21.15.175:0/3024617913 conn(0x7eff34077670 msgr2=0x7eff34078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:41.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.616+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 shutdown_connections 2024-07-29T00:34:41.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:41.617+0000 7eff33e00640 1 -- 172.21.15.175:0/3024617913 wait complete. 2024-07-29T00:34:41.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:41 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3024617913' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-29T00:34:42.002 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:41 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3024617913' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-29T00:34:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:41 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3024617913' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-29T00:34:42.437 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3246788131},{"type":"v1","addr":"172.21.15.175:6801","nonce":3246788131}]},"active_addr":"172.21.15.175:6801/3246788131","active_change":"2024-07-29T00:26:49.412428+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.175:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":1189171033}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":4008532755}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":257505941}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2019123994}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3396809320}]}]} 2024-07-29T00:34:42.443 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-29T00:34:42.443 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-29T00:34:42.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:34:42.937 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:42 smithi181 ceph-mon[36706]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-07-29T00:34:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:42 smithi175 ceph-mon[32138]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-07-29T00:34:43.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:42 smithi204 ceph-mon[37368]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-07-29T00:34:43.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 -- 172.21.15.175:0/2709871041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0109a80 msgr2=0x7fbdd0109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:43.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2709871041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0109a80 0x7fbdd0109e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009900 tx=0x7fbdc002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 -- 172.21.15.175:0/2709871041 shutdown_connections 2024-07-29T00:34:43.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2709871041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdd0104420 0x7fbdd0110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2709871041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0103a80 0x7fbdd0103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2709871041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0109a80 0x7fbdd0109e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 -- 172.21.15.175:0/2709871041 >> 172.21.15.175:0/2709871041 conn(0x7fbdd00ff760 msgr2=0x7fbdd0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:43.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 -- 172.21.15.175:0/2709871041 shutdown_connections 2024-07-29T00:34:43.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.765+0000 7fbdd5000640 1 -- 172.21.15.175:0/2709871041 wait complete. 2024-07-29T00:34:43.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 Processor -- start 2024-07-29T00:34:43.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 -- start start 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 0x7fbdd019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0104420 0x7fbdd019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdd0109a80 0x7fbdd019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0113e80 con 0x7fbdd0104420 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0113d00 con 0x7fbdd0103a80 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.767+0000 7fbdd5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0114000 con 0x7fbdd0109a80 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 0x7fbdd019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 0x7fbdd019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:40974/0 (socket says 172.21.15.175:40974) 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 -- 172.21.15.175:0/2140939245 learned_addr learned my addr 172.21.15.175:0/2140939245 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdc5400640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0104420 0x7fbdd019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:43.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdce800640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdd0109a80 0x7fbdd019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 -- 172.21.15.175:0/2140939245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdd0109a80 msgr2=0x7fbdd019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdd0109a80 0x7fbdd019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 -- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0104420 msgr2=0x7fbdd019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0104420 0x7fbdd019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 -- 172.21.15.175:0/2140939245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd019f330 con 0x7fbdd0103a80 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdc5400640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0104420 0x7fbdd019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:34:43.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.768+0000 7fbdcde00640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 0x7fbdd019a380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006010 tx=0x7fbdc0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:43.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.769+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc003f070 con 0x7fbdd0103a80 2024-07-29T00:34:43.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.769+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc0031c30 con 0x7fbdd0103a80 2024-07-29T00:34:43.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.769+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd019f5c0 con 0x7fbdd0103a80 2024-07-29T00:34:43.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.769+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0035030 con 0x7fbdd0103a80 2024-07-29T00:34:43.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.769+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a6f00 con 0x7fbdd0103a80 2024-07-29T00:34:43.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.770+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd01051c0 con 0x7fbdd0103a80 2024-07-29T00:34:43.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.772+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbdc0042070 con 0x7fbdd0103a80 2024-07-29T00:34:43.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.773+0000 7fbdc7400640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbdb40a2c90 0x7fbdb40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:43.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.774+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fbdc00efc90 con 0x7fbdd0103a80 2024-07-29T00:34:43.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.774+0000 7fbdc5400640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbdb40a2c90 0x7fbdb40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:43.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.774+0000 7fbdc5400640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbdb40a2c90 0x7fbdb40a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdd01090e0 tx=0x7fbdb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:43.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.775+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbdc00bcb40 con 0x7fbdd0103a80 2024-07-29T00:34:43.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.969+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdd010bff0 con 0x7fbdd0103a80 2024-07-29T00:34:43.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.971+0000 7fbdc7400640 1 -- 172.21.15.175:0/2140939245 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22263 (secure 0 0 0) 0x7fbdc00bc440 con 0x7fbdd0103a80 2024-07-29T00:34:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:43.972 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":87,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:33:58.041543+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:34:43.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbdb40a2c90 msgr2=0x7fbdb40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:43.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbdb40a2c90 0x7fbdb40a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdd01090e0 tx=0x7fbdb8005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 msgr2=0x7fbdd019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 0x7fbdd019a380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006010 tx=0x7fbdc0003c10 comp rx=0 tx=0).stop 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 shutdown_connections 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbdb40a2c90 0x7fbdb40a5150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdd0109a80 0x7fbdd019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd0104420 0x7fbdd019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 --2- 172.21.15.175:0/2140939245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd0103a80 0x7fbdd019a380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 >> 172.21.15.175:0/2140939245 conn(0x7fbdd00ff760 msgr2=0x7fbdd0101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.975+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 shutdown_connections 2024-07-29T00:34:43.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:43.976+0000 7fbdd5000640 1 -- 172.21.15.175:0/2140939245 wait complete. 2024-07-29T00:34:44.829 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-29T00:34:44.829 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:34:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:44 smithi181 ceph-mon[36706]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-29T00:34:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:44 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2140939245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:44 smithi175 ceph-mon[32138]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-29T00:34:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:44 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2140939245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:45.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:44 smithi204 ceph-mon[37368]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-29T00:34:45.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:44 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2140939245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:45.284 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 -- 172.21.15.175:0/3454701638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c810dd50 msgr2=0x7fd9c810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 --2- 172.21.15.175:0/3454701638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c810dd50 0x7fd9c810e130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009890 tx=0x7fd9b802a360 comp rx=0 tx=0).stop 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 -- 172.21.15.175:0/3454701638 shutdown_connections 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 --2- 172.21.15.175:0/3454701638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8108670 0x7fd9c8114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 --2- 172.21.15.175:0/3454701638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c8107d10 0x7fd9c8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 --2- 172.21.15.175:0/3454701638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c810dd50 0x7fd9c810e130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:45.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.931+0000 7fd9cd400640 1 -- 172.21.15.175:0/3454701638 >> 172.21.15.175:0/3454701638 conn(0x7fd9c8077670 msgr2=0x7fd9c8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:45.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.932+0000 7fd9cd400640 1 -- 172.21.15.175:0/3454701638 shutdown_connections 2024-07-29T00:34:45.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.932+0000 7fd9cd400640 1 -- 172.21.15.175:0/3454701638 wait complete. 2024-07-29T00:34:45.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.933+0000 7fd9cd400640 1 Processor -- start 2024-07-29T00:34:45.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.933+0000 7fd9cd400640 1 -- start start 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8107d10 0x7fd9c819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8108670 0x7fd9c819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 0x7fd9c81a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cd400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8118060 con 0x7fd9c810dd50 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cd400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8117ee0 con 0x7fd9c8107d10 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cd400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81181e0 con 0x7fd9c8108670 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 0x7fd9c81a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 0x7fd9c81a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49330/0 (socket says 172.21.15.175:49330) 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cca00640 1 -- 172.21.15.175:0/432578758 learned_addr learned my addr 172.21.15.175:0/432578758 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9c7e00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8107d10 0x7fd9c819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9bf400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8108670 0x7fd9c819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cca00640 1 -- 172.21.15.175:0/432578758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8108670 msgr2=0x7fd9c819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cca00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8108670 0x7fd9c819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.934+0000 7fd9cca00640 1 -- 172.21.15.175:0/432578758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8107d10 msgr2=0x7fd9c819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9cca00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8107d10 0x7fd9c819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9cca00640 1 -- 172.21.15.175:0/432578758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c81a3510 con 0x7fd9c810dd50 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9c7e00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8107d10 0x7fd9c819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9cca00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 0x7fd9c81a2e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000ea60 tx=0x7fd9c000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000cd80 con 0x7fd9c810dd50 2024-07-29T00:34:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c0005020 con 0x7fd9c810dd50 2024-07-29T00:34:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81a3800 con 0x7fd9c810dd50 2024-07-29T00:34:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0010690 con 0x7fd9c810dd50 2024-07-29T00:34:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.935+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81ab0e0 con 0x7fd9c810dd50 2024-07-29T00:34:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.937+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c8108130 con 0x7fd9c810dd50 2024-07-29T00:34:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.939+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd9c00051d0 con 0x7fd9c810dd50 2024-07-29T00:34:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.940+0000 7fd9c5600640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9ac0a2c10 0x7fd9ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.940+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd9c00cf620 con 0x7fd9c810dd50 2024-07-29T00:34:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.940+0000 7fd9c7e00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9ac0a2c10 0x7fd9ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.941+0000 7fd9c7e00640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9ac0a2c10 0x7fd9ac0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9b802a950 tx=0x7fd9b80056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:45.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:45.945+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd9c009c4d0 con 0x7fd9c810dd50 2024-07-29T00:34:46.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.134+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9c810fc50 con 0x7fd9c810dd50 2024-07-29T00:34:46.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.139+0000 7fd9c5600640 1 -- 172.21.15.175:0/432578758 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22263 (secure 0 0 0) 0x7fd9c009bdd0 con 0x7fd9c810dd50 2024-07-29T00:34:46.140 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:34:46.140 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":87,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:33:58.041543+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9ac0a2c10 msgr2=0x7fd9ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9ac0a2c10 0x7fd9ac0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9b802a950 tx=0x7fd9b80056d0 comp rx=0 tx=0).stop 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 msgr2=0x7fd9c81a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 0x7fd9c81a2e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000ea60 tx=0x7fd9c000ee60 comp rx=0 tx=0).stop 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 shutdown_connections 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9c810dd50 0x7fd9c81a2e30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9ac0a2c10 0x7fd9ac0a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8108670 0x7fd9c819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:46.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 --2- 172.21.15.175:0/432578758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c8107d10 0x7fd9c819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:46.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.142+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 >> 172.21.15.175:0/432578758 conn(0x7fd9c8077670 msgr2=0x7fd9c8078f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:46.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.143+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 shutdown_connections 2024-07-29T00:34:46.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:46.143+0000 7fd9cd400640 1 -- 172.21.15.175:0/432578758 wait complete. 2024-07-29T00:34:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:46 smithi175 ceph-mon[32138]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-07-29T00:34:46.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:46 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/432578758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:46 smithi181 ceph-mon[36706]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-07-29T00:34:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:46 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/432578758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:47.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-29T00:34:47.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-29T00:34:47.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-29T00:34:47.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-29T00:34:47.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-29T00:34:47.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-29T00:34:47.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-07-29T00:34:47.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-07-29T00:34:47.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-07-29T00:34:47.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-07-29T00:34:47.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-07-29T00:34:47.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-07-29T00:34:47.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:46 smithi204 ceph-mon[37368]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-07-29T00:34:47.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:46 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/432578758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:34:47.992 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:48.009 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:48.091 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:48.777 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:48 smithi175 ceph-mon[32138]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-07-29T00:34:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:48 smithi181 ceph-mon[36706]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-07-29T00:34:49.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:48 smithi204 ceph-mon[37368]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-07-29T00:34:49.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 -- 172.21.15.175:0/1152947571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 msgr2=0x7f5054107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 --2- 172.21.15.175:0/1152947571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f5054107a90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5048009930 tx=0x7f504802a440 comp rx=0 tx=0).stop 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 -- 172.21.15.175:0/1152947571 shutdown_connections 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 --2- 172.21.15.175:0/1152947571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5054102050 0x7f505410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 --2- 172.21.15.175:0/1152947571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50541016b0 0x7f5054101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 --2- 172.21.15.175:0/1152947571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f5054107a90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.130+0000 7f505ac00640 1 -- 172.21.15.175:0/1152947571 >> 172.21.15.175:0/1152947571 conn(0x7f50540fd3d0 msgr2=0x7f50540ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.131+0000 7f505ac00640 1 -- 172.21.15.175:0/1152947571 shutdown_connections 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.131+0000 7f505ac00640 1 -- 172.21.15.175:0/1152947571 wait complete. 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.131+0000 7f505ac00640 1 Processor -- start 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 -- start start 2024-07-29T00:34:49.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50541016b0 0x7f505419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5054102050 0x7f505419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f505419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5054111b00 con 0x7f50541076b0 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5054111980 con 0x7f5054102050 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5054111c80 con 0x7f50541016b0 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f505419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f505419ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33374/0 (socket says 172.21.15.175:33374) 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f505a200640 1 -- 172.21.15.175:0/747808150 learned_addr learned my addr 172.21.15.175:0/747808150 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f5058e00640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5054102050 0x7f505419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.132+0000 7f5059800640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50541016b0 0x7f505419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505a200640 1 -- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50541016b0 msgr2=0x7f505419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505a200640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50541016b0 0x7f505419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505a200640 1 -- 172.21.15.175:0/747808150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5054102050 msgr2=0x7f505419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505a200640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5054102050 0x7f505419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505a200640 1 -- 172.21.15.175:0/747808150 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505419f160 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505a200640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f505419ea80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f504c00eaa0 tx=0x7f504c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c00cdb0 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c005020 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c0116b0 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505419f450 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.133+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50541a6d30 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.134+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50540fe450 con 0x7f50541076b0 2024-07-29T00:34:49.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.136+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f504c005560 con 0x7f50541076b0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5052000640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f503c0a2c90 0x7f503c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f504c0cf600 con 0x7f50541076b0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5059800640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f503c0a2c90 0x7f503c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5052000640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] conn(0x7f503c0ac4e0 0x7f503c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 --> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f504c014ec0 con 0x7f503c0ac4e0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f504c0cf9b0 con 0x7f50541076b0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.138+0000 7f5058e00640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] conn(0x7f503c0ac4e0 0x7f503c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.139+0000 7f5059800640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f503c0a2c90 0x7f503c0a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5048006010 tx=0x7f5048000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.139+0000 7f5058e00640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] conn(0x7f503c0ac4e0 0x7f503c0ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.140+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== osd.2 v2:172.21.15.204:6800/3684418621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f504c014ec0 con 0x7f503c0ac4e0 2024-07-29T00:34:49.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.154+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 --> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5054102df0 con 0x7f503c0ac4e0 2024-07-29T00:34:49.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f5052000640 1 -- 172.21.15.175:0/747808150 <== osd.2 v2:172.21.15.204:6800/3684418621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5054102df0 con 0x7f503c0ac4e0 2024-07-29T00:34:49.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] conn(0x7f503c0ac4e0 msgr2=0x7f503c0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] conn(0x7f503c0ac4e0 0x7f503c0ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f503c0a2c90 msgr2=0x7f503c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f503c0a2c90 0x7f503c0a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5048006010 tx=0x7f5048000bc0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 msgr2=0x7f505419ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 --2- 172.21.15.175:0/747808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50541076b0 0x7f505419ea80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f504c00eaa0 tx=0x7f504c00ef60 comp rx=0 tx=0).stop 2024-07-29T00:34:49.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505a200640 1 -- 172.21.15.175:0/747808150 reap_dead start 2024-07-29T00:34:49.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 shutdown_connections 2024-07-29T00:34:49.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.155+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 >> 172.21.15.175:0/747808150 conn(0x7f50540fd3d0 msgr2=0x7f50540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:49.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.156+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 shutdown_connections 2024-07-29T00:34:49.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.156+0000 7f505ac00640 1 -- 172.21.15.175:0/747808150 wait complete. 2024-07-29T00:34:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 -- 172.21.15.175:0/2708630991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 msgr2=0x7f6c3c108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 --2- 172.21.15.175:0/2708630991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c108520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009900 tx=0x7f6c2802a410 comp rx=0 tx=0).stop 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 -- 172.21.15.175:0/2708630991 shutdown_connections 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 --2- 172.21.15.175:0/2708630991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c10d220 0x7f6c3c114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 --2- 172.21.15.175:0/2708630991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c3c108af0 0x7f6c3c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 --2- 172.21.15.175:0/2708630991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c108520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 -- 172.21.15.175:0/2708630991 >> 172.21.15.175:0/2708630991 conn(0x7f6c3c077890 msgr2=0x7f6c3c079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:49.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 -- 172.21.15.175:0/2708630991 shutdown_connections 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:48.999+0000 7f6c41600640 1 -- 172.21.15.175:0/2708630991 wait complete. 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.000+0000 7f6c41600640 1 Processor -- start 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.000+0000 7f6c41600640 1 -- start start 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c41600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c41600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c3c108af0 0x7f6c3c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c41600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c10d220 0x7f6c3c1a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c41600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1182a0 con 0x7f6c3c108140 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c41600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c118120 con 0x7f6c3c108af0 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c41600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c118420 con 0x7f6c3c10d220 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c3aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c3aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33362/0 (socket says 172.21.15.175:33362) 2024-07-29T00:34:49.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c3aa00640 1 -- 172.21.15.175:0/752367905 learned_addr learned my addr 172.21.15.175:0/752367905 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c3a000640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c3c108af0 0x7f6c3c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.001+0000 7f6c3b400640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c10d220 0x7f6c3c1a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c3aa00640 1 -- 172.21.15.175:0/752367905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c10d220 msgr2=0x7f6c3c1a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c3aa00640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c10d220 0x7f6c3c1a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c3aa00640 1 -- 172.21.15.175:0/752367905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c3c108af0 msgr2=0x7f6c3c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c3aa00640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c3c108af0 0x7f6c3c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c3aa00640 1 -- 172.21.15.175:0/752367905 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1a26f0 con 0x7f6c3c108140 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c3aa00640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c19e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c28004770 tx=0x7f6c28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2803f070 con 0x7f6c3c108140 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28031ca0 con 0x7f6c3c108140 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1a2980 con 0x7f6c3c108140 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28035030 con 0x7f6c3c108140 2024-07-29T00:34:49.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.002+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1ab350 con 0x7f6c3c108140 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.003+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c3c062570 con 0x7f6c3c108140 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.005+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6c28042070 con 0x7f6c3c108140 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c33400640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6c200a2c10 0x7f6c200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6c280efca0 con 0x7f6c3c108140 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c3a000640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6c200a2c10 0x7f6c200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c33400640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] conn(0x7f6c200ac460 0x7f6c200ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 --> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c28038e90 con 0x7f6c200ac460 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6c280f0050 con 0x7f6c3c108140 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c3b400640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] conn(0x7f6c200ac460 0x7f6c200ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.007+0000 7f6c3a000640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6c200a2c10 0x7f6c200a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0095f0 tx=0x7f6c2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:49.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.008+0000 7f6c3b400640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] conn(0x7f6c200ac460 0x7f6c200ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:49.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.009+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== osd.6 v2:172.21.15.175:6818/3660975891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c28038e90 con 0x7f6c200ac460 2024-07-29T00:34:49.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.024+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 --> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c3c062970 con 0x7f6c200ac460 2024-07-29T00:34:49.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.024+0000 7f6c33400640 1 -- 172.21.15.175:0/752367905 <== osd.6 v2:172.21.15.175:6818/3660975891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6c3c062970 con 0x7f6c200ac460 2024-07-29T00:34:49.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] conn(0x7f6c200ac460 msgr2=0x7f6c200ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] conn(0x7f6c200ac460 0x7f6c200ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6c200a2c10 msgr2=0x7f6c200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6c200a2c10 0x7f6c200a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0095f0 tx=0x7f6c2c005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 msgr2=0x7f6c3c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 --2- 172.21.15.175:0/752367905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c3c108140 0x7f6c3c19e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c28004770 tx=0x7f6c28003c10 comp rx=0 tx=0).stop 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c3b400640 1 -- 172.21.15.175:0/752367905 reap_dead start 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 shutdown_connections 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 >> 172.21.15.175:0/752367905 conn(0x7f6c3c077890 msgr2=0x7f6c3c078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 shutdown_connections 2024-07-29T00:34:49.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:49.025+0000 7f6c41600640 1 -- 172.21.15.175:0/752367905 wait complete. 2024-07-29T00:34:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-07-29T00:34:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-07-29T00:34:50.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:49 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-07-29T00:34:50.023 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-07-29T00:34:50.023 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-07-29T00:34:50.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:49 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-07-29T00:34:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-07-29T00:34:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-07-29T00:34:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-07-29T00:34:50.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:49 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-07-29T00:34:50.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 -- 172.21.15.175:0/2136120059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100470 msgr2=0x7f2d18100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 --2- 172.21.15.175:0/2136120059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100470 0x7f2d18100850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009900 tx=0x7f2d0c02a410 comp rx=0 tx=0).stop 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 -- 172.21.15.175:0/2136120059 shutdown_connections 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 --2- 172.21.15.175:0/2136120059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d1810e020 0x7f2d18110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 --2- 172.21.15.175:0/2136120059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d18100e20 0x7f2d1810dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 --2- 172.21.15.175:0/2136120059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100470 0x7f2d18100850 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 -- 172.21.15.175:0/2136120059 >> 172.21.15.175:0/2136120059 conn(0x7f2d180fc300 msgr2=0x7f2d180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 -- 172.21.15.175:0/2136120059 shutdown_connections 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.299+0000 7f2d1e000640 1 -- 172.21.15.175:0/2136120059 wait complete. 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.300+0000 7f2d1e000640 1 Processor -- start 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.300+0000 7f2d1e000640 1 -- start start 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d18100470 0x7f2d1819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100e20 0x7f2d1819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1810e020 0x7f2d1819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113d20 con 0x7f2d1810e020 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113ba0 con 0x7f2d18100470 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113ea0 con 0x7f2d18100e20 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1810e020 0x7f2d1819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100e20 0x7f2d1819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100e20 0x7f2d1819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:43908/0 (socket says 172.21.15.175:43908) 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d18100470 0x7f2d1819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d1d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1810e020 0x7f2d1819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33396/0 (socket says 172.21.15.175:33396) 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 -- 172.21.15.175:0/1824274217 learned_addr learned my addr 172.21.15.175:0/1824274217 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:50.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 -- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d18100470 msgr2=0x7f2d1819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d18100470 0x7f2d1819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 -- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1810e020 msgr2=0x7f2d1819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d1810e020 0x7f2d1819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.301+0000 7f2d17e00640 1 -- 172.21.15.175:0/1824274217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1819f150 con 0x7f2d18100e20 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.302+0000 7f2d17e00640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100e20 0x7f2d1819a6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400ea60 tx=0x7f2d0400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.302+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400cde0 con 0x7f2d18100e20 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.302+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1819f440 con 0x7f2d18100e20 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.302+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d040035f0 con 0x7f2d18100e20 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.302+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04010670 con 0x7f2d18100e20 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.302+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a6d20 con 0x7f2d18100e20 2024-07-29T00:34:50.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.303+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d18062670 con 0x7f2d18100e20 2024-07-29T00:34:50.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.306+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2d040037a0 con 0x7f2d18100e20 2024-07-29T00:34:50.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.307+0000 7f2d15600640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f2d000a2c90 0x7f2d000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.307+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f2d040cf8a0 con 0x7f2d18100e20 2024-07-29T00:34:50.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.308+0000 7f2d1cc00640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f2d000a2c90 0x7f2d000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.308+0000 7f2d15600640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] conn(0x7f2d000ac940 0x7f2d000aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.308+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 --> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d0401ee70 con 0x7f2d000ac940 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.308+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2d040cfc50 con 0x7f2d18100e20 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.308+0000 7f2d1d600640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] conn(0x7f2d000ac940 0x7f2d000aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.308+0000 7f2d1cc00640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f2d000a2c90 0x7f2d000a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000c00 tx=0x7f2d0c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.309+0000 7f2d1d600640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] conn(0x7f2d000ac940 0x7f2d000aede0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.310+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== osd.1 v2:172.21.15.181:6800/2146535950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d0401ee70 con 0x7f2d000ac940 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.325+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 --> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d18062a70 con 0x7f2d000ac940 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.325+0000 7f2d15600640 1 -- 172.21.15.175:0/1824274217 <== osd.1 v2:172.21.15.181:6800/2146535950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d18062a70 con 0x7f2d000ac940 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] conn(0x7f2d000ac940 msgr2=0x7f2d000aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] conn(0x7f2d000ac940 0x7f2d000aede0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f2d000a2c90 msgr2=0x7f2d000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f2d000a2c90 0x7f2d000a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000c00 tx=0x7f2d0c005b50 comp rx=0 tx=0).stop 2024-07-29T00:34:50.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100e20 msgr2=0x7f2d1819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 --2- 172.21.15.175:0/1824274217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d18100e20 0x7f2d1819a6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400ea60 tx=0x7f2d0400ef20 comp rx=0 tx=0).stop 2024-07-29T00:34:50.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1d600640 1 -- 172.21.15.175:0/1824274217 reap_dead start 2024-07-29T00:34:50.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 shutdown_connections 2024-07-29T00:34:50.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 >> 172.21.15.175:0/1824274217 conn(0x7f2d180fc300 msgr2=0x7f2d180fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:50.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 shutdown_connections 2024-07-29T00:34:50.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.326+0000 7f2d1e000640 1 -- 172.21.15.175:0/1824274217 wait complete. 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 -- 172.21.15.175:0/3193833764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 msgr2=0x7f31f4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 --2- 172.21.15.175:0/3193833764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 0x7f31f4107aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009860 tx=0x7f31e402a340 comp rx=0 tx=0).stop 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 -- 172.21.15.175:0/3193833764 shutdown_connections 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 --2- 172.21.15.175:0/3193833764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f4102060 0x7f31f410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 --2- 172.21.15.175:0/3193833764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f41016c0 0x7f31f4101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 --2- 172.21.15.175:0/3193833764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 0x7f31f4107aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 -- 172.21.15.175:0/3193833764 >> 172.21.15.175:0/3193833764 conn(0x7f31f40fd400 msgr2=0x7f31f40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:50.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 -- 172.21.15.175:0/3193833764 shutdown_connections 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.378+0000 7f31f3200640 1 -- 172.21.15.175:0/3193833764 wait complete. 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 Processor -- start 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 -- start start 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f41016c0 0x7f31f40741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f4102060 0x7f31f4074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 0x7f31f4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111ac0 con 0x7f31f4102060 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111940 con 0x7f31f41076c0 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.380+0000 7f31f3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111c40 con 0x7f31f41016c0 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f4102060 0x7f31f4074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f4102060 0x7f31f4074720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33420/0 (socket says 172.21.15.175:33420) 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1400640 1 -- 172.21.15.175:0/2089366673 learned_addr learned my addr 172.21.15.175:0/2089366673 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:50.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f41016c0 0x7f31f40741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f2800640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 0x7f31f4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 -- 172.21.15.175:0/2089366673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 msgr2=0x7f31f4074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f41076c0 0x7f31f4074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 -- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f4102060 msgr2=0x7f31f4074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f4102060 0x7f31f4074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 -- 172.21.15.175:0/2089366673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f41a6430 con 0x7f31f41016c0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1400640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f4102060 0x7f31f4074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.381+0000 7f31f1e00640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f41016c0 0x7f31f40741e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31e402a860 tx=0x7f31e4033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.382+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4037040 con 0x7f31f41016c0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.382+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e4033d50 con 0x7f31f41016c0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.382+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e40039e0 con 0x7f31f41016c0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.382+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a66c0 con 0x7f31f41016c0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.382+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a6b50 con 0x7f31f41016c0 2024-07-29T00:34:50.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.383+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31f40625f0 con 0x7f31f41016c0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.385+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f31e4042020 con 0x7f31f41016c0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.387+0000 7f31eaa00640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f31d80a2c90 0x7f31d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.387+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f31e40eeff0 con 0x7f31f41016c0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.387+0000 7f31f1400640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f31d80a2c90 0x7f31d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.387+0000 7f31eaa00640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] conn(0x7f31d80ac810 0x7f31d80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.387+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 --> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31d80af380 con 0x7f31d80ac810 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.387+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f31e40ef3a0 con 0x7f31f41016c0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.388+0000 7f31f2800640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] conn(0x7f31d80ac810 0x7f31d80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.388+0000 7f31f1400640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f31d80a2c90 0x7f31d80a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31e00096c0 tx=0x7f31e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.388+0000 7f31f2800640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] conn(0x7f31d80ac810 0x7f31d80aecb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.389+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== osd.11 v2:172.21.15.204:6824/945517137 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31d80af380 con 0x7f31d80ac810 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.404+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 --> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31f4062ac0 con 0x7f31d80ac810 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.404+0000 7f31eaa00640 1 -- 172.21.15.175:0/2089366673 <== osd.11 v2:172.21.15.204:6824/945517137 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f31f4062ac0 con 0x7f31d80ac810 2024-07-29T00:34:50.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] conn(0x7f31d80ac810 msgr2=0x7f31d80aecb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] conn(0x7f31d80ac810 0x7f31d80aecb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f31d80a2c90 msgr2=0x7f31d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f31d80a2c90 0x7f31d80a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31e00096c0 tx=0x7f31e0009260 comp rx=0 tx=0).stop 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f41016c0 msgr2=0x7f31f40741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 --2- 172.21.15.175:0/2089366673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f41016c0 0x7f31f40741e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31e402a860 tx=0x7f31e4033b60 comp rx=0 tx=0).stop 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f2800640 1 -- 172.21.15.175:0/2089366673 reap_dead start 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 shutdown_connections 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 >> 172.21.15.175:0/2089366673 conn(0x7f31f40fd400 msgr2=0x7f31f40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 shutdown_connections 2024-07-29T00:34:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:50.405+0000 7f31f3200640 1 -- 172.21.15.175:0/2089366673 wait complete. 2024-07-29T00:34:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-29T00:34:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-07-29T00:34:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-07-29T00:34:51.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:50 smithi181 ceph-mon[36706]: osdmap e88: 12 total, 12 up, 12 in 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-07-29T00:34:51.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-07-29T00:34:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-07-29T00:34:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-07-29T00:34:51.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:50 smithi175 ceph-mon[32138]: osdmap e88: 12 total, 12 up, 12 in 2024-07-29T00:34:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-29T00:34:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-07-29T00:34:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-07-29T00:34:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-07-29T00:34:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-07-29T00:34:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-07-29T00:34:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-07-29T00:34:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-07-29T00:34:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-07-29T00:34:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-07-29T00:34:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-07-29T00:34:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:50 smithi204 ceph-mon[37368]: osdmap e88: 12 total, 12 up, 12 in 2024-07-29T00:34:51.627 INFO:teuthology.orchestra.run.smithi175.stdout:339302416410 2024-07-29T00:34:51.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-07-29T00:34:51.942 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:51 smithi175 ceph-mon[32138]: osdmap e89: 12 total, 12 up, 12 in 2024-07-29T00:34:51.942 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:34:51.943 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:51 smithi175 ceph-mon[32138]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-07-29T00:34:51.974 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:51.976 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:51.977 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:51 smithi181 ceph-mon[36706]: osdmap e89: 12 total, 12 up, 12 in 2024-07-29T00:34:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:34:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:51 smithi181 ceph-mon[36706]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-07-29T00:34:52.004 INFO:teuthology.orchestra.run.smithi175.stdout:55834574923 2024-07-29T00:34:52.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-29T00:34:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:51 smithi204 ceph-mon[37368]: osdmap e89: 12 total, 12 up, 12 in 2024-07-29T00:34:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:34:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:51 smithi204 ceph-mon[37368]: pgmap v320: 97 pgs: 4 peering, 93 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-07-29T00:34:52.116 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:52.117 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:52.125 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:52.126 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:52.129 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:34:52.532 INFO:teuthology.orchestra.run.smithi175.stdout:77309411400 2024-07-29T00:34:52.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-29T00:34:53.401 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.401 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.401 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.401 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:53 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:53 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:53 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:34:54.349 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:54 smithi175 ceph-mon[32138]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-07-29T00:34:54.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 -- 172.21.15.175:0/373722050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c108670 msgr2=0x7fbb7c114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:54.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 --2- 172.21.15.175:0/373722050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c108670 0x7fbb7c114b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009cd0 tx=0x7fbb7402a8f0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 -- 172.21.15.175:0/373722050 shutdown_connections 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 --2- 172.21.15.175:0/373722050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c108670 0x7fbb7c114b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 --2- 172.21.15.175:0/373722050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb7c107d10 0x7fbb7c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 --2- 172.21.15.175:0/373722050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c10dd50 0x7fbb7c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 -- 172.21.15.175:0/373722050 >> 172.21.15.175:0/373722050 conn(0x7fbb7c077670 msgr2=0x7fbb7c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.470+0000 7fbb7be00640 1 -- 172.21.15.175:0/373722050 shutdown_connections 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.471+0000 7fbb7be00640 1 -- 172.21.15.175:0/373722050 wait complete. 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.471+0000 7fbb7be00640 1 Processor -- start 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.471+0000 7fbb7be00640 1 -- start start 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c107d10 0x7fbb7c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb7c108670 0x7fbb7c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c10dd50 0x7fbb7c1a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c118080 con 0x7fbb7c107d10 2024-07-29T00:34:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c117f00 con 0x7fbb7c10dd50 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c118200 con 0x7fbb7c108670 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c107d10 0x7fbb7c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c107d10 0x7fbb7c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33462/0 (socket says 172.21.15.175:33462) 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7aa00640 1 -- 172.21.15.175:0/2675835492 learned_addr learned my addr 172.21.15.175:0/2675835492 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7a000640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb7c108670 0x7fbb7c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.472+0000 7fbb7b400640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c10dd50 0x7fbb7c1a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7aa00640 1 -- 172.21.15.175:0/2675835492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb7c108670 msgr2=0x7fbb7c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7aa00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb7c108670 0x7fbb7c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7aa00640 1 -- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c10dd50 msgr2=0x7fbb7c1a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7aa00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c10dd50 0x7fbb7c1a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7aa00640 1 -- 172.21.15.175:0/2675835492 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c1a3510 con 0x7fbb7c107d10 2024-07-29T00:34:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7a000640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb7c108670 0x7fbb7c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7b400640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb7c10dd50 0x7fbb7c1a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7aa00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c107d10 0x7fbb7c19e560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00e9f0 tx=0x7fbb6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00cd80 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6c010910 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c018990 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c1a3800 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.473+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1ab0e0 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.474+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb7c062630 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.476+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbb6c010ac0 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.477+0000 7fbb80a00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbb600a2c10 0x7fbb600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fbb6c0cfcc0 con 0x7fbb7c107d10 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb7a000640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbb600a2c10 0x7fbb600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb80a00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] conn(0x7fbb600ac8c0 0x7fbb600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 --> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb6c010760 con 0x7fbb600ac8c0 2024-07-29T00:34:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbb6c0d0070 con 0x7fbb7c107d10 2024-07-29T00:34:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb7b400640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] conn(0x7fbb600ac8c0 0x7fbb600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.478+0000 7fbb7a000640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbb600a2c10 0x7fbb600a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c10d440 tx=0x7fbb64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.479+0000 7fbb7b400640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] conn(0x7fbb600ac8c0 0x7fbb600aed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.480+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== osd.7 v2:172.21.15.181:6816/591220240 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb6c010760 con 0x7fbb600ac8c0 2024-07-29T00:34:54.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.495+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 --> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb7c062ab0 con 0x7fbb600ac8c0 2024-07-29T00:34:54.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.496+0000 7fbb80a00640 1 -- 172.21.15.175:0/2675835492 <== osd.7 v2:172.21.15.181:6816/591220240 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb7c062ab0 con 0x7fbb600ac8c0 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] conn(0x7fbb600ac8c0 msgr2=0x7fbb600aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] conn(0x7fbb600ac8c0 0x7fbb600aed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbb600a2c10 msgr2=0x7fbb600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fbb600a2c10 0x7fbb600a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c10d440 tx=0x7fbb64005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c107d10 msgr2=0x7fbb7c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 --2- 172.21.15.175:0/2675835492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb7c107d10 0x7fbb7c19e560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00e9f0 tx=0x7fbb6c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:34:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7b400640 1 -- 172.21.15.175:0/2675835492 reap_dead start 2024-07-29T00:34:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 shutdown_connections 2024-07-29T00:34:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 >> 172.21.15.175:0/2675835492 conn(0x7fbb7c077670 msgr2=0x7fbb7c078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.497+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 shutdown_connections 2024-07-29T00:34:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.498+0000 7fbb7be00640 1 -- 172.21.15.175:0/2675835492 wait complete. 2024-07-29T00:34:54.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:54 smithi204 ceph-mon[37368]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-07-29T00:34:54.651 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:54 smithi181 ceph-mon[36706]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-07-29T00:34:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 -- 172.21.15.175:0/1472824769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc102880 msgr2=0x7f72fc102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 --2- 172.21.15.175:0/1472824769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc102880 0x7f72fc102c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009900 tx=0x7f72f002a480 comp rx=0 tx=0).stop 2024-07-29T00:34:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 -- 172.21.15.175:0/1472824769 shutdown_connections 2024-07-29T00:34:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 --2- 172.21.15.175:0/1472824769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72fc10bcb0 0x7f72fc10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 --2- 172.21.15.175:0/1472824769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc1031a0 0x7f72fc10b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 --2- 172.21.15.175:0/1472824769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc102880 0x7f72fc102c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 -- 172.21.15.175:0/1472824769 >> 172.21.15.175:0/1472824769 conn(0x7f72fc0fc300 msgr2=0x7f72fc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 -- 172.21.15.175:0/1472824769 shutdown_connections 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.376+0000 7f7302400640 1 -- 172.21.15.175:0/1472824769 wait complete. 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.377+0000 7f7302400640 1 Processor -- start 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 -- start start 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc102880 0x7f72fc1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc1031a0 0x7f72fc100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72fc10bcb0 0x7f72fc100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc111b00 con 0x7f72fc1031a0 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc111980 con 0x7f72fc10bcb0 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7302400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc111c80 con 0x7f72fc102880 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7301000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc102880 0x7f72fc1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7301000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc102880 0x7f72fc1004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:43932/0 (socket says 172.21.15.175:43932) 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f7301000640 1 -- 172.21.15.175:0/1729608770 learned_addr learned my addr 172.21.15.175:0/1729608770 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.378+0000 7f72fbe00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc1031a0 0x7f72fc100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301a00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72fc10bcb0 0x7f72fc100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301000640 1 -- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72fc10bcb0 msgr2=0x7f72fc100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301000640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72fc10bcb0 0x7f72fc100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301000640 1 -- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc1031a0 msgr2=0x7f72fc100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301000640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc1031a0 0x7f72fc100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301000640 1 -- 172.21.15.175:0/1729608770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc1a6480 con 0x7f72fc102880 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301a00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72fc10bcb0 0x7f72fc100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f7301000640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc102880 0x7f72fc1004e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f72f0004770 tx=0x7f72f0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:55.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.379+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f003f070 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.380+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f0031ca0 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.380+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0035030 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.380+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc1a6710 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.380+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1a6c40 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.381+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72fc0625f0 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.383+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f72f0042070 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.385+0000 7f72f9600640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f72e40a2c90 0x7f72e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.385+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f72f00efbb0 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.385+0000 7f72fbe00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f72e40a2c90 0x7f72e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.385+0000 7f72f9600640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] conn(0x7f72e40ac940 0x7f72e40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.385+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 --> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72f0001d30 con 0x7f72e40ac940 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.385+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f72f00eff60 con 0x7f72fc102880 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.386+0000 7f72fbe00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f72e40a2c90 0x7f72e40a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72fc101f70 tx=0x7f72e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.386+0000 7f7301a00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] conn(0x7f72e40ac940 0x7f72e40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.386+0000 7f7301a00640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] conn(0x7f72e40ac940 0x7f72e40aede0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.387+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== osd.10 v2:172.21.15.181:6824/2035344536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72f0001d30 con 0x7f72e40ac940 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.401+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 --> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f72fc0629f0 con 0x7f72e40ac940 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.402+0000 7f72f9600640 1 -- 172.21.15.175:0/1729608770 <== osd.10 v2:172.21.15.181:6824/2035344536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f72fc0629f0 con 0x7f72e40ac940 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] conn(0x7f72e40ac940 msgr2=0x7f72e40aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] conn(0x7f72e40ac940 0x7f72e40aede0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f72e40a2c90 msgr2=0x7f72e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f72e40a2c90 0x7f72e40a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72fc101f70 tx=0x7f72e8005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc102880 msgr2=0x7f72fc1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 --2- 172.21.15.175:0/1729608770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72fc102880 0x7f72fc1004e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f72f0004770 tx=0x7f72f0003c10 comp rx=0 tx=0).stop 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7301a00640 1 -- 172.21.15.175:0/1729608770 reap_dead start 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 shutdown_connections 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 >> 172.21.15.175:0/1729608770 conn(0x7f72fc0fc300 msgr2=0x7f72fc0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 shutdown_connections 2024-07-29T00:34:55.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:53.403+0000 7f7302400640 1 -- 172.21.15.175:0/1729608770 wait complete. 2024-07-29T00:34:56.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 -- 172.21.15.175:0/3522562342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 msgr2=0x7f905c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:56.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 --2- 172.21.15.175:0/3522562342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c10e590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9054009be0 tx=0x7f905402a6c0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 -- 172.21.15.175:0/3522562342 shutdown_connections 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 --2- 172.21.15.175:0/3522562342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c10e590 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 --2- 172.21.15.175:0/3522562342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f905c1016c0 0x7f905c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 --2- 172.21.15.175:0/3522562342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905c1076c0 0x7f905c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 -- 172.21.15.175:0/3522562342 >> 172.21.15.175:0/3522562342 conn(0x7f905c0fd400 msgr2=0x7f905c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 -- 172.21.15.175:0/3522562342 shutdown_connections 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.773+0000 7f9061e00640 1 -- 172.21.15.175:0/3522562342 wait complete. 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.774+0000 7f9061e00640 1 Processor -- start 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 -- start start 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f905c1016c0 0x7f905c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905c1076c0 0x7f905c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:56.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f905c111ac0 con 0x7f905c1076c0 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f905c111940 con 0x7f905c102060 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.775+0000 7f9061e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f905c111c40 con 0x7f905c1016c0 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f9060a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f905c1016c0 0x7f905c0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c074720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54544/0 (socket says 172.21.15.175:54544) 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 -- 172.21.15.175:0/1815415877 learned_addr learned my addr 172.21.15.175:0/1815415877 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f9060a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f905c1016c0 0x7f905c0741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:43968/0 (socket says 172.21.15.175:43968) 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f9061400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905c1076c0 0x7f905c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 -- 172.21.15.175:0/1815415877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f905c1016c0 msgr2=0x7f905c0741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f905c1016c0 0x7f905c0741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 -- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905c1076c0 msgr2=0x7f905c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:56.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f905c1076c0 0x7f905c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 -- 172.21.15.175:0/1815415877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c1a63e0 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.776+0000 7f905be00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c074720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f904800e9f0 tx=0x7f904800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.777+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904800cd80 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.777+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048005500 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.777+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048010690 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.777+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c1a66d0 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.777+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1a6c10 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.778+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f905c0fe450 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.780+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f90480046c0 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.782+0000 7f9059600640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f90440a2c90 0x7f90440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.782+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f90480cfa90 con 0x7f905c102060 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.782+0000 7f9060a00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f90440a2c90 0x7f90440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:56.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.782+0000 7f9059600640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] conn(0x7f90440ac940 0x7f90440aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.782+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 --> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f904801aec0 con 0x7f90440ac940 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.782+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f90480cfe40 con 0x7f905c102060 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.783+0000 7f9061400640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] conn(0x7f90440ac940 0x7f90440aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.783+0000 7f9060a00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f90440a2c90 0x7f90440a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90500096c0 tx=0x7f9050009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.783+0000 7f9061400640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] conn(0x7f90440ac940 0x7f90440aede0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.784+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== osd.9 v2:172.21.15.175:6826/2466426097 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f904801aec0 con 0x7f90440ac940 2024-07-29T00:34:56.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.799+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 --> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f905c102e00 con 0x7f90440ac940 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9059600640 1 -- 172.21.15.175:0/1815415877 <== osd.9 v2:172.21.15.175:6826/2466426097 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f905c102e00 con 0x7f90440ac940 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] conn(0x7f90440ac940 msgr2=0x7f90440aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9061e00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] conn(0x7f90440ac940 0x7f90440aede0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f90440a2c90 msgr2=0x7f90440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9061e00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f90440a2c90 0x7f90440a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90500096c0 tx=0x7f9050009260 comp rx=0 tx=0).stop 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 msgr2=0x7f905c074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.800+0000 7f9061e00640 1 --2- 172.21.15.175:0/1815415877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905c102060 0x7f905c074720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f904800e9f0 tx=0x7f904800edf0 comp rx=0 tx=0).stop 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.801+0000 7f9061400640 1 -- 172.21.15.175:0/1815415877 reap_dead start 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.801+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 shutdown_connections 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.801+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 >> 172.21.15.175:0/1815415877 conn(0x7f905c0fd400 msgr2=0x7f905c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.801+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 shutdown_connections 2024-07-29T00:34:56.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:55.801+0000 7f9061e00640 1 -- 172.21.15.175:0/1815415877 wait complete. 2024-07-29T00:34:56.215 INFO:teuthology.orchestra.run.smithi175.stdout:197568495665 2024-07-29T00:34:56.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-07-29T00:34:56.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:56 smithi175 ceph-mon[32138]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s; 7.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:34:56.968 INFO:teuthology.orchestra.run.smithi175.stdout:283467841572 2024-07-29T00:34:56.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-07-29T00:34:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:56 smithi181 ceph-mon[36706]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s; 7.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:34:57.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:56 smithi204 ceph-mon[37368]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s; 7.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:34:57.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 -- 172.21.15.175:0/678218300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 msgr2=0x7fd78c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 --2- 172.21.15.175:0/678218300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 0x7fd78c10e590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd784009930 tx=0x7fd78402a420 comp rx=0 tx=0).stop 2024-07-29T00:34:57.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 -- 172.21.15.175:0/678218300 shutdown_connections 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 --2- 172.21.15.175:0/678218300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 0x7fd78c10e590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 --2- 172.21.15.175:0/678218300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 0x7fd78c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 --2- 172.21.15.175:0/678218300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd78c1076c0 0x7fd78c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 -- 172.21.15.175:0/678218300 >> 172.21.15.175:0/678218300 conn(0x7fd78c0fd400 msgr2=0x7fd78c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 -- 172.21.15.175:0/678218300 shutdown_connections 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.471+0000 7fd790a00640 1 -- 172.21.15.175:0/678218300 wait complete. 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 Processor -- start 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 -- start start 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 0x7fd78c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 0x7fd78c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd78c1076c0 0x7fd78c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c111b10 con 0x7fd78c102060 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c111990 con 0x7fd78c1076c0 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd790a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c111c90 con 0x7fd78c1016c0 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd78a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 0x7fd78c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.473+0000 7fd78a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 0x7fd78c19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:43994/0 (socket says 172.21.15.175:43994) 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 -- 172.21.15.175:0/3390569730 learned_addr learned my addr 172.21.15.175:0/3390569730 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78b200640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd78c1076c0 0x7fd78c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd781e00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 0x7fd78c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 -- 172.21.15.175:0/3390569730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd78c1076c0 msgr2=0x7fd78c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd78c1076c0 0x7fd78c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 -- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 msgr2=0x7fd78c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 0x7fd78c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 -- 172.21.15.175:0/3390569730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c19f0a0 con 0x7fd78c1016c0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd781e00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd78c102060 0x7fd78c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.474+0000 7fd78a800640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 0x7fd78c19a0f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00eab0 tx=0x7fd77c00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.475+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00ce40 con 0x7fd78c1016c0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.475+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c011950 con 0x7fd78c1016c0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.475+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c19f390 con 0x7fd78c1016c0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.475+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c010ae0 con 0x7fd78c1016c0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.475+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c1a6c70 con 0x7fd78c1016c0 2024-07-29T00:34:57.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.476+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd78c0ff340 con 0x7fd78c1016c0 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.478+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd77c011b00 con 0x7fd78c1016c0 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.479+0000 7fd783e00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd7700a2c10 0x7fd7700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.480+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fd77c0cf8b0 con 0x7fd78c1016c0 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.480+0000 7fd781e00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd7700a2c10 0x7fd7700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.480+0000 7fd783e00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] conn(0x7fd7700ac8c0 0x7fd7700aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.480+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 --> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7700af430 con 0x7fd7700ac8c0 2024-07-29T00:34:57.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.480+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd77c0d42a0 con 0x7fd78c1016c0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.480+0000 7fd78b200640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] conn(0x7fd7700ac8c0 0x7fd7700aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.481+0000 7fd781e00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd7700a2c10 0x7fd7700a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0ffca0 tx=0x7fd774005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.481+0000 7fd78b200640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] conn(0x7fd7700ac8c0 0x7fd7700aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.482+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== osd.3 v2:172.21.15.175:6810/2700469757 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7700af430 con 0x7fd7700ac8c0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.500+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 --> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd78c102e00 con 0x7fd7700ac8c0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.500+0000 7fd783e00640 1 -- 172.21.15.175:0/3390569730 <== osd.3 v2:172.21.15.175:6810/2700469757 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd78c102e00 con 0x7fd7700ac8c0 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] conn(0x7fd7700ac8c0 msgr2=0x7fd7700aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] conn(0x7fd7700ac8c0 0x7fd7700aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd7700a2c10 msgr2=0x7fd7700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd7700a2c10 0x7fd7700a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0ffca0 tx=0x7fd774005f90 comp rx=0 tx=0).stop 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 msgr2=0x7fd78c19a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 --2- 172.21.15.175:0/3390569730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd78c1016c0 0x7fd78c19a0f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00eab0 tx=0x7fd77c00ef70 comp rx=0 tx=0).stop 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd78b200640 1 -- 172.21.15.175:0/3390569730 reap_dead start 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 shutdown_connections 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 >> 172.21.15.175:0/3390569730 conn(0x7fd78c0fd400 msgr2=0x7fd78c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 shutdown_connections 2024-07-29T00:34:57.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:57.501+0000 7fd790a00640 1 -- 172.21.15.175:0/3390569730 wait complete. 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 -- 172.21.15.175:0/187782911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 msgr2=0x7fd954107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 --2- 172.21.15.175:0/187782911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 0x7fd954107b20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd944009930 tx=0x7fd94402a420 comp rx=0 tx=0).stop 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 -- 172.21.15.175:0/187782911 shutdown_connections 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 --2- 172.21.15.175:0/187782911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 0x7fd95410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 --2- 172.21.15.175:0/187782911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9540ffc30 0x7fd954100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 --2- 172.21.15.175:0/187782911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 0x7fd954107b20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 -- 172.21.15.175:0/187782911 >> 172.21.15.175:0/187782911 conn(0x7fd954076d60 msgr2=0x7fd9540fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:57.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 -- 172.21.15.175:0/187782911 shutdown_connections 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.199+0000 7fd959200640 1 -- 172.21.15.175:0/187782911 wait complete. 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 Processor -- start 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 -- start start 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9540ffc30 0x7fd954100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 0x7fd954101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 0x7fd954101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd954111b90 con 0x7fd954107740 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd954111a10 con 0x7fd9541005d0 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.200+0000 7fd959200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd954111d10 con 0x7fd9540ffc30 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 0x7fd954101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd953400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9540ffc30 0x7fd954100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 0x7fd954101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54508/0 (socket says 172.21.15.175:54508) 2024-07-29T00:34:57.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd953e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 0x7fd954101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 -- 172.21.15.175:0/2985864998 learned_addr learned my addr 172.21.15.175:0/2985864998 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 -- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9540ffc30 msgr2=0x7fd954100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9540ffc30 0x7fd954100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 -- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 msgr2=0x7fd954101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd954107740 0x7fd954101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 -- 172.21.15.175:0/2985864998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9541a64f0 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.201+0000 7fd94aa00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 0x7fd954101350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9f0 tx=0x7fd93c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.202+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00cd80 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.202+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9541a67e0 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.202+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c005500 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.202+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c010690 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.202+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541a6cd0 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.203+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9540625f0 con 0x7fd9541005d0 2024-07-29T00:34:57.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.205+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd93c0046c0 con 0x7fd9541005d0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.206+0000 7fd950c00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9340a2c10 0x7fd9340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.206+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fd93c0cfa90 con 0x7fd9541005d0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.206+0000 7fd953400640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9340a2c10 0x7fd9340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.207+0000 7fd950c00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] conn(0x7fd9340ac8c0 0x7fd9340aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.207+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 --> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd93c005350 con 0x7fd9340ac8c0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.207+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd93c0cfe40 con 0x7fd9541005d0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.207+0000 7fd953e00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] conn(0x7fd9340ac8c0 0x7fd9340aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.207+0000 7fd953400640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9340a2c10 0x7fd9340a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd944006010 tx=0x7fd944000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.207+0000 7fd953e00640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] conn(0x7fd9340ac8c0 0x7fd9340aed60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.209+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== osd.8 v2:172.21.15.204:6816/1532855581 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd93c005350 con 0x7fd9340ac8c0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.223+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 --> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd9540629f0 con 0x7fd9340ac8c0 2024-07-29T00:34:57.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd950c00640 1 -- 172.21.15.175:0/2985864998 <== osd.8 v2:172.21.15.204:6816/1532855581 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd9540629f0 con 0x7fd9340ac8c0 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] conn(0x7fd9340ac8c0 msgr2=0x7fd9340aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd959200640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] conn(0x7fd9340ac8c0 0x7fd9340aed60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9340a2c10 msgr2=0x7fd9340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd959200640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fd9340a2c10 0x7fd9340a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd944006010 tx=0x7fd944000bc0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 msgr2=0x7fd954101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd959200640 1 --2- 172.21.15.175:0/2985864998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9541005d0 0x7fd954101350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9f0 tx=0x7fd93c00edf0 comp rx=0 tx=0).stop 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.224+0000 7fd953e00640 1 -- 172.21.15.175:0/2985864998 reap_dead start 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.225+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 shutdown_connections 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.225+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 >> 172.21.15.175:0/2985864998 conn(0x7fd954076d60 msgr2=0x7fd954104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:57.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.225+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 shutdown_connections 2024-07-29T00:34:57.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:54.225+0000 7fd959200640 1 -- 172.21.15.175:0/2985864998 wait complete. 2024-07-29T00:34:58.452 INFO:teuthology.orchestra.run.smithi175.stdout:304942678046 2024-07-29T00:34:58.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-07-29T00:34:58.644 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:34:58 smithi175 ceph-mon[32138]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5 op/s; 7.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:34:58.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 -- 172.21.15.175:0/114727990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac1126d0 msgr2=0x7ff2ac114ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/114727990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac1126d0 0x7ff2ac114ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009be0 tx=0x7ff2a402a750 comp rx=0 tx=0).stop 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 -- 172.21.15.175:0/114727990 shutdown_connections 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/114727990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac1126d0 0x7ff2ac114ac0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/114727990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 0x7ff2ac112190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/114727990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2ac074280 0x7ff2ac074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 -- 172.21.15.175:0/114727990 >> 172.21.15.175:0/114727990 conn(0x7ff2ac0fd400 msgr2=0x7ff2ac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 -- 172.21.15.175:0/114727990 shutdown_connections 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.636+0000 7ff2b0c00640 1 -- 172.21.15.175:0/114727990 wait complete. 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.637+0000 7ff2b0c00640 1 Processor -- start 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.637+0000 7ff2b0c00640 1 -- start start 2024-07-29T00:34:58.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.637+0000 7ff2b0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac074280 0x7ff2ac19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.637+0000 7ff2b0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 0x7ff2ac19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.637+0000 7ff2b0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2ac1126d0 0x7ff2ac1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2b0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac118090 con 0x7ff2ac1126d0 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2b0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac117f10 con 0x7ff2ac074ba0 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2b0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac118210 con 0x7ff2ac074280 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 0x7ff2ac19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 0x7ff2ac19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54564/0 (socket says 172.21.15.175:54564) 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 -- 172.21.15.175:0/676686321 learned_addr learned my addr 172.21.15.175:0/676686321 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2abe00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2ac1126d0 0x7ff2ac1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2ab400640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac074280 0x7ff2ac19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 -- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac074280 msgr2=0x7ff2ac19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:58.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ac074280 0x7ff2ac19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 -- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2ac1126d0 msgr2=0x7ff2ac1a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2ac1126d0 0x7ff2ac1a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.638+0000 7ff2aaa00640 1 -- 172.21.15.175:0/676686321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac1a3570 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.639+0000 7ff2aaa00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 0x7ff2ac19eb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff29800e9f0 tx=0x7ff29800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.639+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29800cd80 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.639+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff298005500 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.639+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298010690 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.639+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac1a3860 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.639+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac1ab140 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.640+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2ac062670 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.642+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff2980046c0 con 0x7ff2ac074ba0 2024-07-29T00:34:58.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.643+0000 7ff2a3e00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2900a2c10 0x7ff2900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.643+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff2980cf590 con 0x7ff2ac074ba0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.643+0000 7ff2ab400640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2900a2c10 0x7ff2900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.644+0000 7ff2a3e00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] conn(0x7ff2900ac8c0 0x7ff2900aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.644+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 --> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2900af430 con 0x7ff2900ac8c0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.644+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff2980d6050 con 0x7ff2ac074ba0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.644+0000 7ff2abe00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] conn(0x7ff2900ac8c0 0x7ff2900aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.645+0000 7ff2abe00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] conn(0x7ff2900ac8c0 0x7ff2900aed60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.645+0000 7ff2ab400640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2900a2c10 0x7ff2900a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009530 tx=0x7ff29c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.646+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== osd.5 v2:172.21.15.204:6808/377725660 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2900af430 con 0x7ff2900ac8c0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.662+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 --> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2ac062af0 con 0x7ff2900ac8c0 2024-07-29T00:34:58.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.663+0000 7ff2a3e00640 1 -- 172.21.15.175:0/676686321 <== osd.5 v2:172.21.15.204:6808/377725660 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff2ac062af0 con 0x7ff2900ac8c0 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] conn(0x7ff2900ac8c0 msgr2=0x7ff2900aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] conn(0x7ff2900ac8c0 0x7ff2900aed60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2900a2c10 msgr2=0x7ff2900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2900a2c10 0x7ff2900a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009530 tx=0x7ff29c009260 comp rx=0 tx=0).stop 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 msgr2=0x7ff2ac19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 --2- 172.21.15.175:0/676686321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ac074ba0 0x7ff2ac19eb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff29800e9f0 tx=0x7ff29800edf0 comp rx=0 tx=0).stop 2024-07-29T00:34:58.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2abe00640 1 -- 172.21.15.175:0/676686321 reap_dead start 2024-07-29T00:34:58.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 shutdown_connections 2024-07-29T00:34:58.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 >> 172.21.15.175:0/676686321 conn(0x7ff2ac0fd400 msgr2=0x7ff2ac0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:58.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 shutdown_connections 2024-07-29T00:34:58.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:58.664+0000 7ff2b0c00640 1 -- 172.21.15.175:0/676686321 wait complete. 2024-07-29T00:34:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:34:58 smithi181 ceph-mon[36706]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5 op/s; 7.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:34:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:34:58 smithi204 ceph-mon[37368]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5 op/s; 7.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:34:59.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 -- 172.21.15.175:0/3441304996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10101880 msgr2=0x7f9d10101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:59.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 --2- 172.21.15.175:0/3441304996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10101880 0x7f9d10101ce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009cd0 tx=0x7f9cf802a8e0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 -- 172.21.15.175:0/3441304996 shutdown_connections 2024-07-29T00:34:59.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 --2- 172.21.15.175:0/3441304996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10102220 0x7f9d1010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 --2- 172.21.15.175:0/3441304996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10101880 0x7f9d10101ce0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 --2- 172.21.15.175:0/3441304996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d10107880 0x7f9d10107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 -- 172.21.15.175:0/3441304996 >> 172.21.15.175:0/3441304996 conn(0x7f9d100fd5a0 msgr2=0x7f9d100ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 -- 172.21.15.175:0/3441304996 shutdown_connections 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.515+0000 7f9d16400640 1 -- 172.21.15.175:0/3441304996 wait complete. 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.516+0000 7f9d16400640 1 Processor -- start 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.516+0000 7f9d16400640 1 -- start start 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d16400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10101880 0x7f9d1019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d16400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d10102220 0x7f9d1019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d16400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10107880 0x7f9d1019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d16400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10111c90 con 0x7f9d10107880 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d16400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10111b10 con 0x7f9d10101880 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d16400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10111e10 con 0x7f9d10102220 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10107880 0x7f9d1019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10101880 0x7f9d1019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:59.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10101880 0x7f9d1019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58348/0 (socket says 172.21.15.175:58348) 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0f400640 1 -- 172.21.15.175:0/1541687773 learned_addr learned my addr 172.21.15.175:0/1541687773 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0fe00640 1 -- 172.21.15.175:0/1541687773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d10102220 msgr2=0x7f9d1019a880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0fe00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d10102220 0x7f9d1019a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0fe00640 1 -- 172.21.15.175:0/1541687773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10101880 msgr2=0x7f9d1019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0fe00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10101880 0x7f9d1019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0fe00640 1 -- 172.21.15.175:0/1541687773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1019f2f0 con 0x7f9d10107880 2024-07-29T00:34:59.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.517+0000 7f9d0f400640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d10101880 0x7f9d1019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.518+0000 7f9d0fe00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10107880 0x7f9d1019ec10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800e970 tx=0x7f9d0800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.518+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800ccc0 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.518+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d08005020 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.518+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08004a00 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.518+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1019f5e0 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.518+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a6ec0 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.519+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d1010d600 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.522+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9d080051d0 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.523+0000 7f9d0cc00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9cf40a2c90 0x7f9cf40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.523+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f9d080cf4d0 con 0x7f9d10107880 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.523+0000 7f9d0f400640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9cf40a2c90 0x7f9cf40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.524+0000 7f9d0cc00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] conn(0x7f9cf40ac940 0x7f9cf40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.524+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 --> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cf40af4b0 con 0x7f9cf40ac940 2024-07-29T00:34:59.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.524+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9d080cf880 con 0x7f9d10107880 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.524+0000 7f9d06a00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] conn(0x7f9cf40ac940 0x7f9cf40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.524+0000 7f9d0f400640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9cf40a2c90 0x7f9cf40a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d00005fd0 tx=0x7f9d0000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.524+0000 7f9d06a00640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] conn(0x7f9cf40ac940 0x7f9cf40aede0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.525+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== osd.0 v2:172.21.15.175:6802/311033878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9cf40af4b0 con 0x7f9cf40ac940 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.540+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 --> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d10102fc0 con 0x7f9cf40ac940 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.541+0000 7f9d0cc00640 1 -- 172.21.15.175:0/1541687773 <== osd.0 v2:172.21.15.175:6802/311033878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9d10102fc0 con 0x7f9cf40ac940 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.541+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] conn(0x7f9cf40ac940 msgr2=0x7f9cf40aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.541+0000 7f9d16400640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] conn(0x7f9cf40ac940 0x7f9cf40aede0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9cf40a2c90 msgr2=0x7f9cf40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9cf40a2c90 0x7f9cf40a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d00005fd0 tx=0x7f9d0000b4a0 comp rx=0 tx=0).stop 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10107880 msgr2=0x7f9d1019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 --2- 172.21.15.175:0/1541687773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d10107880 0x7f9d1019ec10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800e970 tx=0x7f9d0800ee30 comp rx=0 tx=0).stop 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d0fe00640 1 -- 172.21.15.175:0/1541687773 reap_dead start 2024-07-29T00:34:59.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 shutdown_connections 2024-07-29T00:34:59.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 >> 172.21.15.175:0/1541687773 conn(0x7f9d100fd5a0 msgr2=0x7f9d1010cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:34:59.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 shutdown_connections 2024-07-29T00:34:59.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:34:59.542+0000 7f9d16400640 1 -- 172.21.15.175:0/1541687773 wait complete. 2024-07-29T00:35:00.462 INFO:teuthology.orchestra.run.smithi175.stdout:219043332142 2024-07-29T00:35:00.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-07-29T00:35:00.645 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:00 smithi175 ceph-mon[32138]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 6.0 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:35:00.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 -- 172.21.15.175:0/2949168451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c0102880 msgr2=0x7f32c0102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:00.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 --2- 172.21.15.175:0/2949168451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c0102880 0x7f32c0102c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009900 tx=0x7f32b002a480 comp rx=0 tx=0).stop 2024-07-29T00:35:00.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 -- 172.21.15.175:0/2949168451 shutdown_connections 2024-07-29T00:35:00.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 --2- 172.21.15.175:0/2949168451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c010bcb0 0x7f32c010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:00.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 --2- 172.21.15.175:0/2949168451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c01031a0 0x7f32c010b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:00.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 --2- 172.21.15.175:0/2949168451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c0102880 0x7f32c0102c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:00.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 -- 172.21.15.175:0/2949168451 >> 172.21.15.175:0/2949168451 conn(0x7f32c00fc300 msgr2=0x7f32c00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:00.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 -- 172.21.15.175:0/2949168451 shutdown_connections 2024-07-29T00:35:00.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.667+0000 7f32bf200640 1 -- 172.21.15.175:0/2949168451 wait complete. 2024-07-29T00:35:00.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.668+0000 7f32bf200640 1 Processor -- start 2024-07-29T00:35:00.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.668+0000 7f32bf200640 1 -- start start 2024-07-29T00:35:00.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c0102880 0x7f32c01004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c01031a0 0x7f32c0100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c010bcb0 0x7f32c0100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111ab0 con 0x7f32c01031a0 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111930 con 0x7f32c010bcb0 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111c30 con 0x7f32c0102880 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c01031a0 0x7f32c0100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c0102880 0x7f32c01004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c0102880 0x7f32c01004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52512/0 (socket says 172.21.15.175:52512) 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32bde00640 1 -- 172.21.15.175:0/86454372 learned_addr learned my addr 172.21.15.175:0/86454372 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:00.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.669+0000 7f32be800640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c010bcb0 0x7f32c0100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:00.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bde00640 1 -- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c010bcb0 msgr2=0x7f32c0100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:00.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bde00640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c010bcb0 0x7f32c0100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:00.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bde00640 1 -- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c01031a0 msgr2=0x7f32c0100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:00.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bde00640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c01031a0 0x7f32c0100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bde00640 1 -- 172.21.15.175:0/86454372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01a64a0 con 0x7f32c0102880 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bde00640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c0102880 0x7f32c01004e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b003f030 con 0x7f32c0102880 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b0004440 con 0x7f32c0102880 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0035070 con 0x7f32c0102880 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.670+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01a6730 con 0x7f32c0102880 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.671+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a6c60 con 0x7f32c0102880 2024-07-29T00:35:00.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.672+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32c00625f0 con 0x7f32c0102880 2024-07-29T00:35:00.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.674+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f32b0042030 con 0x7f32c0102880 2024-07-29T00:35:00.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.675+0000 7f32b6a00640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32a40a2c90 0x7f32a40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:00.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.675+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f32b00eff80 con 0x7f32c0102880 2024-07-29T00:35:00.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.675+0000 7f32bd400640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32a40a2c90 0x7f32a40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:00.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.675+0000 7f32b6a00640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] conn(0x7f32a40ac940 0x7f32a40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:00.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.676+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 --> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32a40af4b0 con 0x7f32a40ac940 2024-07-29T00:35:00.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.676+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f32b00f0330 con 0x7f32c0102880 2024-07-29T00:35:00.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.676+0000 7f32be800640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] conn(0x7f32a40ac940 0x7f32a40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:00.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.676+0000 7f32bd400640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32a40a2c90 0x7f32a40a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f32c010d0e0 tx=0x7f32a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:00.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.676+0000 7f32be800640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] conn(0x7f32a40ac940 0x7f32a40aede0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:00.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.677+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== osd.4 v2:172.21.15.181:6808/1605621704 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32a40af4b0 con 0x7f32a40ac940 2024-07-29T00:35:00.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.692+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 --> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32c00629f0 con 0x7f32a40ac940 2024-07-29T00:35:00.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.693+0000 7f32b6a00640 1 -- 172.21.15.175:0/86454372 <== osd.4 v2:172.21.15.181:6808/1605621704 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32c00629f0 con 0x7f32a40ac940 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] conn(0x7f32a40ac940 msgr2=0x7f32a40aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] conn(0x7f32a40ac940 0x7f32a40aede0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32a40a2c90 msgr2=0x7f32a40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32a40a2c90 0x7f32a40a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f32c010d0e0 tx=0x7f32a8005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c0102880 msgr2=0x7f32c01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 --2- 172.21.15.175:0/86454372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32c0102880 0x7f32c01004e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b0004040 comp rx=0 tx=0).stop 2024-07-29T00:35:00.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32be800640 1 -- 172.21.15.175:0/86454372 reap_dead start 2024-07-29T00:35:00.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 shutdown_connections 2024-07-29T00:35:00.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 >> 172.21.15.175:0/86454372 conn(0x7f32c00fc300 msgr2=0x7f32c00fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:00.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 shutdown_connections 2024-07-29T00:35:00.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:00.694+0000 7f32bf200640 1 -- 172.21.15.175:0/86454372 wait complete. 2024-07-29T00:35:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:00 smithi181 ceph-mon[36706]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 6.0 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:35:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:00 smithi204 ceph-mon[37368]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 6.0 KiB/s, 1 keys/s, 1 objects/s recovering 2024-07-29T00:35:01.941 INFO:teuthology.orchestra.run.smithi175.stdout:249108103209 2024-07-29T00:35:01.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-07-29T00:35:02.413 INFO:teuthology.orchestra.run.smithi175.stdout:107374182466 2024-07-29T00:35:02.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-29T00:35:02.725 INFO:teuthology.orchestra.run.smithi175.stdout:163208757306 2024-07-29T00:35:02.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-29T00:35:02.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:02 smithi175 ceph-mon[32138]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 5 op/s; 5.5 KiB/s, 0 keys/s, 1 objects/s recovering 2024-07-29T00:35:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:02 smithi181 ceph-mon[36706]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 5 op/s; 5.5 KiB/s, 0 keys/s, 1 objects/s recovering 2024-07-29T00:35:03.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:02 smithi204 ceph-mon[37368]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 5 op/s; 5.5 KiB/s, 0 keys/s, 1 objects/s recovering 2024-07-29T00:35:03.623 INFO:teuthology.orchestra.run.smithi175.stdout:141733920830 2024-07-29T00:35:03.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-29T00:35:03.676 INFO:teuthology.orchestra.run.smithi175.stdout:34359738449 2024-07-29T00:35:03.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-29T00:35:03.737 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:03.953 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:03.964 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:03.966 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:04.225 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:04.225 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:04.238 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:04.248 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:04.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 -- 172.21.15.175:0/2446093290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c1005d0 msgr2=0x7efd3c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 --2- 172.21.15.175:0/2446093290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c1005d0 0x7efd3c10e610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efd34009cd0 tx=0x7efd3402a8e0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 -- 172.21.15.175:0/2446093290 shutdown_connections 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 --2- 172.21.15.175:0/2446093290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c1005d0 0x7efd3c10e610 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 --2- 172.21.15.175:0/2446093290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd3c0ffc30 0x7efd3c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 --2- 172.21.15.175:0/2446093290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 -- 172.21.15.175:0/2446093290 >> 172.21.15.175:0/2446093290 conn(0x7efd3c076d60 msgr2=0x7efd3c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 -- 172.21.15.175:0/2446093290 shutdown_connections 2024-07-29T00:35:04.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.776+0000 7efd41a00640 1 -- 172.21.15.175:0/2446093290 wait complete. 2024-07-29T00:35:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 Processor -- start 2024-07-29T00:35:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 -- start start 2024-07-29T00:35:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c0ffc30 0x7efd3c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd3c1005d0 0x7efd3c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c111b90 con 0x7efd3c0ffc30 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c111a10 con 0x7efd3c107740 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.777+0000 7efd41a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c111d10 con 0x7efd3c1005d0 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd3be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c0ffc30 0x7efd3c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58384/0 (socket says 172.21.15.175:58384) 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 -- 172.21.15.175:0/2143936453 learned_addr learned my addr 172.21.15.175:0/2143936453 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd3b400640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd3c1005d0 0x7efd3c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 -- 172.21.15.175:0/2143936453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd3c1005d0 msgr2=0x7efd3c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd3c1005d0 0x7efd3c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 -- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c0ffc30 msgr2=0x7efd3c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c0ffc30 0x7efd3c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 -- 172.21.15.175:0/2143936453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c19cfe0 con 0x7efd3c107740 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd3be00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c0ffc30 0x7efd3c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.778+0000 7efd41000640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c19c900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efd34033040 tx=0x7efd34031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.779+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34004040 con 0x7efd3c107740 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.779+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd340046c0 con 0x7efd3c107740 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.779+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34030ed0 con 0x7efd3c107740 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.779+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c19d270 con 0x7efd3c107740 2024-07-29T00:35:04.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.779+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c1a4bb0 con 0x7efd3c107740 2024-07-29T00:35:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.780+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c102f50 con 0x7efd3c107740 2024-07-29T00:35:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.782+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efd340041e0 con 0x7efd3c107740 2024-07-29T00:35:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.783+0000 7efd38c00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7efd200a2c10 0x7efd200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.783+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7efd3402e030 con 0x7efd3c107740 2024-07-29T00:35:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.783+0000 7efd3be00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7efd200a2c10 0x7efd200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.784+0000 7efd3be00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7efd200a2c10 0x7efd200a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009f40 tx=0x7efd2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:04.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.787+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efd3404b060 con 0x7efd3c107740 2024-07-29T00:35:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.967+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efd3c1a4e10 con 0x7efd3c107740 2024-07-29T00:35:04.968 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:04 smithi175 ceph-mon[32138]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6 op/s; 5.0 KiB/s, 0 keys/s, 1 objects/s recovering 2024-07-29T00:35:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.968+0000 7efd38c00640 1 -- 172.21.15.175:0/2143936453 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efd340bd610 con 0x7efd3c107740 2024-07-29T00:35:04.969 INFO:teuthology.orchestra.run.smithi175.stdout:55834574925 2024-07-29T00:35:04.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.971+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7efd200a2c10 msgr2=0x7efd200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:04.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.971+0000 7efd41a00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7efd200a2c10 0x7efd200a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009f40 tx=0x7efd2c005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:04.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 msgr2=0x7efd3c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:04.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c19c900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efd34033040 tx=0x7efd34031e60 comp rx=0 tx=0).stop 2024-07-29T00:35:04.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 shutdown_connections 2024-07-29T00:35:04.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd3c107740 0x7efd3c19c900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7efd200a2c10 0x7efd200a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd3c1005d0 0x7efd3c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 --2- 172.21.15.175:0/2143936453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd3c0ffc30 0x7efd3c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:04.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.972+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 >> 172.21.15.175:0/2143936453 conn(0x7efd3c076d60 msgr2=0x7efd3c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:04.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.973+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 shutdown_connections 2024-07-29T00:35:04.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:04.973+0000 7efd41a00640 1 -- 172.21.15.175:0/2143936453 wait complete. 2024-07-29T00:35:04.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:04 smithi181 ceph-mon[36706]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6 op/s; 5.0 KiB/s, 0 keys/s, 1 objects/s recovering 2024-07-29T00:35:05.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:04 smithi204 ceph-mon[37368]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6 op/s; 5.0 KiB/s, 0 keys/s, 1 objects/s recovering 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 -- 172.21.15.175:0/1751177176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1076b0 msgr2=0x7f59ec107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 --2- 172.21.15.175:0/1751177176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1076b0 0x7f59ec107a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009930 tx=0x7f59dc02a440 comp rx=0 tx=0).stop 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 -- 172.21.15.175:0/1751177176 shutdown_connections 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 --2- 172.21.15.175:0/1751177176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 --2- 172.21.15.175:0/1751177176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec1016b0 0x7f59ec101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 --2- 172.21.15.175:0/1751177176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1076b0 0x7f59ec107a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 -- 172.21.15.175:0/1751177176 >> 172.21.15.175:0/1751177176 conn(0x7f59ec0fd430 msgr2=0x7f59ec0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 -- 172.21.15.175:0/1751177176 shutdown_connections 2024-07-29T00:35:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.651+0000 7f59f0a00640 1 -- 172.21.15.175:0/1751177176 wait complete. 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 Processor -- start 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 -- start start 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1016b0 0x7f59ec1a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec1a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec1076b0 0x7f59ec10af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec075560 con 0x7f59ec102050 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.652+0000 7f59f0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec0753e0 con 0x7f59ec1076b0 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59f0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec0756e0 con 0x7f59ec1016b0 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec1a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec1a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43420/0 (socket says 172.21.15.175:43420) 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 -- 172.21.15.175:0/3824362936 learned_addr learned my addr 172.21.15.175:0/3824362936 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59ea800640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1016b0 0x7f59ec1a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59eb200640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec1076b0 0x7f59ec10af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 -- 172.21.15.175:0/3824362936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1016b0 msgr2=0x7f59ec1a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1016b0 0x7f59ec1a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 -- 172.21.15.175:0/3824362936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec1076b0 msgr2=0x7f59ec10af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec1076b0 0x7f59ec10af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 -- 172.21.15.175:0/3824362936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec10b7d0 con 0x7f59ec102050 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.653+0000 7f59e3e00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec1a38e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e9f0 tx=0x7f59d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.655+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d400cd80 con 0x7f59ec102050 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.655+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59d4005500 con 0x7f59ec102050 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.655+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec10bac0 con 0x7f59ec102050 2024-07-29T00:35:05.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.655+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4010690 con 0x7f59ec102050 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.655+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec1ab280 con 0x7f59ec102050 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.657+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ec102df0 con 0x7f59ec102050 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.658+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f59d40046c0 con 0x7f59ec102050 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.660+0000 7f59e3400640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f59d00a2c90 0x7f59d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.660+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f59d40cf5d0 con 0x7f59ec102050 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.660+0000 7f59ea800640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f59d00a2c90 0x7f59d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.661+0000 7f59ea800640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f59d00a2c90 0x7f59d00a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f59dc006010 tx=0x7f59dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.663+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f59d409c350 con 0x7f59ec102050 2024-07-29T00:35:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.851+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f59ec1ab450 con 0x7f59ec102050 2024-07-29T00:35:05.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.851+0000 7f59e3400640 1 -- 172.21.15.175:0/3824362936 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f59d409bc50 con 0x7f59ec102050 2024-07-29T00:35:05.852 INFO:teuthology.orchestra.run.smithi175.stdout:197568495669 2024-07-29T00:35:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f59d00a2c90 msgr2=0x7f59d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f59d00a2c90 0x7f59d00a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f59dc006010 tx=0x7f59dc000bc0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 msgr2=0x7f59ec1a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec1a38e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e9f0 tx=0x7f59d400edf0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 shutdown_connections 2024-07-29T00:35:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec1076b0 0x7f59ec10af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59ec102050 0x7f59ec1a38e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f59d00a2c90 0x7f59d00a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 --2- 172.21.15.175:0/3824362936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ec1016b0 0x7f59ec1a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.854+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 >> 172.21.15.175:0/3824362936 conn(0x7f59ec0fd430 msgr2=0x7f59ec0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:05.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.855+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 shutdown_connections 2024-07-29T00:35:05.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.855+0000 7f59f0a00640 1 -- 172.21.15.175:0/3824362936 wait complete. 2024-07-29T00:35:05.923 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:05 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2143936453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-29T00:35:05.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 -- 172.21.15.175:0/1753830032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b410df50 msgr2=0x7f75b4110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 --2- 172.21.15.175:0/1753830032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b410df50 0x7f75b4110340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009ca0 tx=0x7f75ac02a950 comp rx=0 tx=0).stop 2024-07-29T00:35:05.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 -- 172.21.15.175:0/1753830032 shutdown_connections 2024-07-29T00:35:05.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 --2- 172.21.15.175:0/1753830032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b410df50 0x7f75b4110340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 --2- 172.21.15.175:0/1753830032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b4074bd0 0x7f75b410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 --2- 172.21.15.175:0/1753830032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 0x7f75b4074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 -- 172.21.15.175:0/1753830032 >> 172.21.15.175:0/1753830032 conn(0x7f75b40fd430 msgr2=0x7f75b40ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:05.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 -- 172.21.15.175:0/1753830032 shutdown_connections 2024-07-29T00:35:05.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.942+0000 7f75ba000640 1 -- 172.21.15.175:0/1753830032 wait complete. 2024-07-29T00:35:05.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.943+0000 7f75ba000640 1 Processor -- start 2024-07-29T00:35:05.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.943+0000 7f75ba000640 1 -- start start 2024-07-29T00:35:05.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75ba000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 0x7f75b419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75ba000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 0x7f75b419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75ba000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b410df50 0x7f75b419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75ba000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75b4113cf0 con 0x7f75b4074bd0 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75ba000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f75b4113b70 con 0x7f75b40742b0 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75ba000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75b4113e70 con 0x7f75b410df50 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 0x7f75b419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 0x7f75b419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43436/0 (socket says 172.21.15.175:43436) 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75b3e00640 1 -- 172.21.15.175:0/199773729 learned_addr learned my addr 172.21.15.175:0/199773729 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75b8c00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 0x7f75b419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.944+0000 7f75b9600640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b410df50 0x7f75b419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b3e00640 1 -- 172.21.15.175:0/199773729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b410df50 msgr2=0x7f75b419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b3e00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b410df50 0x7f75b419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b3e00640 1 -- 172.21.15.175:0/199773729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 msgr2=0x7f75b419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b3e00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 0x7f75b419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b3e00640 1 -- 172.21.15.175:0/199773729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b419f160 con 0x7f75b4074bd0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b8c00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 0x7f75b419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b3e00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 0x7f75b419a6f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75a000e9f0 tx=0x7f75a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a000cd80 con 0x7f75b4074bd0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a0010910 con 0x7f75b4074bd0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0018990 con 0x7f75b4074bd0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b419f450 con 0x7f75b4074bd0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.945+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41a6d30 con 0x7f75b4074bd0 2024-07-29T00:35:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.946+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b4074690 con 0x7f75b4074bd0 2024-07-29T00:35:05.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.949+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f75a0010ac0 con 0x7f75b4074bd0 2024-07-29T00:35:05.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.950+0000 7f75b1600640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f759c0a2c10 0x7f759c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:05.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.951+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f75a00cf940 con 0x7f75b4074bd0 2024-07-29T00:35:05.953 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.951+0000 7f75b8c00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f759c0a2c10 0x7f759c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:05.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.951+0000 7f75b8c00640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f759c0a2c10 0x7f759c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fc0 tx=0x7f75a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:05.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:05.952+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f75a009c070 con 0x7f75b4074bd0 2024-07-29T00:35:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:05 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2143936453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-29T00:35:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:05 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2143936453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-29T00:35:06.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.142+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f75b4109100 con 0x7f75b4074bd0 2024-07-29T00:35:06.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.143+0000 7f75b1600640 1 -- 172.21.15.175:0/199773729 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f75a00a0370 con 0x7f75b4074bd0 2024-07-29T00:35:06.144 INFO:teuthology.orchestra.run.smithi175.stdout:249108103212 2024-07-29T00:35:06.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f759c0a2c10 msgr2=0x7f759c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:06.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f759c0a2c10 0x7f759c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fc0 tx=0x7f75a8005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 msgr2=0x7f75b419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 0x7f75b419a6f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75a000e9f0 tx=0x7f75a000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 shutdown_connections 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f759c0a2c10 0x7f759c0a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b410df50 0x7f75b419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b4074bd0 0x7f75b419a6f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 --2- 172.21.15.175:0/199773729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b40742b0 0x7f75b419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 >> 172.21.15.175:0/199773729 conn(0x7f75b40fd430 msgr2=0x7f75b40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 shutdown_connections 2024-07-29T00:35:06.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:06.146+0000 7f75ba000640 1 -- 172.21.15.175:0/199773729 wait complete. 2024-07-29T00:35:06.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:06 smithi175 ceph-mon[32138]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-07-29T00:35:06.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:06 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3824362936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-29T00:35:06.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:06 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/199773729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-29T00:35:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:06 smithi181 ceph-mon[36706]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-07-29T00:35:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:06 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3824362936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-29T00:35:07.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:06 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/199773729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-29T00:35:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:06 smithi204 ceph-mon[37368]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-07-29T00:35:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:06 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3824362936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-29T00:35:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:06 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/199773729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-29T00:35:07.761 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574925 for osd.1 2024-07-29T00:35:07.761 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:07.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 -- 172.21.15.175:0/3995881350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 msgr2=0x7f5a30100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:07.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 --2- 172.21.15.175:0/3995881350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a30100870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009900 tx=0x7f5a2402a410 comp rx=0 tx=0).stop 2024-07-29T00:35:07.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 -- 172.21.15.175:0/3995881350 shutdown_connections 2024-07-29T00:35:07.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 --2- 172.21.15.175:0/3995881350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010e040 0x7f5a30110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:07.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 --2- 172.21.15.175:0/3995881350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 0x7f5a3010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:07.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 --2- 172.21.15.175:0/3995881350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a30100870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:07.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 -- 172.21.15.175:0/3995881350 >> 172.21.15.175:0/3995881350 conn(0x7f5a300fc300 msgr2=0x7f5a300fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:07.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 -- 172.21.15.175:0/3995881350 shutdown_connections 2024-07-29T00:35:07.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.937+0000 7f5a36c00640 1 -- 172.21.15.175:0/3995881350 wait complete. 2024-07-29T00:35:07.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.938+0000 7f5a36c00640 1 Processor -- start 2024-07-29T00:35:07.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.938+0000 7f5a36c00640 1 -- start start 2024-07-29T00:35:07.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.938+0000 7f5a36c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a3019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:07.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.938+0000 7f5a36c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 0x7f5a3019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:07.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a36c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010e040 0x7f5a3019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a36c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30113cc0 con 0x7f5a30100490 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a36c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30113b40 con 0x7f5a3010e040 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a36c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30113e40 con 0x7f5a30100e40 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a35800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a3019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a34e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 0x7f5a3019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a36200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010e040 0x7f5a3019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a35800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a3019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43466/0 (socket says 172.21.15.175:43466) 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a35800640 1 -- 172.21.15.175:0/1402432181 learned_addr learned my addr 172.21.15.175:0/1402432181 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a34e00640 1 -- 172.21.15.175:0/1402432181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010e040 msgr2=0x7f5a3019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:07.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a34e00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010e040 0x7f5a3019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:07.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a34e00640 1 -- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 msgr2=0x7f5a3019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:07.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a34e00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a3019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:07.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a34e00640 1 -- 172.21.15.175:0/1402432181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3019f180 con 0x7f5a30100e40 2024-07-29T00:35:07.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.939+0000 7f5a35800640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a3019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:07.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.940+0000 7f5a34e00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 0x7f5a3019a710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00e9f0 tx=0x7f5a1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:07.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.940+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00ccc0 con 0x7f5a30100e40 2024-07-29T00:35:07.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.940+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3019f470 con 0x7f5a30100e40 2024-07-29T00:35:07.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.940+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301a6d50 con 0x7f5a30100e40 2024-07-29T00:35:07.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.940+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1c010910 con 0x7f5a30100e40 2024-07-29T00:35:07.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.940+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c018880 con 0x7f5a30100e40 2024-07-29T00:35:07.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.942+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a10005290 con 0x7f5a30100e40 2024-07-29T00:35:07.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.944+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5a1c010430 con 0x7f5a30100e40 2024-07-29T00:35:07.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.946+0000 7f5a2e000640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5a180a2c90 0x7f5a180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:07.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.946+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f5a1c0cf410 con 0x7f5a30100e40 2024-07-29T00:35:07.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.946+0000 7f5a35800640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5a180a2c90 0x7f5a180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:07.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.947+0000 7f5a35800640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5a180a2c90 0x7f5a180a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a240096c0 tx=0x7f5a24005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:07.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:07.949+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5a1c09bb40 con 0x7f5a30100e40 2024-07-29T00:35:08.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.151+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5a10005850 con 0x7f5a30100e40 2024-07-29T00:35:08.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.152+0000 7f5a2e000640 1 -- 172.21.15.175:0/1402432181 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5a1c09fe40 con 0x7f5a30100e40 2024-07-29T00:35:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:339302416413 2024-07-29T00:35:08.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5a180a2c90 msgr2=0x7f5a180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5a180a2c90 0x7f5a180a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a240096c0 tx=0x7f5a24005a20 comp rx=0 tx=0).stop 2024-07-29T00:35:08.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 msgr2=0x7f5a3019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 0x7f5a3019a710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00e9f0 tx=0x7f5a1c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 shutdown_connections 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010e040 0x7f5a3019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5a180a2c90 0x7f5a180a5150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100e40 0x7f5a3019a710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 --2- 172.21.15.175:0/1402432181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30100490 0x7f5a3019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.155+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 >> 172.21.15.175:0/1402432181 conn(0x7f5a300fc300 msgr2=0x7f5a300fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.156+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 shutdown_connections 2024-07-29T00:35:08.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.156+0000 7f5a36c00640 1 -- 172.21.15.175:0/1402432181 wait complete. 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 -- 172.21.15.175:0/771013599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 msgr2=0x7f5e90103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 --2- 172.21.15.175:0/771013599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e90103ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009930 tx=0x7f5e7c02a420 comp rx=0 tx=0).stop 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 -- 172.21.15.175:0/771013599 shutdown_connections 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 --2- 172.21.15.175:0/771013599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90104420 0x7f5e90110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 --2- 172.21.15.175:0/771013599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e90103ee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 --2- 172.21.15.175:0/771013599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90109a80 0x7f5e90109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 -- 172.21.15.175:0/771013599 >> 172.21.15.175:0/771013599 conn(0x7f5e900ff760 msgr2=0x7f5e90101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 -- 172.21.15.175:0/771013599 shutdown_connections 2024-07-29T00:35:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.748+0000 7f5e97a00640 1 -- 172.21.15.175:0/771013599 wait complete. 2024-07-29T00:35:08.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:08 smithi175 ceph-mon[32138]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-07-29T00:35:08.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:08 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1402432181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.749+0000 7f5e97a00640 1 Processor -- start 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.749+0000 7f5e97a00640 1 -- start start 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e97a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e9019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e97a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90104420 0x7f5e9019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e97a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90109a80 0x7f5e9019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e97a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90113e80 con 0x7f5e90103a80 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e97a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90113d00 con 0x7f5e90104420 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e97a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90114000 con 0x7f5e90109a80 2024-07-29T00:35:08.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e95000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e9019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e95000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e9019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43494/0 (socket says 172.21.15.175:43494) 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e95000640 1 -- 172.21.15.175:0/2434262542 learned_addr learned my addr 172.21.15.175:0/2434262542 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e87e00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90104420 0x7f5e9019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.750+0000 7f5e95a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90109a80 0x7f5e9019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95000640 1 -- 172.21.15.175:0/2434262542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90109a80 msgr2=0x7f5e9019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95000640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90109a80 0x7f5e9019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95000640 1 -- 172.21.15.175:0/2434262542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90104420 msgr2=0x7f5e9019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95000640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90104420 0x7f5e9019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95000640 1 -- 172.21.15.175:0/2434262542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9019f310 con 0x7f5e90103a80 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90109a80 0x7f5e9019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:08.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e95000640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e9019a360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000ea60 tx=0x7f5e8000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:08.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.751+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000cde0 con 0x7f5e90103a80 2024-07-29T00:35:08.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.752+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e800035f0 con 0x7f5e90103a80 2024-07-29T00:35:08.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.752+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80010670 con 0x7f5e90103a80 2024-07-29T00:35:08.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.752+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9019f600 con 0x7f5e90103a80 2024-07-29T00:35:08.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.752+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e901a6ee0 con 0x7f5e90103a80 2024-07-29T00:35:08.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.753+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e901051c0 con 0x7f5e90103a80 2024-07-29T00:35:08.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.757+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5e800037a0 con 0x7f5e90103a80 2024-07-29T00:35:08.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.759+0000 7f5e8e000640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5e740a2c90 0x7f5e740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:08.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.759+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f5e800cf490 con 0x7f5e90103a80 2024-07-29T00:35:08.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.759+0000 7f5e87e00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5e740a2c90 0x7f5e740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:08.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.760+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5e8009c300 con 0x7f5e90103a80 2024-07-29T00:35:08.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.760+0000 7f5e87e00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5e740a2c90 0x7f5e740a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009900 tx=0x7f5e7c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:08.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.984+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5e90103ee0 con 0x7f5e90103a80 2024-07-29T00:35:08.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.984+0000 7f5e8e000640 1 -- 172.21.15.175:0/2434262542 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5e8009bc00 con 0x7f5e90103a80 2024-07-29T00:35:08.985 INFO:teuthology.orchestra.run.smithi175.stdout:304942678049 2024-07-29T00:35:08.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.987+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5e740a2c90 msgr2=0x7f5e740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5e740a2c90 0x7f5e740a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009900 tx=0x7f5e7c0058d0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 msgr2=0x7f5e9019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:08.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e9019a360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000ea60 tx=0x7f5e8000ef20 comp rx=0 tx=0).stop 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 shutdown_connections 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f5e740a2c90 0x7f5e740a5150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90109a80 0x7f5e9019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90104420 0x7f5e9019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 --2- 172.21.15.175:0/2434262542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90103a80 0x7f5e9019a360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 >> 172.21.15.175:0/2434262542 conn(0x7f5e900ff760 msgr2=0x7f5e900fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 shutdown_connections 2024-07-29T00:35:08.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:08.988+0000 7f5e97a00640 1 -- 172.21.15.175:0/2434262542 wait complete. 2024-07-29T00:35:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:08 smithi181 ceph-mon[36706]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-07-29T00:35:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:08 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1402432181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-29T00:35:09.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:08 smithi204 ceph-mon[37368]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-07-29T00:35:09.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:08 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1402432181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 -- 172.21.15.175:0/1671665400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28141031a0 msgr2=0x7f281410b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 --2- 172.21.15.175:0/1671665400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28141031a0 0x7f281410b690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2800009900 tx=0x7f280002a480 comp rx=0 tx=0).stop 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 -- 172.21.15.175:0/1671665400 shutdown_connections 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 --2- 172.21.15.175:0/1671665400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f281410bcb0 0x7f281410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 --2- 172.21.15.175:0/1671665400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28141031a0 0x7f281410b690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 --2- 172.21.15.175:0/1671665400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2814102880 0x7f2814102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 -- 172.21.15.175:0/1671665400 >> 172.21.15.175:0/1671665400 conn(0x7f28140fc300 msgr2=0x7f28140fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 -- 172.21.15.175:0/1671665400 shutdown_connections 2024-07-29T00:35:09.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.606+0000 7f2818e00640 1 -- 172.21.15.175:0/1671665400 wait complete. 2024-07-29T00:35:09.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.607+0000 7f2818e00640 1 Processor -- start 2024-07-29T00:35:09.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 -- start start 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2814102880 0x7f2814115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28141031a0 0x7f28141162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 0x7f281411bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28141020d0 con 0x7f28141031a0 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2814101f50 con 0x7f2814102880 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2818e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2814102250 con 0x7f281410bcb0 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.608+0000 7f2813e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 0x7f281411bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2812a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28141031a0 0x7f28141162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2814102880 0x7f2814115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 0x7f281411bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55980/0 (socket says 172.21.15.175:55980) 2024-07-29T00:35:09.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 -- 172.21.15.175:0/4085529953 learned_addr learned my addr 172.21.15.175:0/4085529953 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:09.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 -- 172.21.15.175:0/4085529953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2814102880 msgr2=0x7f2814115db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2814102880 0x7f2814115db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 -- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28141031a0 msgr2=0x7f28141162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28141031a0 0x7f28141162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 -- 172.21.15.175:0/4085529953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281411c390 con 0x7f281410bcb0 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.609+0000 7f2813e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 0x7f281411bad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f280c00efd0 tx=0x7f280c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.610+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c003b70 con 0x7f281410bcb0 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.610+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c003d10 con 0x7f281410bcb0 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.610+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0109f0 con 0x7f281410bcb0 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.610+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281411c680 con 0x7f281410bcb0 2024-07-29T00:35:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.610+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141110a0 con 0x7f281410bcb0 2024-07-29T00:35:09.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.612+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281406ad60 con 0x7f281410bcb0 2024-07-29T00:35:09.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.615+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f280c004020 con 0x7f281410bcb0 2024-07-29T00:35:09.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.616+0000 7f280be00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f27f80a2ce0 0x7f27f80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.617+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f280c0cf530 con 0x7f281410bcb0 2024-07-29T00:35:09.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.617+0000 7f2813400640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f27f80a2ce0 0x7f27f80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.618+0000 7f2813400640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f27f80a2ce0 0x7f27f80a51a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2804009bd0 tx=0x7f2804005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:09.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.620+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f280c09c310 con 0x7f281410bcb0 2024-07-29T00:35:09.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 -- 172.21.15.175:0/3353405683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101038e0 msgr2=0x7fb410103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 --2- 172.21.15.175:0/3353405683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101038e0 0x7fb410103d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009930 tx=0x7fb3fc02a440 comp rx=0 tx=0).stop 2024-07-29T00:35:09.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 -- 172.21.15.175:0/3353405683 shutdown_connections 2024-07-29T00:35:09.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 --2- 172.21.15.175:0/3353405683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb410104280 0x7fb4101108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 --2- 172.21.15.175:0/3353405683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101038e0 0x7fb410103d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 --2- 172.21.15.175:0/3353405683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4101099f0 0x7fb410109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 -- 172.21.15.175:0/3353405683 >> 172.21.15.175:0/3353405683 conn(0x7fb4100ff5a0 msgr2=0x7fb4101019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 -- 172.21.15.175:0/3353405683 shutdown_connections 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.404+0000 7fb416800640 1 -- 172.21.15.175:0/3353405683 wait complete. 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.405+0000 7fb416800640 1 Processor -- start 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.405+0000 7fb416800640 1 -- start start 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb416800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4101038e0 0x7fb41019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410104280 0x7fb41019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb416800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 0x7fb41019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb416800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb410113e40 con 0x7fb4101099f0 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb416800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb410113cc0 con 0x7fb4101038e0 2024-07-29T00:35:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb416800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb410113fc0 con 0x7fb410104280 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 0x7fb41019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 0x7fb41019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33694/0 (socket says 172.21.15.175:33694) 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 -- 172.21.15.175:0/568295490 learned_addr learned my addr 172.21.15.175:0/568295490 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb414a00640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410104280 0x7fb41019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415400640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4101038e0 0x7fb41019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 -- 172.21.15.175:0/568295490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410104280 msgr2=0x7fb41019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410104280 0x7fb41019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 -- 172.21.15.175:0/568295490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4101038e0 msgr2=0x7fb41019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4101038e0 0x7fb41019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415e00640 1 -- 172.21.15.175:0/568295490 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41019f2b0 con 0x7fb4101099f0 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.406+0000 7fb415400640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4101038e0 0x7fb41019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:09.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.407+0000 7fb415e00640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 0x7fb41019ebd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb40800eb10 tx=0x7fb40800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.407+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408004b20 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.407+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408004cc0 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.407+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41019f5a0 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.407+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408010760 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.407+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a6e80 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.408+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410103d40 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.410+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb408005020 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.413+0000 7fb40e000640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb3f80a2c10 0x7fb3f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.413+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fb4080cf8b0 con 0x7fb4101099f0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.413+0000 7fb415400640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb3f80a2c10 0x7fb3f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.414+0000 7fb415400640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb3f80a2c10 0x7fb3f80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb404009ba0 tx=0x7fb404005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:09.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.417+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb40809c630 con 0x7fb4101099f0 2024-07-29T00:35:09.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.651+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb410105080 con 0x7fb4101099f0 2024-07-29T00:35:09.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.652+0000 7fb40e000640 1 -- 172.21.15.175:0/568295490 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb40809bf30 con 0x7fb4101099f0 2024-07-29T00:35:09.743 INFO:teuthology.orchestra.run.smithi175.stdout:107374182468 2024-07-29T00:35:09.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.655+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb3f80a2c10 msgr2=0x7fb3f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.655+0000 7fb416800640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb3f80a2c10 0x7fb3f80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb404009ba0 tx=0x7fb404005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:09.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.655+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 msgr2=0x7fb41019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.655+0000 7fb416800640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 0x7fb41019ebd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb40800eb10 tx=0x7fb40800c2d0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 shutdown_connections 2024-07-29T00:35:09.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4101099f0 0x7fb41019ebd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb3f80a2c10 0x7fb3f80a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410104280 0x7fb41019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 --2- 172.21.15.175:0/568295490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4101038e0 0x7fb41019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 >> 172.21.15.175:0/568295490 conn(0x7fb4100ff5a0 msgr2=0x7fb4100ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 shutdown_connections 2024-07-29T00:35:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.656+0000 7fb416800640 1 -- 172.21.15.175:0/568295490 wait complete. 2024-07-29T00:35:09.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.824+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2814111af0 con 0x7f281410bcb0 2024-07-29T00:35:09.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.825+0000 7f280be00640 1 -- 172.21.15.175:0/4085529953 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f280c09bc10 con 0x7f281410bcb0 2024-07-29T00:35:09.825 INFO:teuthology.orchestra.run.smithi175.stdout:163208757308 2024-07-29T00:35:09.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f27f80a2ce0 msgr2=0x7f27f80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f27f80a2ce0 0x7f27f80a51a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2804009bd0 tx=0x7f2804005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:09.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 msgr2=0x7f281411bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:09.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 0x7f281411bad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f280c00efd0 tx=0x7f280c00c6a0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 shutdown_connections 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f27f80a2ce0 0x7f27f80a51a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f281410bcb0 0x7f281411bad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28141031a0 0x7f28141162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 --2- 172.21.15.175:0/4085529953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2814102880 0x7f2814115db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 >> 172.21.15.175:0/4085529953 conn(0x7f28140fc300 msgr2=0x7f28140fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 shutdown_connections 2024-07-29T00:35:09.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:09.828+0000 7f2818e00640 1 -- 172.21.15.175:0/4085529953 wait complete. 2024-07-29T00:35:09.950 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:09 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2434262542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-29T00:35:09.950 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:09 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/568295490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-29T00:35:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:09 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2434262542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-29T00:35:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:09 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/568295490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-29T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:09 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2434262542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-29T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:09 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/568295490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-29T00:35:10.698 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:10 smithi175 ceph-mon[32138]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:10.698 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:10 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/4085529953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-29T00:35:10.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.988+0000 7f4ed8c00640 1 -- 172.21.15.175:0/2164923015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076410 msgr2=0x7f4ed40767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.988+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/2164923015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076410 0x7f4ed40767f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009860 tx=0x7f4ec402a360 comp rx=0 tx=0).stop 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 -- 172.21.15.175:0/2164923015 shutdown_connections 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/2164923015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4112560 0x7f4ed4114950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/2164923015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed4076d30 0x7f4ed4112020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/2164923015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076410 0x7f4ed40767f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 -- 172.21.15.175:0/2164923015 >> 172.21.15.175:0/2164923015 conn(0x7f4ed40ff600 msgr2=0x7f4ed4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 -- 172.21.15.175:0/2164923015 shutdown_connections 2024-07-29T00:35:10.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.989+0000 7f4ed8c00640 1 -- 172.21.15.175:0/2164923015 wait complete. 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.990+0000 7f4ed8c00640 1 Processor -- start 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.990+0000 7f4ed8c00640 1 -- start start 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 0x7f4ed419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076d30 0x7f4ed419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed4112560 0x7f4ed41a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41180c0 con 0x7f4ed4112560 2024-07-29T00:35:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4117f40 con 0x7f4ed4076d30 2024-07-29T00:35:10.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4118240 con 0x7f4ed4076410 2024-07-29T00:35:10.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076d30 0x7f4ed419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 0x7f4ed419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 0x7f4ed419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:56030/0 (socket says 172.21.15.175:56030) 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed3400640 1 -- 172.21.15.175:0/3742818805 learned_addr learned my addr 172.21.15.175:0/3742818805 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed4112560 0x7f4ed41a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.991+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076d30 0x7f4ed419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35006/0 (socket says 172.21.15.175:35006) 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ed3400640 1 -- 172.21.15.175:0/3742818805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076d30 msgr2=0x7f4ed419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ed3400640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076d30 0x7f4ed419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ed3400640 1 -- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed4112560 msgr2=0x7f4ed41a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ed3400640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed4112560 0x7f4ed41a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ed3400640 1 -- 172.21.15.175:0/3742818805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41a3570 con 0x7f4ed4076410 2024-07-29T00:35:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ed3400640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 0x7f4ed419e5c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009980 tx=0x7f4ec4032740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:10.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.992+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4037040 con 0x7f4ed4076410 2024-07-29T00:35:10.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.993+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41a3800 con 0x7f4ed4076410 2024-07-29T00:35:10.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.993+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec4042070 con 0x7f4ed4076410 2024-07-29T00:35:10.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.993+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4003850 con 0x7f4ed4076410 2024-07-29T00:35:10.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.993+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41ab140 con 0x7f4ed4076410 2024-07-29T00:35:10.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.994+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed4077b50 con 0x7f4ed4076410 2024-07-29T00:35:10.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.996+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4ec40039f0 con 0x7f4ed4076410 2024-07-29T00:35:10.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.997+0000 7f4ecbe00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4eb80a2c90 0x7f4eb80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:10.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.997+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f4ec40ef1f0 con 0x7f4ed4076410 2024-07-29T00:35:10.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.997+0000 7f4ed2a00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4eb80a2c90 0x7f4eb80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:10.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.998+0000 7f4ed2a00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4eb80a2c90 0x7f4eb80a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4079ad0 tx=0x7f4ec0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:10 smithi181 ceph-mon[36706]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:10.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:10 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/4085529953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-29T00:35:11.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.001+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4ec40bc080 con 0x7f4ed4076410 2024-07-29T00:35:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:10 smithi204 ceph-mon[37368]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:10 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/4085529953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-29T00:35:11.167 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:11.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.205+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4ed41ab310 con 0x7f4ed4076410 2024-07-29T00:35:11.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.206+0000 7f4ecbe00640 1 -- 172.21.15.175:0/3742818805 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ec40bb980 con 0x7f4ed4076410 2024-07-29T00:35:11.206 INFO:teuthology.orchestra.run.smithi175.stdout:283467841575 2024-07-29T00:35:11.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4eb80a2c90 msgr2=0x7f4eb80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:11.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4eb80a2c90 0x7f4eb80a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4079ad0 tx=0x7f4ec0009260 comp rx=0 tx=0).stop 2024-07-29T00:35:11.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 msgr2=0x7f4ed419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:11.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 0x7f4ed419e5c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009980 tx=0x7f4ec4032740 comp rx=0 tx=0).stop 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 shutdown_connections 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ed4112560 0x7f4ed41a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed4076d30 0x7f4ed419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f4eb80a2c90 0x7f4eb80a5150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 --2- 172.21.15.175:0/3742818805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4076410 0x7f4ed419e5c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 >> 172.21.15.175:0/3742818805 conn(0x7f4ed40ff600 msgr2=0x7f4ed41019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:11.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.209+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 shutdown_connections 2024-07-29T00:35:11.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.210+0000 7f4ed8c00640 1 -- 172.21.15.175:0/3742818805 wait complete. 2024-07-29T00:35:11.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495669 for osd.6 2024-07-29T00:35:11.881 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:11 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3594355043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-29T00:35:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:11 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3742818805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-29T00:35:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:11 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3594355043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-29T00:35:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:11 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3742818805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-29T00:35:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:11 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3594355043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-29T00:35:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:11 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3742818805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-29T00:35:12.931 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:12 smithi175 ceph-mon[32138]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:12.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416410 got 339302416413 for osd.11 2024-07-29T00:35:12.986 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:12.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:12 smithi181 ceph-mon[36706]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:13.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 -- 172.21.15.175:0/3361866693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241098a0 msgr2=0x7fe724109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:13.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 --2- 172.21.15.175:0/3361866693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241098a0 0x7fe724109c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe714009930 tx=0x7fe71402a420 comp rx=0 tx=0).stop 2024-07-29T00:35:13.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 -- 172.21.15.175:0/3361866693 shutdown_connections 2024-07-29T00:35:13.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 --2- 172.21.15.175:0/3361866693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe724104240 0x7fe724110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 --2- 172.21.15.175:0/3361866693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7241038a0 0x7fe724103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 --2- 172.21.15.175:0/3361866693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241098a0 0x7fe724109c80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 -- 172.21.15.175:0/3361866693 >> 172.21.15.175:0/3361866693 conn(0x7fe7240ff560 msgr2=0x7fe724101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 -- 172.21.15.175:0/3361866693 shutdown_connections 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.850+0000 7fe72a400640 1 -- 172.21.15.175:0/3361866693 wait complete. 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.851+0000 7fe72a400640 1 Processor -- start 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 -- start start 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 0x7fe72419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe724104240 0x7fe72419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7241098a0 0x7fe72419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:13.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113cb0 con 0x7fe7241038a0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113b30 con 0x7fe7241098a0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe72a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113e30 con 0x7fe724104240 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe729000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 0x7fe72419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe729000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 0x7fe72419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33740/0 (socket says 172.21.15.175:33740) 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe729000640 1 -- 172.21.15.175:0/3594355043 learned_addr learned my addr 172.21.15.175:0/3594355043 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe71be00640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe724104240 0x7fe72419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.852+0000 7fe729a00640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7241098a0 0x7fe72419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe729000640 1 -- 172.21.15.175:0/3594355043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe724104240 msgr2=0x7fe72419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe729000640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe724104240 0x7fe72419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe729000640 1 -- 172.21.15.175:0/3594355043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7241098a0 msgr2=0x7fe72419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe729000640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7241098a0 0x7fe72419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe729000640 1 -- 172.21.15.175:0/3594355043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72419f1a0 con 0x7fe7241038a0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe729000640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 0x7fe72419a1f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe71402a940 tx=0x7fe714003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:13.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71403f070 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72419f430 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe714031c90 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714035030 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.853+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a6d70 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.855+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe724104fe0 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.856+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe714042070 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.858+0000 7fe722000640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe7080a2c10 0x7fe7080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.858+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fe7140ef5b0 con 0x7fe7241038a0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.858+0000 7fe71be00640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe7080a2c10 0x7fe7080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.859+0000 7fe71be00640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe7080a2c10 0x7fe7080a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe724101b90 tx=0x7fe70c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:13.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:10.863+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7140bbce0 con 0x7fe7241038a0 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stdout:77309411404 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.064+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe72410be10 con 0x7fe7241038a0 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.065+0000 7fe722000640 1 -- 172.21.15.175:0/3594355043 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe7140bffe0 con 0x7fe7241038a0 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe7080a2c10 msgr2=0x7fe7080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe7080a2c10 0x7fe7080a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe724101b90 tx=0x7fe70c005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 msgr2=0x7fe72419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:13.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 0x7fe72419a1f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe71402a940 tx=0x7fe714003b90 comp rx=0 tx=0).stop 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 shutdown_connections 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7241098a0 0x7fe72419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe7080a2c10 0x7fe7080a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe724104240 0x7fe72419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 --2- 172.21.15.175:0/3594355043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7241038a0 0x7fe72419a1f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.068+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 >> 172.21.15.175:0/3594355043 conn(0x7fe7240ff560 msgr2=0x7fe7240ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.069+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 shutdown_connections 2024-07-29T00:35:13.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:11.069+0000 7fe72a400640 1 -- 172.21.15.175:0/3594355043 wait complete. 2024-07-29T00:35:13.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:12 smithi204 ceph-mon[37368]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:14.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103209 got 249108103212 for osd.8 2024-07-29T00:35:14.024 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:14.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:14 smithi175 ceph-mon[32138]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:14 smithi181 ceph-mon[36706]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:15.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:14 smithi204 ceph-mon[37368]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:15.142 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841572 got 283467841575 for osd.9 2024-07-29T00:35:15.142 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:15.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757308 for osd.5 2024-07-29T00:35:15.986 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 -- 172.21.15.175:0/1850771423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e0102a30 msgr2=0x7f24e0102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1850771423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e0102a30 0x7f24e0102e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009900 tx=0x7f24d402a480 comp rx=0 tx=0).stop 2024-07-29T00:35:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 -- 172.21.15.175:0/1850771423 shutdown_connections 2024-07-29T00:35:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1850771423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e010be60 0x7f24e010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1850771423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 0x7f24e010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1850771423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e0102a30 0x7f24e0102e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 -- 172.21.15.175:0/1850771423 >> 172.21.15.175:0/1850771423 conn(0x7f24e00fc490 msgr2=0x7f24e00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 -- 172.21.15.175:0/1850771423 shutdown_connections 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.164+0000 7f24e7a00640 1 -- 172.21.15.175:0/1850771423 wait complete. 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.165+0000 7f24e7a00640 1 Processor -- start 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.165+0000 7f24e7a00640 1 -- start start 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.165+0000 7f24e7a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e0102a30 0x7f24e019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.165+0000 7f24e7a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 0x7f24e019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.165+0000 7f24e7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 0x7f24e019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24e0111c60 con 0x7f24e010be60 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e7a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24e0111ae0 con 0x7f24e0102a30 2024-07-29T00:35:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e7a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24e0111de0 con 0x7f24e0103350 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 0x7f24e019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 0x7f24e019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33794/0 (socket says 172.21.15.175:33794) 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 -- 172.21.15.175:0/1207757574 learned_addr learned my addr 172.21.15.175:0/1207757574 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24dfe00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 0x7f24e019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 -- 172.21.15.175:0/1207757574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 msgr2=0x7f24e019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 0x7f24e019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 -- 172.21.15.175:0/1207757574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e0102a30 msgr2=0x7f24e019a360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e0102a30 0x7f24e019a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24e5a00640 1 -- 172.21.15.175:0/1207757574 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e019f310 con 0x7f24e010be60 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.166+0000 7f24dfe00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 0x7f24e019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.167+0000 7f24e5a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 0x7f24e019ec30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f24d800eb10 tx=0x7f24d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.167+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8004b20 con 0x7f24e010be60 2024-07-29T00:35:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.167+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d8004cc0 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.167+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8010760 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.167+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e019f600 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.167+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01a6ee0 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.168+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e0102e10 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.170+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f24d8005020 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.172+0000 7f24dd600640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24c80a2c10 0x7f24c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.172+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f24d80cf7f0 con 0x7f24e010be60 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.173+0000 7f24e5000640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24c80a2c10 0x7f24c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.173+0000 7f24e5000640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24c80a2c10 0x7f24c80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f24d402a9a0 tx=0x7f24d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.176+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f24d809c570 con 0x7f24e010be60 2024-07-29T00:35:16.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.369+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f24e01a7140 con 0x7f24e010be60 2024-07-29T00:35:16.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.370+0000 7f24dd600640 1 -- 172.21.15.175:0/1207757574 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f24d809be70 con 0x7f24e010be60 2024-07-29T00:35:16.371 INFO:teuthology.orchestra.run.smithi175.stdout:219043332146 2024-07-29T00:35:16.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24c80a2c10 msgr2=0x7f24c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:16.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24c80a2c10 0x7f24c80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f24d402a9a0 tx=0x7f24d4000bc0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 msgr2=0x7f24e019ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:16.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 0x7f24e019ec30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f24d800eb10 tx=0x7f24d800c2d0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 shutdown_connections 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24e010be60 0x7f24e019ec30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f24c80a2c10 0x7f24c80a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e0103350 0x7f24e019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 --2- 172.21.15.175:0/1207757574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e0102a30 0x7f24e019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 >> 172.21.15.175:0/1207757574 conn(0x7f24e00fc490 msgr2=0x7f24e00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 shutdown_connections 2024-07-29T00:35:16.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:16.373+0000 7f24e7a00640 1 -- 172.21.15.175:0/1207757574 wait complete. 2024-07-29T00:35:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:16 smithi181 ceph-mon[36706]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:16 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1207757574' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-29T00:35:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:16 smithi175 ceph-mon[32138]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:16 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1207757574' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-29T00:35:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:16 smithi204 ceph-mon[37368]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:16 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1207757574' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-29T00:35:17.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182468 for osd.3 2024-07-29T00:35:17.327 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:17.406 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678049 for osd.10 2024-07-29T00:35:17.407 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:17.495 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:17.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411404 for osd.2 2024-07-29T00:35:17.516 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:18.059 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:18.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332146 for osd.7 2024-07-29T00:35:18.103 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:18.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 -- 172.21.15.175:0/2613477329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540109a80 msgr2=0x7fc540109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 --2- 172.21.15.175:0/2613477329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540109a80 0x7fc540109e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc530009930 tx=0x7fc53002a420 comp rx=0 tx=0).stop 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 -- 172.21.15.175:0/2613477329 shutdown_connections 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 --2- 172.21.15.175:0/2613477329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540104420 0x7fc540110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 --2- 172.21.15.175:0/2613477329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540103a80 0x7fc540103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 --2- 172.21.15.175:0/2613477329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540109a80 0x7fc540109e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 -- 172.21.15.175:0/2613477329 >> 172.21.15.175:0/2613477329 conn(0x7fc5400ff780 msgr2=0x7fc540101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 -- 172.21.15.175:0/2613477329 shutdown_connections 2024-07-29T00:35:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.884+0000 7fc545c00640 1 -- 172.21.15.175:0/2613477329 wait complete. 2024-07-29T00:35:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.885+0000 7fc545c00640 1 Processor -- start 2024-07-29T00:35:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 -- start start 2024-07-29T00:35:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540103a80 0x7fc54019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540104420 0x7fc54019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540109a80 0x7fc54019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:18.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc540113ed0 con 0x7fc540104420 2024-07-29T00:35:18.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc540113d50 con 0x7fc540103a80 2024-07-29T00:35:18.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.886+0000 7fc545c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc540114050 con 0x7fc540109a80 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540103a80 0x7fc54019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540109a80 0x7fc54019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540104420 0x7fc54019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540103a80 0x7fc54019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:49444/0 (socket says 172.21.15.175:49444) 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53ea00640 1 -- 172.21.15.175:0/3687764601 learned_addr learned my addr 172.21.15.175:0/3687764601 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 -- 172.21.15.175:0/3687764601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540103a80 msgr2=0x7fc54019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540103a80 0x7fc54019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 -- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540104420 msgr2=0x7fc54019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540104420 0x7fc54019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 -- 172.21.15.175:0/3687764601 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54019f340 con 0x7fc540109a80 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.887+0000 7fc53f400640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540109a80 0x7fc54019ec60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc538009fd0 tx=0x7fc53800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.888+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800ce20 con 0x7fc540109a80 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.888+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc538005020 con 0x7fc540109a80 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.888+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538010720 con 0x7fc540109a80 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.888+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54019f630 con 0x7fc540109a80 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.888+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5401a6f10 con 0x7fc540109a80 2024-07-29T00:35:18.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.889+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc540103ee0 con 0x7fc540109a80 2024-07-29T00:35:18.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.891+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc538005540 con 0x7fc540109a80 2024-07-29T00:35:18.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.893+0000 7fc537400640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc5240a2c10 0x7fc5240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:18.893 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:18 smithi175 ceph-mon[32138]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:35:18.893 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:18.894 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:18 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:18.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.893+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc5380cf840 con 0x7fc540109a80 2024-07-29T00:35:18.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.894+0000 7fc53ea00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc5240a2c10 0x7fc5240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:18.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.895+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc53809c5c0 con 0x7fc540109a80 2024-07-29T00:35:18.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:18.895+0000 7fc53ea00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc5240a2c10 0x7fc5240a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc5300096c0 tx=0x7fc530005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:18 smithi181 ceph-mon[36706]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:35:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:18 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:18 smithi204 ceph-mon[37368]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:35:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:18 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:19.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.092+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc54010ba60 con 0x7fc540109a80 2024-07-29T00:35:19.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.092+0000 7fc537400640 1 -- 172.21.15.175:0/3687764601 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc53809bec0 con 0x7fc540109a80 2024-07-29T00:35:19.093 INFO:teuthology.orchestra.run.smithi175.stdout:34359738453 2024-07-29T00:35:19.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc5240a2c10 msgr2=0x7fc5240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc5240a2c10 0x7fc5240a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc5300096c0 tx=0x7fc530005b50 comp rx=0 tx=0).stop 2024-07-29T00:35:19.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540109a80 msgr2=0x7fc54019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540109a80 0x7fc54019ec60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc538009fd0 tx=0x7fc53800ef10 comp rx=0 tx=0).stop 2024-07-29T00:35:19.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 shutdown_connections 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc5240a2c10 0x7fc5240a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc540109a80 0x7fc54019ec60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540104420 0x7fc54019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 --2- 172.21.15.175:0/3687764601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc540103a80 0x7fc54019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 >> 172.21.15.175:0/3687764601 conn(0x7fc5400ff780 msgr2=0x7fc540100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 shutdown_connections 2024-07-29T00:35:19.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.095+0000 7fc545c00640 1 -- 172.21.15.175:0/3687764601 wait complete. 2024-07-29T00:35:19.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 -- 172.21.15.175:0/1602820950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8109a80 msgr2=0x7fb1b8109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 --2- 172.21.15.175:0/1602820950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8109a80 0x7fb1b8109e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009930 tx=0x7fb1a802a420 comp rx=0 tx=0).stop 2024-07-29T00:35:19.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 -- 172.21.15.175:0/1602820950 shutdown_connections 2024-07-29T00:35:19.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 --2- 172.21.15.175:0/1602820950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 0x7fb1b8110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 --2- 172.21.15.175:0/1602820950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b8103a80 0x7fb1b8103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 --2- 172.21.15.175:0/1602820950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8109a80 0x7fb1b8109e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 -- 172.21.15.175:0/1602820950 >> 172.21.15.175:0/1602820950 conn(0x7fb1b80ff780 msgr2=0x7fb1b8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:19.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 -- 172.21.15.175:0/1602820950 shutdown_connections 2024-07-29T00:35:19.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.183+0000 7fb1bde00640 1 -- 172.21.15.175:0/1602820950 wait complete. 2024-07-29T00:35:19.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 Processor -- start 2024-07-29T00:35:19.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 -- start start 2024-07-29T00:35:19.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8103a80 0x7fb1b819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:19.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 0x7fb1b819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:19.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b8109a80 0x7fb1b819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8113ed0 con 0x7fb1b8104420 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8113d50 con 0x7fb1b8109a80 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.184+0000 7fb1bde00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8114050 con 0x7fb1b8103a80 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1b6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8103a80 0x7fb1b819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 0x7fb1b819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1b6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8103a80 0x7fb1b819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:39178/0 (socket says 172.21.15.175:39178) 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1b6a00640 1 -- 172.21.15.175:0/3939534273 learned_addr learned my addr 172.21.15.175:0/3939534273 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1b7400640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b8109a80 0x7fb1b819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 -- 172.21.15.175:0/3939534273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8103a80 msgr2=0x7fb1b819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8103a80 0x7fb1b819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 -- 172.21.15.175:0/3939534273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b8109a80 msgr2=0x7fb1b819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b8109a80 0x7fb1b819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 -- 172.21.15.175:0/3939534273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b819f330 con 0x7fb1b8104420 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1ae000640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 0x7fb1b819a8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d6d0 tx=0x7fb1a000dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.185+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0017070 con 0x7fb1b8104420 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.186+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a00047a0 con 0x7fb1b8104420 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.186+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b819f620 con 0x7fb1b8104420 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.186+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0009c80 con 0x7fb1b8104420 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.186+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81a6f00 con 0x7fb1b8104420 2024-07-29T00:35:19.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.187+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8103ee0 con 0x7fb1b8104420 2024-07-29T00:35:19.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.189+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb1a0004950 con 0x7fb1b8104420 2024-07-29T00:35:19.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.190+0000 7fb1bca00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb19c0a2c90 0x7fb19c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:19.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.190+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fb1a00cea00 con 0x7fb1b8104420 2024-07-29T00:35:19.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.190+0000 7fb1b6a00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb19c0a2c90 0x7fb19c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:19.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.191+0000 7fb1b6a00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb19c0a2c90 0x7fb19c0a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80096c0 tx=0x7fb1a8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:19.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.194+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb1a0009e20 con 0x7fb1b8104420 2024-07-29T00:35:19.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.388+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb1b810ba60 con 0x7fb1b8104420 2024-07-29T00:35:19.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.389+0000 7fb1bca00640 1 -- 172.21.15.175:0/3939534273 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1a009b780 con 0x7fb1b8104420 2024-07-29T00:35:19.389 INFO:teuthology.orchestra.run.smithi175.stdout:141733920834 2024-07-29T00:35:19.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb19c0a2c90 msgr2=0x7fb19c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb19c0a2c90 0x7fb19c0a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80096c0 tx=0x7fb1a8005b50 comp rx=0 tx=0).stop 2024-07-29T00:35:19.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 msgr2=0x7fb1b819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:19.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 0x7fb1b819a8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d6d0 tx=0x7fb1a000dad0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 shutdown_connections 2024-07-29T00:35:19.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b8109a80 0x7fb1b819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1b8104420 0x7fb1b819a8c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb19c0a2c90 0x7fb19c0a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 --2- 172.21.15.175:0/3939534273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1b8103a80 0x7fb1b819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 >> 172.21.15.175:0/3939534273 conn(0x7fb1b80ff780 msgr2=0x7fb1b8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 shutdown_connections 2024-07-29T00:35:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:19.392+0000 7fb1bde00640 1 -- 172.21.15.175:0/3939534273 wait complete. 2024-07-29T00:35:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:35:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:35:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:19 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3687764601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-29T00:35:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:19 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3939534273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-29T00:35:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:35:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:35:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:19 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3687764601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-29T00:35:20.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:19 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3939534273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-29T00:35:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:35:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:35:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:35:20.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:19 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3687764601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-29T00:35:20.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:19 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3939534273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-29T00:35:20.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738453 for osd.0 2024-07-29T00:35:20.463 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:20.969 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:20 smithi175 ceph-mon[32138]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:20 smithi181 ceph-mon[36706]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:21.038 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920834 for osd.4 2024-07-29T00:35:21.039 DEBUG:teuthology.parallel:result is None 2024-07-29T00:35:21.039 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-29T00:35:21.039 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:35:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:20 smithi204 ceph-mon[37368]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:21.504 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:22.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 -- 172.21.15.175:0/3839093050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 msgr2=0x7ff314109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:22.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 --2- 172.21.15.175:0/3839093050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 0x7ff314109e80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff308009860 tx=0x7ff30802a340 comp rx=0 tx=0).stop 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 -- 172.21.15.175:0/3839093050 shutdown_connections 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 --2- 172.21.15.175:0/3839093050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314104440 0x7ff314110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 --2- 172.21.15.175:0/3839093050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314103aa0 0x7ff314103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 --2- 172.21.15.175:0/3839093050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 0x7ff314109e80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 -- 172.21.15.175:0/3839093050 >> 172.21.15.175:0/3839093050 conn(0x7ff3140ff740 msgr2=0x7ff314101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 -- 172.21.15.175:0/3839093050 shutdown_connections 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.134+0000 7ff31b400640 1 -- 172.21.15.175:0/3839093050 wait complete. 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.135+0000 7ff31b400640 1 Processor -- start 2024-07-29T00:35:22.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.135+0000 7ff31b400640 1 -- start start 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff31b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314103aa0 0x7ff31419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff31b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 0x7ff31419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff31b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 0x7ff31419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff31b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113ea0 con 0x7ff314103aa0 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff31b400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113d20 con 0x7ff314109aa0 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff31b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff314114020 con 0x7ff314104440 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff318a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314103aa0 0x7ff31419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff313e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 0x7ff31419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:22.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff313e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 0x7ff31419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:39192/0 (socket says 172.21.15.175:39192) 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff313e00640 1 -- 172.21.15.175:0/2012450375 learned_addr learned my addr 172.21.15.175:0/2012450375 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.136+0000 7ff319400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 0x7ff31419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff313e00640 1 -- 172.21.15.175:0/2012450375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 msgr2=0x7ff31419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff313e00640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 0x7ff31419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff313e00640 1 -- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314103aa0 msgr2=0x7ff31419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff313e00640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314103aa0 0x7ff31419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff313e00640 1 -- 172.21.15.175:0/2012450375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31419f330 con 0x7ff314104440 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff318a00640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314103aa0 0x7ff31419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff313e00640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 0x7ff31419a8c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff30000e9f0 tx=0x7ff30000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30000cd80 con 0x7ff314104440 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff300005500 con 0x7ff314104440 2024-07-29T00:35:22.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.137+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31419f620 con 0x7ff314104440 2024-07-29T00:35:22.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.138+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300010690 con 0x7ff314104440 2024-07-29T00:35:22.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.138+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3141a6f00 con 0x7ff314104440 2024-07-29T00:35:22.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.139+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff314103f00 con 0x7ff314104440 2024-07-29T00:35:22.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.143+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff3000046c0 con 0x7ff314104440 2024-07-29T00:35:22.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.145+0000 7ff311600640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2fc0a2c10 0x7ff2fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:22.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.145+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff3000cf630 con 0x7ff314104440 2024-07-29T00:35:22.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.145+0000 7ff318a00640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2fc0a2c10 0x7ff2fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:22.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.146+0000 7ff318a00640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2fc0a2c10 0x7ff2fc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff30802a930 tx=0x7ff3080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:22.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.149+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff30009c3b0 con 0x7ff314104440 2024-07-29T00:35:22.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.339+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff314062320 con 0x7ff2fc0a2c10 2024-07-29T00:35:22.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.349+0000 7ff311600640 1 -- 172.21.15.175:0/2012450375 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259029 (secure 0 0 0) 0x7ff314062320 con 0x7ff2fc0a2c10 2024-07-29T00:35:22.352 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:22.353 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.355+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2fc0a2c10 msgr2=0x7ff2fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.355+0000 7ff31b400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2fc0a2c10 0x7ff2fc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff30802a930 tx=0x7ff3080058d0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.355+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 msgr2=0x7ff31419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.355+0000 7ff31b400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 0x7ff31419a8c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff30000e9f0 tx=0x7ff30000edf0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 shutdown_connections 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff314109aa0 0x7ff31419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff2fc0a2c10 0x7ff2fc0a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314104440 0x7ff31419a8c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 --2- 172.21.15.175:0/2012450375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff314103aa0 0x7ff31419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:22.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 >> 172.21.15.175:0/2012450375 conn(0x7ff3140ff740 msgr2=0x7ff3140fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:22.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 shutdown_connections 2024-07-29T00:35:22.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:22.356+0000 7ff31b400640 1 -- 172.21.15.175:0/2012450375 wait complete. 2024-07-29T00:35:22.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:22 smithi181 ceph-mon[36706]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:23.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:22 smithi175 ceph-mon[32138]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:23.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:22 smithi204 ceph-mon[37368]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:23.359 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-07-29T00:35:21.570524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657175,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":787,"num_read_kb":537,"num_write":428,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357032,"kb_used_data":11956,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584792,"statfs":{"total":1151940427776,"available":1151574827008,"internally_reserved":0,"allocated":12242944,"data_stored":7658325,"data_compressed":186355,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005353"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703303+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:34:50.703303+0000","last_peered":"2024-07-29T00:34:50.703303+0000","last_clean":"2024-07-29T00:34:50.703303+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:34:50.703303+0000","last_undegraded":"2024-07-29T00:34:50.703303+0000","last_fullsized":"2024-07-29T00:34:50.703303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719061+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:34:50.719061+0000","last_peered":"2024-07-29T00:34:50.719061+0000","last_clean":"2024-07-29T00:34:50.719061+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:34:50.719061+0000","last_undegraded":"2024-07-29T00:34:50.719061+0000","last_fullsized":"2024-07-29T00:34:50.719061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.717894+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:34:50.717894+0000","last_peered":"2024-07-29T00:34:50.717894+0000","last_clean":"2024-07-29T00:34:50.717894+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:34:50.717894+0000","last_undegraded":"2024-07-29T00:34:50.717894+0000","last_fullsized":"2024-07-29T00:34:50.717894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704803+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:34:50.704803+0000","last_peered":"2024-07-29T00:34:50.704803+0000","last_clean":"2024-07-29T00:34:50.704803+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:34:50.704803+0000","last_undegraded":"2024-07-29T00:34:50.704803+0000","last_fullsized":"2024-07-29T00:34:50.704803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705530+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:34:50.705530+0000","last_peered":"2024-07-29T00:34:50.705530+0000","last_clean":"2024-07-29T00:34:50.705530+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:34:50.705530+0000","last_undegraded":"2024-07-29T00:34:50.705530+0000","last_fullsized":"2024-07-29T00:34:50.705530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700671+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:34:50.700671+0000","last_peered":"2024-07-29T00:34:50.700671+0000","last_clean":"2024-07-29T00:34:50.700671+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:34:50.700671+0000","last_undegraded":"2024-07-29T00:34:50.700671+0000","last_fullsized":"2024-07-29T00:34:50.700671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.713666+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:34:50.713666+0000","last_peered":"2024-07-29T00:34:50.713666+0000","last_clean":"2024-07-29T00:34:50.713666+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:34:50.713666+0000","last_undegraded":"2024-07-29T00:34:50.713666+0000","last_fullsized":"2024-07-29T00:34:50.713666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.707186+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:34:50.707186+0000","last_peered":"2024-07-29T00:34:50.707186+0000","last_clean":"2024-07-29T00:34:50.707186+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:34:50.707186+0000","last_undegraded":"2024-07-29T00:34:50.707186+0000","last_fullsized":"2024-07-29T00:34:50.707186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704617+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:34:50.704617+0000","last_peered":"2024-07-29T00:34:50.704617+0000","last_clean":"2024-07-29T00:34:50.704617+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:34:50.704617+0000","last_undegraded":"2024-07-29T00:34:50.704617+0000","last_fullsized":"2024-07-29T00:34:50.704617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.711612+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:34:50.711612+0000","last_peered":"2024-07-29T00:34:50.711612+0000","last_clean":"2024-07-29T00:34:50.711612+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:34:50.711612+0000","last_undegraded":"2024-07-29T00:34:50.711612+0000","last_fullsized":"2024-07-29T00:34:50.711612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705847+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:34:50.705847+0000","last_peered":"2024-07-29T00:34:50.705847+0000","last_clean":"2024-07-29T00:34:50.705847+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:34:50.705847+0000","last_undegraded":"2024-07-29T00:34:50.705847+0000","last_fullsized":"2024-07-29T00:34:50.705847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704931+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:34:50.704931+0000","last_peered":"2024-07-29T00:34:50.704931+0000","last_clean":"2024-07-29T00:34:50.704931+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:34:50.704931+0000","last_undegraded":"2024-07-29T00:34:50.704931+0000","last_fullsized":"2024-07-29T00:34:50.704931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721203+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:34:50.721203+0000","last_peered":"2024-07-29T00:34:50.721203+0000","last_clean":"2024-07-29T00:34:50.721203+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:34:50.721203+0000","last_undegraded":"2024-07-29T00:34:50.721203+0000","last_fullsized":"2024-07-29T00:34:50.721203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.718861+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:34:50.718861+0000","last_peered":"2024-07-29T00:34:50.718861+0000","last_clean":"2024-07-29T00:34:50.718861+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:34:50.718861+0000","last_undegraded":"2024-07-29T00:34:50.718861+0000","last_fullsized":"2024-07-29T00:34:50.718861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719821+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:34:50.719821+0000","last_peered":"2024-07-29T00:34:50.719821+0000","last_clean":"2024-07-29T00:34:50.719821+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:34:50.719821+0000","last_undegraded":"2024-07-29T00:34:50.719821+0000","last_fullsized":"2024-07-29T00:34:50.719821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703557+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:34:50.703557+0000","last_peered":"2024-07-29T00:34:50.703557+0000","last_clean":"2024-07-29T00:34:50.703557+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:34:50.703557+0000","last_undegraded":"2024-07-29T00:34:50.703557+0000","last_fullsized":"2024-07-29T00:34:50.703557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703375+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:34:50.703375+0000","last_peered":"2024-07-29T00:34:50.703375+0000","last_clean":"2024-07-29T00:34:50.703375+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:34:50.703375+0000","last_undegraded":"2024-07-29T00:34:50.703375+0000","last_fullsized":"2024-07-29T00:34:50.703375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705353+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:34:50.705353+0000","last_peered":"2024-07-29T00:34:50.705353+0000","last_clean":"2024-07-29T00:34:50.705353+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:34:50.705353+0000","last_undegraded":"2024-07-29T00:34:50.705353+0000","last_fullsized":"2024-07-29T00:34:50.705353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700645+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:34:50.700645+0000","last_peered":"2024-07-29T00:34:50.700645+0000","last_clean":"2024-07-29T00:34:50.700645+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:34:50.700645+0000","last_undegraded":"2024-07-29T00:34:50.700645+0000","last_fullsized":"2024-07-29T00:34:50.700645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702773+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:34:50.702773+0000","last_peered":"2024-07-29T00:34:50.702773+0000","last_clean":"2024-07-29T00:34:50.702773+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:34:50.702773+0000","last_undegraded":"2024-07-29T00:34:50.702773+0000","last_fullsized":"2024-07-29T00:34:50.702773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721310+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:34:50.721310+0000","last_peered":"2024-07-29T00:34:50.721310+0000","last_clean":"2024-07-29T00:34:50.721310+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:34:50.721310+0000","last_undegraded":"2024-07-29T00:34:50.721310+0000","last_fullsized":"2024-07-29T00:34:50.721310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704792+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:34:50.704792+0000","last_peered":"2024-07-29T00:34:50.704792+0000","last_clean":"2024-07-29T00:34:50.704792+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:34:50.704792+0000","last_undegraded":"2024-07-29T00:34:50.704792+0000","last_fullsized":"2024-07-29T00:34:50.704792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.707587+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:34:50.707587+0000","last_peered":"2024-07-29T00:34:50.707587+0000","last_clean":"2024-07-29T00:34:50.707587+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:34:50.707587+0000","last_undegraded":"2024-07-29T00:34:50.707587+0000","last_fullsized":"2024-07-29T00:34:50.707587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703874+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:34:50.703874+0000","last_peered":"2024-07-29T00:34:50.703874+0000","last_clean":"2024-07-29T00:34:50.703874+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:34:50.703874+0000","last_undegraded":"2024-07-29T00:34:50.703874+0000","last_fullsized":"2024-07-29T00:34:50.703874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.717173+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:34:50.717173+0000","last_peered":"2024-07-29T00:34:50.717173+0000","last_clean":"2024-07-29T00:34:50.717173+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:34:50.717173+0000","last_undegraded":"2024-07-29T00:34:50.717173+0000","last_fullsized":"2024-07-29T00:34:50.717173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.701012+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:34:50.701012+0000","last_peered":"2024-07-29T00:34:50.701012+0000","last_clean":"2024-07-29T00:34:50.701012+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:34:50.701012+0000","last_undegraded":"2024-07-29T00:34:50.701012+0000","last_fullsized":"2024-07-29T00:34:50.701012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703571+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:34:50.703571+0000","last_peered":"2024-07-29T00:34:50.703571+0000","last_clean":"2024-07-29T00:34:50.703571+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:34:50.703571+0000","last_undegraded":"2024-07-29T00:34:50.703571+0000","last_fullsized":"2024-07-29T00:34:50.703571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721769+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:34:50.721769+0000","last_peered":"2024-07-29T00:34:50.721769+0000","last_clean":"2024-07-29T00:34:50.721769+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:34:50.721769+0000","last_undegraded":"2024-07-29T00:34:50.721769+0000","last_fullsized":"2024-07-29T00:34:50.721769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.718554+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:34:50.718554+0000","last_peered":"2024-07-29T00:34:50.718554+0000","last_clean":"2024-07-29T00:34:50.718554+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:34:50.718554+0000","last_undegraded":"2024-07-29T00:34:50.718554+0000","last_fullsized":"2024-07-29T00:34:50.718554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700545+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:34:50.700545+0000","last_peered":"2024-07-29T00:34:50.700545+0000","last_clean":"2024-07-29T00:34:50.700545+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:34:50.700545+0000","last_undegraded":"2024-07-29T00:34:50.700545+0000","last_fullsized":"2024-07-29T00:34:50.700545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700790+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:34:50.700790+0000","last_peered":"2024-07-29T00:34:50.700790+0000","last_clean":"2024-07-29T00:34:50.700790+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:34:50.700790+0000","last_undegraded":"2024-07-29T00:34:50.700790+0000","last_fullsized":"2024-07-29T00:34:50.700790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702858+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:34:50.702858+0000","last_peered":"2024-07-29T00:34:50.702858+0000","last_clean":"2024-07-29T00:34:50.702858+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:34:50.702858+0000","last_undegraded":"2024-07-29T00:34:50.702858+0000","last_fullsized":"2024-07-29T00:34:50.702858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705851+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:34:50.705851+0000","last_peered":"2024-07-29T00:34:50.705851+0000","last_clean":"2024-07-29T00:34:50.705851+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:34:50.705851+0000","last_undegraded":"2024-07-29T00:34:50.705851+0000","last_fullsized":"2024-07-29T00:34:50.705851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:16.783207+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:35:16.783207+0000","last_peered":"2024-07-29T00:35:16.783207+0000","last_clean":"2024-07-29T00:35:16.783207+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:35:16.783207+0000","last_undegraded":"2024-07-29T00:35:16.783207+0000","last_fullsized":"2024-07-29T00:35:16.783207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703969+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:34:50.703969+0000","last_peered":"2024-07-29T00:34:50.703969+0000","last_clean":"2024-07-29T00:34:50.703969+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:34:50.703969+0000","last_undegraded":"2024-07-29T00:34:50.703969+0000","last_fullsized":"2024-07-29T00:34:50.703969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700826+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:34:50.700826+0000","last_peered":"2024-07-29T00:34:50.700826+0000","last_clean":"2024-07-29T00:34:50.700826+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:34:50.700826+0000","last_undegraded":"2024-07-29T00:34:50.700826+0000","last_fullsized":"2024-07-29T00:34:50.700826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720638+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:34:50.720638+0000","last_peered":"2024-07-29T00:34:50.720638+0000","last_clean":"2024-07-29T00:34:50.720638+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:34:50.720638+0000","last_undegraded":"2024-07-29T00:34:50.720638+0000","last_fullsized":"2024-07-29T00:34:50.720638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.707380+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:34:50.707380+0000","last_peered":"2024-07-29T00:34:50.707380+0000","last_clean":"2024-07-29T00:34:50.707380+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:34:50.707380+0000","last_undegraded":"2024-07-29T00:34:50.707380+0000","last_fullsized":"2024-07-29T00:34:50.707380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704260+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:34:50.704260+0000","last_peered":"2024-07-29T00:34:50.704260+0000","last_clean":"2024-07-29T00:34:50.704260+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:34:50.704260+0000","last_undegraded":"2024-07-29T00:34:50.704260+0000","last_fullsized":"2024-07-29T00:34:50.704260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702586+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:34:50.702586+0000","last_peered":"2024-07-29T00:34:50.702586+0000","last_clean":"2024-07-29T00:34:50.702586+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:34:50.702586+0000","last_undegraded":"2024-07-29T00:34:50.702586+0000","last_fullsized":"2024-07-29T00:34:50.702586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:19.756535+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:35:19.756535+0000","last_peered":"2024-07-29T00:35:19.756535+0000","last_clean":"2024-07-29T00:35:19.756535+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:35:19.756535+0000","last_undegraded":"2024-07-29T00:35:19.756535+0000","last_fullsized":"2024-07-29T00:35:19.756535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704354+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:34:50.704354+0000","last_peered":"2024-07-29T00:34:50.704354+0000","last_clean":"2024-07-29T00:34:50.704354+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:34:50.704354+0000","last_undegraded":"2024-07-29T00:34:50.704354+0000","last_fullsized":"2024-07-29T00:34:50.704354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704906+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:34:50.704906+0000","last_peered":"2024-07-29T00:34:50.704906+0000","last_clean":"2024-07-29T00:34:50.704906+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:34:50.704906+0000","last_undegraded":"2024-07-29T00:34:50.704906+0000","last_fullsized":"2024-07-29T00:34:50.704906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720205+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:34:50.720205+0000","last_peered":"2024-07-29T00:34:50.720205+0000","last_clean":"2024-07-29T00:34:50.720205+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:34:50.720205+0000","last_undegraded":"2024-07-29T00:34:50.720205+0000","last_fullsized":"2024-07-29T00:34:50.720205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704640+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:34:50.704640+0000","last_peered":"2024-07-29T00:34:50.704640+0000","last_clean":"2024-07-29T00:34:50.704640+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:34:50.704640+0000","last_undegraded":"2024-07-29T00:34:50.704640+0000","last_fullsized":"2024-07-29T00:34:50.704640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703890+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:34:50.703890+0000","last_peered":"2024-07-29T00:34:50.703890+0000","last_clean":"2024-07-29T00:34:50.703890+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:34:50.703890+0000","last_undegraded":"2024-07-29T00:34:50.703890+0000","last_fullsized":"2024-07-29T00:34:50.703890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":218,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:19.754750+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:35:19.754750+0000","last_peered":"2024-07-29T00:35:19.754750+0000","last_clean":"2024-07-29T00:35:19.754750+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:35:19.754750+0000","last_undegraded":"2024-07-29T00:35:19.754750+0000","last_fullsized":"2024-07-29T00:35:19.754750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59761,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":141,"num_read_kb":128,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706278+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:34:50.706278+0000","last_peered":"2024-07-29T00:34:50.706278+0000","last_clean":"2024-07-29T00:34:50.706278+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:34:50.706278+0000","last_undegraded":"2024-07-29T00:34:50.706278+0000","last_fullsized":"2024-07-29T00:34:50.706278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700884+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:34:50.700884+0000","last_peered":"2024-07-29T00:34:50.700884+0000","last_clean":"2024-07-29T00:34:50.700884+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:34:50.700884+0000","last_undegraded":"2024-07-29T00:34:50.700884+0000","last_fullsized":"2024-07-29T00:34:50.700884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721188+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:34:50.721188+0000","last_peered":"2024-07-29T00:34:50.721188+0000","last_clean":"2024-07-29T00:34:50.721188+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:34:50.721188+0000","last_undegraded":"2024-07-29T00:34:50.721188+0000","last_fullsized":"2024-07-29T00:34:50.721188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703792+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:34:50.703792+0000","last_peered":"2024-07-29T00:34:50.703792+0000","last_clean":"2024-07-29T00:34:50.703792+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:34:50.703792+0000","last_undegraded":"2024-07-29T00:34:50.703792+0000","last_fullsized":"2024-07-29T00:34:50.703792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704862+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:34:50.704862+0000","last_peered":"2024-07-29T00:34:50.704862+0000","last_clean":"2024-07-29T00:34:50.704862+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:34:50.704862+0000","last_undegraded":"2024-07-29T00:34:50.704862+0000","last_fullsized":"2024-07-29T00:34:50.704862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:15.835437+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:35:15.835437+0000","last_peered":"2024-07-29T00:35:15.835437+0000","last_clean":"2024-07-29T00:35:15.835437+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:35:15.835437+0000","last_undegraded":"2024-07-29T00:35:15.835437+0000","last_fullsized":"2024-07-29T00:35:15.835437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705397+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:34:50.705397+0000","last_peered":"2024-07-29T00:34:50.705397+0000","last_clean":"2024-07-29T00:34:50.705397+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:34:50.705397+0000","last_undegraded":"2024-07-29T00:34:50.705397+0000","last_fullsized":"2024-07-29T00:34:50.705397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706440+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:34:50.706440+0000","last_peered":"2024-07-29T00:34:50.706440+0000","last_clean":"2024-07-29T00:34:50.706440+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:34:50.706440+0000","last_undegraded":"2024-07-29T00:34:50.706440+0000","last_fullsized":"2024-07-29T00:34:50.706440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703734+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:34:50.703734+0000","last_peered":"2024-07-29T00:34:50.703734+0000","last_clean":"2024-07-29T00:34:50.703734+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:34:50.703734+0000","last_undegraded":"2024-07-29T00:34:50.703734+0000","last_fullsized":"2024-07-29T00:34:50.703734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706411+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:34:50.706411+0000","last_peered":"2024-07-29T00:34:50.706411+0000","last_clean":"2024-07-29T00:34:50.706411+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:34:50.706411+0000","last_undegraded":"2024-07-29T00:34:50.706411+0000","last_fullsized":"2024-07-29T00:34:50.706411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.712384+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:34:50.712384+0000","last_peered":"2024-07-29T00:34:50.712384+0000","last_clean":"2024-07-29T00:34:50.712384+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:34:50.712384+0000","last_undegraded":"2024-07-29T00:34:50.712384+0000","last_fullsized":"2024-07-29T00:34:50.712384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719859+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:34:50.719859+0000","last_peered":"2024-07-29T00:34:50.719859+0000","last_clean":"2024-07-29T00:34:50.719859+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:34:50.719859+0000","last_undegraded":"2024-07-29T00:34:50.719859+0000","last_fullsized":"2024-07-29T00:34:50.719859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705135+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:34:50.705135+0000","last_peered":"2024-07-29T00:34:50.705135+0000","last_clean":"2024-07-29T00:34:50.705135+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:34:50.705135+0000","last_undegraded":"2024-07-29T00:34:50.705135+0000","last_fullsized":"2024-07-29T00:34:50.705135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704649+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:34:50.704649+0000","last_peered":"2024-07-29T00:34:50.704649+0000","last_clean":"2024-07-29T00:34:50.704649+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:34:50.704649+0000","last_undegraded":"2024-07-29T00:34:50.704649+0000","last_fullsized":"2024-07-29T00:34:50.704649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704753+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:34:50.704753+0000","last_peered":"2024-07-29T00:34:50.704753+0000","last_clean":"2024-07-29T00:34:50.704753+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:34:50.704753+0000","last_undegraded":"2024-07-29T00:34:50.704753+0000","last_fullsized":"2024-07-29T00:34:50.704753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:15.752325+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:35:15.752325+0000","last_peered":"2024-07-29T00:35:15.752325+0000","last_clean":"2024-07-29T00:35:15.752325+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:35:15.752325+0000","last_undegraded":"2024-07-29T00:35:15.752325+0000","last_fullsized":"2024-07-29T00:35:15.752325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705922+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:34:50.705922+0000","last_peered":"2024-07-29T00:34:50.705922+0000","last_clean":"2024-07-29T00:34:50.705922+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:34:50.705922+0000","last_undegraded":"2024-07-29T00:34:50.705922+0000","last_fullsized":"2024-07-29T00:34:50.705922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705711+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:34:50.705711+0000","last_peered":"2024-07-29T00:34:50.705711+0000","last_clean":"2024-07-29T00:34:50.705711+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:34:50.705711+0000","last_undegraded":"2024-07-29T00:34:50.705711+0000","last_fullsized":"2024-07-29T00:34:50.705711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704769+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:34:50.704769+0000","last_peered":"2024-07-29T00:34:50.704769+0000","last_clean":"2024-07-29T00:34:50.704769+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:34:50.704769+0000","last_undegraded":"2024-07-29T00:34:50.704769+0000","last_fullsized":"2024-07-29T00:34:50.704769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.708949+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:34:50.708949+0000","last_peered":"2024-07-29T00:34:50.708949+0000","last_clean":"2024-07-29T00:34:50.708949+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:34:50.708949+0000","last_undegraded":"2024-07-29T00:34:50.708949+0000","last_fullsized":"2024-07-29T00:34:50.708949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705563+0000","last_change":"2024-07-29T00:33:53.680861+0000","last_active":"2024-07-29T00:34:50.705563+0000","last_peered":"2024-07-29T00:34:50.705563+0000","last_clean":"2024-07-29T00:34:50.705563+0000","last_became_active":"2024-07-29T00:33:53.680260+0000","last_became_peered":"2024-07-29T00:33:53.680260+0000","last_unstale":"2024-07-29T00:34:50.705563+0000","last_undegraded":"2024-07-29T00:34:50.705563+0000","last_fullsized":"2024-07-29T00:34:50.705563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:50:13.931513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720501+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:34:50.720501+0000","last_peered":"2024-07-29T00:34:50.720501+0000","last_clean":"2024-07-29T00:34:50.720501+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:34:50.720501+0000","last_undegraded":"2024-07-29T00:34:50.720501+0000","last_fullsized":"2024-07-29T00:34:50.720501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:17.839448+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:35:17.839448+0000","last_peered":"2024-07-29T00:35:17.839448+0000","last_clean":"2024-07-29T00:35:17.839448+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:35:17.839448+0000","last_undegraded":"2024-07-29T00:35:17.839448+0000","last_fullsized":"2024-07-29T00:35:17.839448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.722534+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:34:50.722534+0000","last_peered":"2024-07-29T00:34:50.722534+0000","last_clean":"2024-07-29T00:34:50.722534+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:34:50.722534+0000","last_undegraded":"2024-07-29T00:34:50.722534+0000","last_fullsized":"2024-07-29T00:34:50.722534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:15.826462+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:35:15.826462+0000","last_peered":"2024-07-29T00:35:15.826462+0000","last_clean":"2024-07-29T00:35:15.826462+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:35:15.826462+0000","last_undegraded":"2024-07-29T00:35:15.826462+0000","last_fullsized":"2024-07-29T00:35:15.826462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700858+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:34:50.700858+0000","last_peered":"2024-07-29T00:34:50.700858+0000","last_clean":"2024-07-29T00:34:50.700858+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:34:50.700858+0000","last_undegraded":"2024-07-29T00:34:50.700858+0000","last_fullsized":"2024-07-29T00:34:50.700858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704182+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:34:50.704182+0000","last_peered":"2024-07-29T00:34:50.704182+0000","last_clean":"2024-07-29T00:34:50.704182+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:34:50.704182+0000","last_undegraded":"2024-07-29T00:34:50.704182+0000","last_fullsized":"2024-07-29T00:34:50.704182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.701026+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:34:50.701026+0000","last_peered":"2024-07-29T00:34:50.701026+0000","last_clean":"2024-07-29T00:34:50.701026+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:34:50.701026+0000","last_undegraded":"2024-07-29T00:34:50.701026+0000","last_fullsized":"2024-07-29T00:34:50.701026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706398+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:34:50.706398+0000","last_peered":"2024-07-29T00:34:50.706398+0000","last_clean":"2024-07-29T00:34:50.706398+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:34:50.706398+0000","last_undegraded":"2024-07-29T00:34:50.706398+0000","last_fullsized":"2024-07-29T00:34:50.706398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721872+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:34:50.721872+0000","last_peered":"2024-07-29T00:34:50.721872+0000","last_clean":"2024-07-29T00:34:50.721872+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:34:50.721872+0000","last_undegraded":"2024-07-29T00:34:50.721872+0000","last_fullsized":"2024-07-29T00:34:50.721872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705416+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:34:50.705416+0000","last_peered":"2024-07-29T00:34:50.705416+0000","last_clean":"2024-07-29T00:34:50.705416+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:34:50.705416+0000","last_undegraded":"2024-07-29T00:34:50.705416+0000","last_fullsized":"2024-07-29T00:34:50.705416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.710003+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:34:50.710003+0000","last_peered":"2024-07-29T00:34:50.710003+0000","last_clean":"2024-07-29T00:34:50.710003+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:34:50.710003+0000","last_undegraded":"2024-07-29T00:34:50.710003+0000","last_fullsized":"2024-07-29T00:34:50.710003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704759+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:34:50.704759+0000","last_peered":"2024-07-29T00:34:50.704759+0000","last_clean":"2024-07-29T00:34:50.704759+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:34:50.704759+0000","last_undegraded":"2024-07-29T00:34:50.704759+0000","last_fullsized":"2024-07-29T00:34:50.704759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719233+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:34:50.719233+0000","last_peered":"2024-07-29T00:34:50.719233+0000","last_clean":"2024-07-29T00:34:50.719233+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:34:50.719233+0000","last_undegraded":"2024-07-29T00:34:50.719233+0000","last_fullsized":"2024-07-29T00:34:50.719233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703842+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:34:50.703842+0000","last_peered":"2024-07-29T00:34:50.703842+0000","last_clean":"2024-07-29T00:34:50.703842+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:34:50.703842+0000","last_undegraded":"2024-07-29T00:34:50.703842+0000","last_fullsized":"2024-07-29T00:34:50.703842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705626+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:34:50.705626+0000","last_peered":"2024-07-29T00:34:50.705626+0000","last_clean":"2024-07-29T00:34:50.705626+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:34:50.705626+0000","last_undegraded":"2024-07-29T00:34:50.705626+0000","last_fullsized":"2024-07-29T00:34:50.705626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.708382+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:34:50.708382+0000","last_peered":"2024-07-29T00:34:50.708382+0000","last_clean":"2024-07-29T00:34:50.708382+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:34:50.708382+0000","last_undegraded":"2024-07-29T00:34:50.708382+0000","last_fullsized":"2024-07-29T00:34:50.708382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702968+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:34:50.702968+0000","last_peered":"2024-07-29T00:34:50.702968+0000","last_clean":"2024-07-29T00:34:50.702968+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:34:50.702968+0000","last_undegraded":"2024-07-29T00:34:50.702968+0000","last_fullsized":"2024-07-29T00:34:50.702968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700557+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:34:50.700557+0000","last_peered":"2024-07-29T00:34:50.700557+0000","last_clean":"2024-07-29T00:34:50.700557+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:34:50.700557+0000","last_undegraded":"2024-07-29T00:34:50.700557+0000","last_fullsized":"2024-07-29T00:34:50.700557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.709356+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:34:50.709356+0000","last_peered":"2024-07-29T00:34:50.709356+0000","last_clean":"2024-07-29T00:34:50.709356+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:34:50.709356+0000","last_undegraded":"2024-07-29T00:34:50.709356+0000","last_fullsized":"2024-07-29T00:34:50.709356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705687+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:34:50.705687+0000","last_peered":"2024-07-29T00:34:50.705687+0000","last_clean":"2024-07-29T00:34:50.705687+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:34:50.705687+0000","last_undegraded":"2024-07-29T00:34:50.705687+0000","last_fullsized":"2024-07-29T00:34:50.705687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705851+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:34:50.705851+0000","last_peered":"2024-07-29T00:34:50.705851+0000","last_clean":"2024-07-29T00:34:50.705851+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:34:50.705851+0000","last_undegraded":"2024-07-29T00:34:50.705851+0000","last_fullsized":"2024-07-29T00:34:50.705851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704533+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:34:50.704533+0000","last_peered":"2024-07-29T00:34:50.704533+0000","last_clean":"2024-07-29T00:34:50.704533+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:34:50.704533+0000","last_undegraded":"2024-07-29T00:34:50.704533+0000","last_fullsized":"2024-07-29T00:34:50.704533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706325+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:34:50.706325+0000","last_peered":"2024-07-29T00:34:50.706325+0000","last_clean":"2024-07-29T00:34:50.706325+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:34:50.706325+0000","last_undegraded":"2024-07-29T00:34:50.706325+0000","last_fullsized":"2024-07-29T00:34:50.706325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705213+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:34:50.705213+0000","last_peered":"2024-07-29T00:34:50.705213+0000","last_clean":"2024-07-29T00:34:50.705213+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:34:50.705213+0000","last_undegraded":"2024-07-29T00:34:50.705213+0000","last_fullsized":"2024-07-29T00:34:50.705213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720521+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:34:50.720521+0000","last_peered":"2024-07-29T00:34:50.720521+0000","last_clean":"2024-07-29T00:34:50.720521+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:34:50.720521+0000","last_undegraded":"2024-07-29T00:34:50.720521+0000","last_fullsized":"2024-07-29T00:34:50.720521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720056+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:34:50.720056+0000","last_peered":"2024-07-29T00:34:50.720056+0000","last_clean":"2024-07-29T00:34:50.720056+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:34:50.720056+0000","last_undegraded":"2024-07-29T00:34:50.720056+0000","last_fullsized":"2024-07-29T00:34:50.720056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.711057+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:34:50.711057+0000","last_peered":"2024-07-29T00:34:50.711057+0000","last_clean":"2024-07-29T00:34:50.711057+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:34:50.711057+0000","last_undegraded":"2024-07-29T00:34:50.711057+0000","last_fullsized":"2024-07-29T00:34:50.711057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.716169+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:34:50.716169+0000","last_peered":"2024-07-29T00:34:50.716169+0000","last_clean":"2024-07-29T00:34:50.716169+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:34:50.716169+0000","last_undegraded":"2024-07-29T00:34:50.716169+0000","last_fullsized":"2024-07-29T00:34:50.716169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":104,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:15.825580+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:35:15.825580+0000","last_peered":"2024-07-29T00:35:15.825580+0000","last_clean":"2024-07-29T00:35:15.825580+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:35:15.825580+0000","last_undegraded":"2024-07-29T00:35:15.825580+0000","last_fullsized":"2024-07-29T00:35:15.825580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66573,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":651,"num_read_kb":418,"num_write":245,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":199719,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247654,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":475092,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1126272,"data_compressed":41617,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":475022,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":475400,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1067917,"data_compressed":41617,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":474739,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1067205,"data_compressed":41617,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":535921,"data_compressed":15352,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":474564,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":474702,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":535413,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":476078,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":62021,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:35:23.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:35:23.833 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:23.979 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:23 smithi175 ceph-mon[32138]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:35:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:23 smithi181 ceph-mon[36706]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:35:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:23 smithi204 ceph-mon[37368]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:35:24.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.486+0000 7f95cbe00640 1 -- 172.21.15.175:0/130968566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076d30 msgr2=0x7f95cc112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.486+0000 7f95cbe00640 1 --2- 172.21.15.175:0/130968566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076d30 0x7f95cc112040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009900 tx=0x7f95b802a410 comp rx=0 tx=0).stop 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 -- 172.21.15.175:0/130968566 shutdown_connections 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 --2- 172.21.15.175:0/130968566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc112580 0x7f95cc114970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 --2- 172.21.15.175:0/130968566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076d30 0x7f95cc112040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 --2- 172.21.15.175:0/130968566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc076410 0x7f95cc0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 -- 172.21.15.175:0/130968566 >> 172.21.15.175:0/130968566 conn(0x7f95cc0ff5a0 msgr2=0x7f95cc1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 -- 172.21.15.175:0/130968566 shutdown_connections 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.487+0000 7f95cbe00640 1 -- 172.21.15.175:0/130968566 wait complete. 2024-07-29T00:35:24.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 Processor -- start 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 -- start start 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 0x7f95cc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc076d30 0x7f95cc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc112580 0x7f95cc1a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc118220 con 0x7f95cc112580 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1180a0 con 0x7f95cc076410 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.488+0000 7f95cbe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1183a0 con 0x7f95cc076d30 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc112580 0x7f95cc1a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 0x7f95cc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc112580 0x7f95cc1a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51166/0 (socket says 172.21.15.175:51166) 2024-07-29T00:35:24.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95cb400640 1 -- 172.21.15.175:0/2048266164 learned_addr learned my addr 172.21.15.175:0/2048266164 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 0x7f95cc19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:49516/0 (socket says 172.21.15.175:49516) 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95ca000640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc076d30 0x7f95cc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 -- 172.21.15.175:0/2048266164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc076d30 msgr2=0x7f95cc19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc076d30 0x7f95cc19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 -- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc112580 msgr2=0x7f95cc1a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc112580 0x7f95cc1a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 -- 172.21.15.175:0/2048266164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc1a3670 con 0x7f95cc076410 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.489+0000 7f95caa00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 0x7f95cc19e6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.490+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00ccc0 con 0x7f95cc076410 2024-07-29T00:35:24.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.490+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95bc010910 con 0x7f95cc076410 2024-07-29T00:35:24.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.490+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc1a3960 con 0x7f95cc076410 2024-07-29T00:35:24.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.490+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc018960 con 0x7f95cc076410 2024-07-29T00:35:24.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.490+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc1ab240 con 0x7f95cc076410 2024-07-29T00:35:24.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.491+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc077b50 con 0x7f95cc076410 2024-07-29T00:35:24.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.493+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f95bc010ac0 con 0x7f95cc076410 2024-07-29T00:35:24.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.494+0000 7f95d0a00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f95b00a2c10 0x7f95b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:24.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.494+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f95bc0cf930 con 0x7f95cc076410 2024-07-29T00:35:24.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.495+0000 7f95ca000640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f95b00a2c10 0x7f95b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:24.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.495+0000 7f95ca000640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f95b00a2c10 0x7f95b00a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009230 tx=0x7f95b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:24.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.496+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f95bc09c060 con 0x7f95cc076410 2024-07-29T00:35:24.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.683+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95cc1141f0 con 0x7f95b00a2c10 2024-07-29T00:35:24.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.703+0000 7f95d0a00640 1 -- 172.21.15.175:0/2048266164 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259032 (secure 0 0 0) 0x7f95cc1141f0 con 0x7f95b00a2c10 2024-07-29T00:35:24.706 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:24.709 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f95b00a2c10 msgr2=0x7f95b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f95b00a2c10 0x7f95b00a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009230 tx=0x7f95b80058d0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 msgr2=0x7f95cc19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 0x7f95cc19e6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 shutdown_connections 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95cc112580 0x7f95cc1a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f95b00a2c10 0x7f95b00a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc076d30 0x7f95cc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 --2- 172.21.15.175:0/2048266164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95cc076410 0x7f95cc19e6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:24.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.710+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 >> 172.21.15.175:0/2048266164 conn(0x7f95cc0ff5a0 msgr2=0x7f95cc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:24.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.712+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 shutdown_connections 2024-07-29T00:35:24.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:24.712+0000 7f95cbe00640 1 -- 172.21.15.175:0/2048266164 wait complete. 2024-07-29T00:35:24.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:24 smithi181 ceph-mon[36706]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:35:25.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:24 smithi175 ceph-mon[32138]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:35:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:24 smithi204 ceph-mon[37368]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:35:25.743 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-07-29T00:35:23.570980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657175,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":812,"num_read_kb":542,"num_write":428,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357032,"kb_used_data":11956,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584792,"statfs":{"total":1151940427776,"available":1151574827008,"internally_reserved":0,"allocated":12242944,"data_stored":7658325,"data_compressed":186355,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004673"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703303+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:34:50.703303+0000","last_peered":"2024-07-29T00:34:50.703303+0000","last_clean":"2024-07-29T00:34:50.703303+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:34:50.703303+0000","last_undegraded":"2024-07-29T00:34:50.703303+0000","last_fullsized":"2024-07-29T00:34:50.703303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719061+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:34:50.719061+0000","last_peered":"2024-07-29T00:34:50.719061+0000","last_clean":"2024-07-29T00:34:50.719061+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:34:50.719061+0000","last_undegraded":"2024-07-29T00:34:50.719061+0000","last_fullsized":"2024-07-29T00:34:50.719061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.717894+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:34:50.717894+0000","last_peered":"2024-07-29T00:34:50.717894+0000","last_clean":"2024-07-29T00:34:50.717894+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:34:50.717894+0000","last_undegraded":"2024-07-29T00:34:50.717894+0000","last_fullsized":"2024-07-29T00:34:50.717894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704803+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:34:50.704803+0000","last_peered":"2024-07-29T00:34:50.704803+0000","last_clean":"2024-07-29T00:34:50.704803+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:34:50.704803+0000","last_undegraded":"2024-07-29T00:34:50.704803+0000","last_fullsized":"2024-07-29T00:34:50.704803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705530+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:34:50.705530+0000","last_peered":"2024-07-29T00:34:50.705530+0000","last_clean":"2024-07-29T00:34:50.705530+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:34:50.705530+0000","last_undegraded":"2024-07-29T00:34:50.705530+0000","last_fullsized":"2024-07-29T00:34:50.705530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700671+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:34:50.700671+0000","last_peered":"2024-07-29T00:34:50.700671+0000","last_clean":"2024-07-29T00:34:50.700671+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:34:50.700671+0000","last_undegraded":"2024-07-29T00:34:50.700671+0000","last_fullsized":"2024-07-29T00:34:50.700671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.713666+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:34:50.713666+0000","last_peered":"2024-07-29T00:34:50.713666+0000","last_clean":"2024-07-29T00:34:50.713666+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:34:50.713666+0000","last_undegraded":"2024-07-29T00:34:50.713666+0000","last_fullsized":"2024-07-29T00:34:50.713666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.707186+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:34:50.707186+0000","last_peered":"2024-07-29T00:34:50.707186+0000","last_clean":"2024-07-29T00:34:50.707186+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:34:50.707186+0000","last_undegraded":"2024-07-29T00:34:50.707186+0000","last_fullsized":"2024-07-29T00:34:50.707186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704617+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:34:50.704617+0000","last_peered":"2024-07-29T00:34:50.704617+0000","last_clean":"2024-07-29T00:34:50.704617+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:34:50.704617+0000","last_undegraded":"2024-07-29T00:34:50.704617+0000","last_fullsized":"2024-07-29T00:34:50.704617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.711612+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:34:50.711612+0000","last_peered":"2024-07-29T00:34:50.711612+0000","last_clean":"2024-07-29T00:34:50.711612+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:34:50.711612+0000","last_undegraded":"2024-07-29T00:34:50.711612+0000","last_fullsized":"2024-07-29T00:34:50.711612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705847+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:34:50.705847+0000","last_peered":"2024-07-29T00:34:50.705847+0000","last_clean":"2024-07-29T00:34:50.705847+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:34:50.705847+0000","last_undegraded":"2024-07-29T00:34:50.705847+0000","last_fullsized":"2024-07-29T00:34:50.705847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704931+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:34:50.704931+0000","last_peered":"2024-07-29T00:34:50.704931+0000","last_clean":"2024-07-29T00:34:50.704931+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:34:50.704931+0000","last_undegraded":"2024-07-29T00:34:50.704931+0000","last_fullsized":"2024-07-29T00:34:50.704931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721203+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:34:50.721203+0000","last_peered":"2024-07-29T00:34:50.721203+0000","last_clean":"2024-07-29T00:34:50.721203+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:34:50.721203+0000","last_undegraded":"2024-07-29T00:34:50.721203+0000","last_fullsized":"2024-07-29T00:34:50.721203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.718861+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:34:50.718861+0000","last_peered":"2024-07-29T00:34:50.718861+0000","last_clean":"2024-07-29T00:34:50.718861+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:34:50.718861+0000","last_undegraded":"2024-07-29T00:34:50.718861+0000","last_fullsized":"2024-07-29T00:34:50.718861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719821+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:34:50.719821+0000","last_peered":"2024-07-29T00:34:50.719821+0000","last_clean":"2024-07-29T00:34:50.719821+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:34:50.719821+0000","last_undegraded":"2024-07-29T00:34:50.719821+0000","last_fullsized":"2024-07-29T00:34:50.719821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703557+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:34:50.703557+0000","last_peered":"2024-07-29T00:34:50.703557+0000","last_clean":"2024-07-29T00:34:50.703557+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:34:50.703557+0000","last_undegraded":"2024-07-29T00:34:50.703557+0000","last_fullsized":"2024-07-29T00:34:50.703557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703375+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:34:50.703375+0000","last_peered":"2024-07-29T00:34:50.703375+0000","last_clean":"2024-07-29T00:34:50.703375+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:34:50.703375+0000","last_undegraded":"2024-07-29T00:34:50.703375+0000","last_fullsized":"2024-07-29T00:34:50.703375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705353+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:34:50.705353+0000","last_peered":"2024-07-29T00:34:50.705353+0000","last_clean":"2024-07-29T00:34:50.705353+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:34:50.705353+0000","last_undegraded":"2024-07-29T00:34:50.705353+0000","last_fullsized":"2024-07-29T00:34:50.705353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700645+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:34:50.700645+0000","last_peered":"2024-07-29T00:34:50.700645+0000","last_clean":"2024-07-29T00:34:50.700645+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:34:50.700645+0000","last_undegraded":"2024-07-29T00:34:50.700645+0000","last_fullsized":"2024-07-29T00:34:50.700645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702773+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:34:50.702773+0000","last_peered":"2024-07-29T00:34:50.702773+0000","last_clean":"2024-07-29T00:34:50.702773+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:34:50.702773+0000","last_undegraded":"2024-07-29T00:34:50.702773+0000","last_fullsized":"2024-07-29T00:34:50.702773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721310+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:34:50.721310+0000","last_peered":"2024-07-29T00:34:50.721310+0000","last_clean":"2024-07-29T00:34:50.721310+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:34:50.721310+0000","last_undegraded":"2024-07-29T00:34:50.721310+0000","last_fullsized":"2024-07-29T00:34:50.721310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704792+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:34:50.704792+0000","last_peered":"2024-07-29T00:34:50.704792+0000","last_clean":"2024-07-29T00:34:50.704792+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:34:50.704792+0000","last_undegraded":"2024-07-29T00:34:50.704792+0000","last_fullsized":"2024-07-29T00:34:50.704792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.707587+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:34:50.707587+0000","last_peered":"2024-07-29T00:34:50.707587+0000","last_clean":"2024-07-29T00:34:50.707587+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:34:50.707587+0000","last_undegraded":"2024-07-29T00:34:50.707587+0000","last_fullsized":"2024-07-29T00:34:50.707587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703874+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:34:50.703874+0000","last_peered":"2024-07-29T00:34:50.703874+0000","last_clean":"2024-07-29T00:34:50.703874+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:34:50.703874+0000","last_undegraded":"2024-07-29T00:34:50.703874+0000","last_fullsized":"2024-07-29T00:34:50.703874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.717173+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:34:50.717173+0000","last_peered":"2024-07-29T00:34:50.717173+0000","last_clean":"2024-07-29T00:34:50.717173+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:34:50.717173+0000","last_undegraded":"2024-07-29T00:34:50.717173+0000","last_fullsized":"2024-07-29T00:34:50.717173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.701012+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:34:50.701012+0000","last_peered":"2024-07-29T00:34:50.701012+0000","last_clean":"2024-07-29T00:34:50.701012+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:34:50.701012+0000","last_undegraded":"2024-07-29T00:34:50.701012+0000","last_fullsized":"2024-07-29T00:34:50.701012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703571+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:34:50.703571+0000","last_peered":"2024-07-29T00:34:50.703571+0000","last_clean":"2024-07-29T00:34:50.703571+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:34:50.703571+0000","last_undegraded":"2024-07-29T00:34:50.703571+0000","last_fullsized":"2024-07-29T00:34:50.703571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721769+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:34:50.721769+0000","last_peered":"2024-07-29T00:34:50.721769+0000","last_clean":"2024-07-29T00:34:50.721769+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:34:50.721769+0000","last_undegraded":"2024-07-29T00:34:50.721769+0000","last_fullsized":"2024-07-29T00:34:50.721769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.718554+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:34:50.718554+0000","last_peered":"2024-07-29T00:34:50.718554+0000","last_clean":"2024-07-29T00:34:50.718554+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:34:50.718554+0000","last_undegraded":"2024-07-29T00:34:50.718554+0000","last_fullsized":"2024-07-29T00:34:50.718554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700545+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:34:50.700545+0000","last_peered":"2024-07-29T00:34:50.700545+0000","last_clean":"2024-07-29T00:34:50.700545+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:34:50.700545+0000","last_undegraded":"2024-07-29T00:34:50.700545+0000","last_fullsized":"2024-07-29T00:34:50.700545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700790+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:34:50.700790+0000","last_peered":"2024-07-29T00:34:50.700790+0000","last_clean":"2024-07-29T00:34:50.700790+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:34:50.700790+0000","last_undegraded":"2024-07-29T00:34:50.700790+0000","last_fullsized":"2024-07-29T00:34:50.700790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702858+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:34:50.702858+0000","last_peered":"2024-07-29T00:34:50.702858+0000","last_clean":"2024-07-29T00:34:50.702858+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:34:50.702858+0000","last_undegraded":"2024-07-29T00:34:50.702858+0000","last_fullsized":"2024-07-29T00:34:50.702858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705851+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:34:50.705851+0000","last_peered":"2024-07-29T00:34:50.705851+0000","last_clean":"2024-07-29T00:34:50.705851+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:34:50.705851+0000","last_undegraded":"2024-07-29T00:34:50.705851+0000","last_fullsized":"2024-07-29T00:34:50.705851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":103,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:21.790842+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:35:21.790842+0000","last_peered":"2024-07-29T00:35:21.790842+0000","last_clean":"2024-07-29T00:35:21.790842+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:35:21.790842+0000","last_undegraded":"2024-07-29T00:35:21.790842+0000","last_fullsized":"2024-07-29T00:35:21.790842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703969+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:34:50.703969+0000","last_peered":"2024-07-29T00:34:50.703969+0000","last_clean":"2024-07-29T00:34:50.703969+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:34:50.703969+0000","last_undegraded":"2024-07-29T00:34:50.703969+0000","last_fullsized":"2024-07-29T00:34:50.703969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700826+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:34:50.700826+0000","last_peered":"2024-07-29T00:34:50.700826+0000","last_clean":"2024-07-29T00:34:50.700826+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:34:50.700826+0000","last_undegraded":"2024-07-29T00:34:50.700826+0000","last_fullsized":"2024-07-29T00:34:50.700826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720638+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:34:50.720638+0000","last_peered":"2024-07-29T00:34:50.720638+0000","last_clean":"2024-07-29T00:34:50.720638+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:34:50.720638+0000","last_undegraded":"2024-07-29T00:34:50.720638+0000","last_fullsized":"2024-07-29T00:34:50.720638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.707380+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:34:50.707380+0000","last_peered":"2024-07-29T00:34:50.707380+0000","last_clean":"2024-07-29T00:34:50.707380+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:34:50.707380+0000","last_undegraded":"2024-07-29T00:34:50.707380+0000","last_fullsized":"2024-07-29T00:34:50.707380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704260+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:34:50.704260+0000","last_peered":"2024-07-29T00:34:50.704260+0000","last_clean":"2024-07-29T00:34:50.704260+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:34:50.704260+0000","last_undegraded":"2024-07-29T00:34:50.704260+0000","last_fullsized":"2024-07-29T00:34:50.704260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702586+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:34:50.702586+0000","last_peered":"2024-07-29T00:34:50.702586+0000","last_clean":"2024-07-29T00:34:50.702586+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:34:50.702586+0000","last_undegraded":"2024-07-29T00:34:50.702586+0000","last_fullsized":"2024-07-29T00:34:50.702586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:19.756535+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:35:19.756535+0000","last_peered":"2024-07-29T00:35:19.756535+0000","last_clean":"2024-07-29T00:35:19.756535+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:35:19.756535+0000","last_undegraded":"2024-07-29T00:35:19.756535+0000","last_fullsized":"2024-07-29T00:35:19.756535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704354+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:34:50.704354+0000","last_peered":"2024-07-29T00:34:50.704354+0000","last_clean":"2024-07-29T00:34:50.704354+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:34:50.704354+0000","last_undegraded":"2024-07-29T00:34:50.704354+0000","last_fullsized":"2024-07-29T00:34:50.704354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704906+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:34:50.704906+0000","last_peered":"2024-07-29T00:34:50.704906+0000","last_clean":"2024-07-29T00:34:50.704906+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:34:50.704906+0000","last_undegraded":"2024-07-29T00:34:50.704906+0000","last_fullsized":"2024-07-29T00:34:50.704906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720205+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:34:50.720205+0000","last_peered":"2024-07-29T00:34:50.720205+0000","last_clean":"2024-07-29T00:34:50.720205+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:34:50.720205+0000","last_undegraded":"2024-07-29T00:34:50.720205+0000","last_fullsized":"2024-07-29T00:34:50.720205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704640+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:34:50.704640+0000","last_peered":"2024-07-29T00:34:50.704640+0000","last_clean":"2024-07-29T00:34:50.704640+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:34:50.704640+0000","last_undegraded":"2024-07-29T00:34:50.704640+0000","last_fullsized":"2024-07-29T00:34:50.704640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703890+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:34:50.703890+0000","last_peered":"2024-07-29T00:34:50.703890+0000","last_clean":"2024-07-29T00:34:50.703890+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:34:50.703890+0000","last_undegraded":"2024-07-29T00:34:50.703890+0000","last_fullsized":"2024-07-29T00:34:50.703890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":218,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:19.754750+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:35:19.754750+0000","last_peered":"2024-07-29T00:35:19.754750+0000","last_clean":"2024-07-29T00:35:19.754750+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:35:19.754750+0000","last_undegraded":"2024-07-29T00:35:19.754750+0000","last_fullsized":"2024-07-29T00:35:19.754750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59761,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":141,"num_read_kb":128,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706278+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:34:50.706278+0000","last_peered":"2024-07-29T00:34:50.706278+0000","last_clean":"2024-07-29T00:34:50.706278+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:34:50.706278+0000","last_undegraded":"2024-07-29T00:34:50.706278+0000","last_fullsized":"2024-07-29T00:34:50.706278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700884+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:34:50.700884+0000","last_peered":"2024-07-29T00:34:50.700884+0000","last_clean":"2024-07-29T00:34:50.700884+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:34:50.700884+0000","last_undegraded":"2024-07-29T00:34:50.700884+0000","last_fullsized":"2024-07-29T00:34:50.700884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721188+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:34:50.721188+0000","last_peered":"2024-07-29T00:34:50.721188+0000","last_clean":"2024-07-29T00:34:50.721188+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:34:50.721188+0000","last_undegraded":"2024-07-29T00:34:50.721188+0000","last_fullsized":"2024-07-29T00:34:50.721188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703792+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:34:50.703792+0000","last_peered":"2024-07-29T00:34:50.703792+0000","last_clean":"2024-07-29T00:34:50.703792+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:34:50.703792+0000","last_undegraded":"2024-07-29T00:34:50.703792+0000","last_fullsized":"2024-07-29T00:34:50.703792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704862+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:34:50.704862+0000","last_peered":"2024-07-29T00:34:50.704862+0000","last_clean":"2024-07-29T00:34:50.704862+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:34:50.704862+0000","last_undegraded":"2024-07-29T00:34:50.704862+0000","last_fullsized":"2024-07-29T00:34:50.704862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":104,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:20.847046+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:35:20.847046+0000","last_peered":"2024-07-29T00:35:20.847046+0000","last_clean":"2024-07-29T00:35:20.847046+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:35:20.847046+0000","last_undegraded":"2024-07-29T00:35:20.847046+0000","last_fullsized":"2024-07-29T00:35:20.847046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705397+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:34:50.705397+0000","last_peered":"2024-07-29T00:34:50.705397+0000","last_clean":"2024-07-29T00:34:50.705397+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:34:50.705397+0000","last_undegraded":"2024-07-29T00:34:50.705397+0000","last_fullsized":"2024-07-29T00:34:50.705397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706440+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:34:50.706440+0000","last_peered":"2024-07-29T00:34:50.706440+0000","last_clean":"2024-07-29T00:34:50.706440+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:34:50.706440+0000","last_undegraded":"2024-07-29T00:34:50.706440+0000","last_fullsized":"2024-07-29T00:34:50.706440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703734+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:34:50.703734+0000","last_peered":"2024-07-29T00:34:50.703734+0000","last_clean":"2024-07-29T00:34:50.703734+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:34:50.703734+0000","last_undegraded":"2024-07-29T00:34:50.703734+0000","last_fullsized":"2024-07-29T00:34:50.703734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706411+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:34:50.706411+0000","last_peered":"2024-07-29T00:34:50.706411+0000","last_clean":"2024-07-29T00:34:50.706411+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:34:50.706411+0000","last_undegraded":"2024-07-29T00:34:50.706411+0000","last_fullsized":"2024-07-29T00:34:50.706411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.712384+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:34:50.712384+0000","last_peered":"2024-07-29T00:34:50.712384+0000","last_clean":"2024-07-29T00:34:50.712384+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:34:50.712384+0000","last_undegraded":"2024-07-29T00:34:50.712384+0000","last_fullsized":"2024-07-29T00:34:50.712384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719859+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:34:50.719859+0000","last_peered":"2024-07-29T00:34:50.719859+0000","last_clean":"2024-07-29T00:34:50.719859+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:34:50.719859+0000","last_undegraded":"2024-07-29T00:34:50.719859+0000","last_fullsized":"2024-07-29T00:34:50.719859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705135+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:34:50.705135+0000","last_peered":"2024-07-29T00:34:50.705135+0000","last_clean":"2024-07-29T00:34:50.705135+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:34:50.705135+0000","last_undegraded":"2024-07-29T00:34:50.705135+0000","last_fullsized":"2024-07-29T00:34:50.705135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704649+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:34:50.704649+0000","last_peered":"2024-07-29T00:34:50.704649+0000","last_clean":"2024-07-29T00:34:50.704649+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:34:50.704649+0000","last_undegraded":"2024-07-29T00:34:50.704649+0000","last_fullsized":"2024-07-29T00:34:50.704649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704753+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:34:50.704753+0000","last_peered":"2024-07-29T00:34:50.704753+0000","last_clean":"2024-07-29T00:34:50.704753+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:34:50.704753+0000","last_undegraded":"2024-07-29T00:34:50.704753+0000","last_fullsized":"2024-07-29T00:34:50.704753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:20.758035+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:35:20.758035+0000","last_peered":"2024-07-29T00:35:20.758035+0000","last_clean":"2024-07-29T00:35:20.758035+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:35:20.758035+0000","last_undegraded":"2024-07-29T00:35:20.758035+0000","last_fullsized":"2024-07-29T00:35:20.758035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705922+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:34:50.705922+0000","last_peered":"2024-07-29T00:34:50.705922+0000","last_clean":"2024-07-29T00:34:50.705922+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:34:50.705922+0000","last_undegraded":"2024-07-29T00:34:50.705922+0000","last_fullsized":"2024-07-29T00:34:50.705922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705711+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:34:50.705711+0000","last_peered":"2024-07-29T00:34:50.705711+0000","last_clean":"2024-07-29T00:34:50.705711+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:34:50.705711+0000","last_undegraded":"2024-07-29T00:34:50.705711+0000","last_fullsized":"2024-07-29T00:34:50.705711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704769+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:34:50.704769+0000","last_peered":"2024-07-29T00:34:50.704769+0000","last_clean":"2024-07-29T00:34:50.704769+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:34:50.704769+0000","last_undegraded":"2024-07-29T00:34:50.704769+0000","last_fullsized":"2024-07-29T00:34:50.704769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.708949+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:34:50.708949+0000","last_peered":"2024-07-29T00:34:50.708949+0000","last_clean":"2024-07-29T00:34:50.708949+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:34:50.708949+0000","last_undegraded":"2024-07-29T00:34:50.708949+0000","last_fullsized":"2024-07-29T00:34:50.708949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705563+0000","last_change":"2024-07-29T00:33:53.680861+0000","last_active":"2024-07-29T00:34:50.705563+0000","last_peered":"2024-07-29T00:34:50.705563+0000","last_clean":"2024-07-29T00:34:50.705563+0000","last_became_active":"2024-07-29T00:33:53.680260+0000","last_became_peered":"2024-07-29T00:33:53.680260+0000","last_unstale":"2024-07-29T00:34:50.705563+0000","last_undegraded":"2024-07-29T00:34:50.705563+0000","last_fullsized":"2024-07-29T00:34:50.705563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:50:13.931513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720501+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:34:50.720501+0000","last_peered":"2024-07-29T00:34:50.720501+0000","last_clean":"2024-07-29T00:34:50.720501+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:34:50.720501+0000","last_undegraded":"2024-07-29T00:34:50.720501+0000","last_fullsized":"2024-07-29T00:34:50.720501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:17.839448+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:35:17.839448+0000","last_peered":"2024-07-29T00:35:17.839448+0000","last_clean":"2024-07-29T00:35:17.839448+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:35:17.839448+0000","last_undegraded":"2024-07-29T00:35:17.839448+0000","last_fullsized":"2024-07-29T00:35:17.839448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.722534+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:34:50.722534+0000","last_peered":"2024-07-29T00:34:50.722534+0000","last_clean":"2024-07-29T00:34:50.722534+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:34:50.722534+0000","last_undegraded":"2024-07-29T00:34:50.722534+0000","last_fullsized":"2024-07-29T00:34:50.722534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:20.836742+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:35:20.836742+0000","last_peered":"2024-07-29T00:35:20.836742+0000","last_clean":"2024-07-29T00:35:20.836742+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:35:20.836742+0000","last_undegraded":"2024-07-29T00:35:20.836742+0000","last_fullsized":"2024-07-29T00:35:20.836742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700858+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:34:50.700858+0000","last_peered":"2024-07-29T00:34:50.700858+0000","last_clean":"2024-07-29T00:34:50.700858+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:34:50.700858+0000","last_undegraded":"2024-07-29T00:34:50.700858+0000","last_fullsized":"2024-07-29T00:34:50.700858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704182+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:34:50.704182+0000","last_peered":"2024-07-29T00:34:50.704182+0000","last_clean":"2024-07-29T00:34:50.704182+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:34:50.704182+0000","last_undegraded":"2024-07-29T00:34:50.704182+0000","last_fullsized":"2024-07-29T00:34:50.704182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.701026+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:34:50.701026+0000","last_peered":"2024-07-29T00:34:50.701026+0000","last_clean":"2024-07-29T00:34:50.701026+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:34:50.701026+0000","last_undegraded":"2024-07-29T00:34:50.701026+0000","last_fullsized":"2024-07-29T00:34:50.701026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706398+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:34:50.706398+0000","last_peered":"2024-07-29T00:34:50.706398+0000","last_clean":"2024-07-29T00:34:50.706398+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:34:50.706398+0000","last_undegraded":"2024-07-29T00:34:50.706398+0000","last_fullsized":"2024-07-29T00:34:50.706398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.721872+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:34:50.721872+0000","last_peered":"2024-07-29T00:34:50.721872+0000","last_clean":"2024-07-29T00:34:50.721872+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:34:50.721872+0000","last_undegraded":"2024-07-29T00:34:50.721872+0000","last_fullsized":"2024-07-29T00:34:50.721872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705416+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:34:50.705416+0000","last_peered":"2024-07-29T00:34:50.705416+0000","last_clean":"2024-07-29T00:34:50.705416+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:34:50.705416+0000","last_undegraded":"2024-07-29T00:34:50.705416+0000","last_fullsized":"2024-07-29T00:34:50.705416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.710003+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:34:50.710003+0000","last_peered":"2024-07-29T00:34:50.710003+0000","last_clean":"2024-07-29T00:34:50.710003+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:34:50.710003+0000","last_undegraded":"2024-07-29T00:34:50.710003+0000","last_fullsized":"2024-07-29T00:34:50.710003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704759+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:34:50.704759+0000","last_peered":"2024-07-29T00:34:50.704759+0000","last_clean":"2024-07-29T00:34:50.704759+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:34:50.704759+0000","last_undegraded":"2024-07-29T00:34:50.704759+0000","last_fullsized":"2024-07-29T00:34:50.704759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.719233+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:34:50.719233+0000","last_peered":"2024-07-29T00:34:50.719233+0000","last_clean":"2024-07-29T00:34:50.719233+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:34:50.719233+0000","last_undegraded":"2024-07-29T00:34:50.719233+0000","last_fullsized":"2024-07-29T00:34:50.719233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.703842+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:34:50.703842+0000","last_peered":"2024-07-29T00:34:50.703842+0000","last_clean":"2024-07-29T00:34:50.703842+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:34:50.703842+0000","last_undegraded":"2024-07-29T00:34:50.703842+0000","last_fullsized":"2024-07-29T00:34:50.703842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705626+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:34:50.705626+0000","last_peered":"2024-07-29T00:34:50.705626+0000","last_clean":"2024-07-29T00:34:50.705626+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:34:50.705626+0000","last_undegraded":"2024-07-29T00:34:50.705626+0000","last_fullsized":"2024-07-29T00:34:50.705626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.708382+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:34:50.708382+0000","last_peered":"2024-07-29T00:34:50.708382+0000","last_clean":"2024-07-29T00:34:50.708382+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:34:50.708382+0000","last_undegraded":"2024-07-29T00:34:50.708382+0000","last_fullsized":"2024-07-29T00:34:50.708382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.702968+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:34:50.702968+0000","last_peered":"2024-07-29T00:34:50.702968+0000","last_clean":"2024-07-29T00:34:50.702968+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:34:50.702968+0000","last_undegraded":"2024-07-29T00:34:50.702968+0000","last_fullsized":"2024-07-29T00:34:50.702968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.700557+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:34:50.700557+0000","last_peered":"2024-07-29T00:34:50.700557+0000","last_clean":"2024-07-29T00:34:50.700557+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:34:50.700557+0000","last_undegraded":"2024-07-29T00:34:50.700557+0000","last_fullsized":"2024-07-29T00:34:50.700557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.709356+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:34:50.709356+0000","last_peered":"2024-07-29T00:34:50.709356+0000","last_clean":"2024-07-29T00:34:50.709356+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:34:50.709356+0000","last_undegraded":"2024-07-29T00:34:50.709356+0000","last_fullsized":"2024-07-29T00:34:50.709356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705687+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:34:50.705687+0000","last_peered":"2024-07-29T00:34:50.705687+0000","last_clean":"2024-07-29T00:34:50.705687+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:34:50.705687+0000","last_undegraded":"2024-07-29T00:34:50.705687+0000","last_fullsized":"2024-07-29T00:34:50.705687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705851+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:34:50.705851+0000","last_peered":"2024-07-29T00:34:50.705851+0000","last_clean":"2024-07-29T00:34:50.705851+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:34:50.705851+0000","last_undegraded":"2024-07-29T00:34:50.705851+0000","last_fullsized":"2024-07-29T00:34:50.705851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.704533+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:34:50.704533+0000","last_peered":"2024-07-29T00:34:50.704533+0000","last_clean":"2024-07-29T00:34:50.704533+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:34:50.704533+0000","last_undegraded":"2024-07-29T00:34:50.704533+0000","last_fullsized":"2024-07-29T00:34:50.704533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.706325+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:34:50.706325+0000","last_peered":"2024-07-29T00:34:50.706325+0000","last_clean":"2024-07-29T00:34:50.706325+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:34:50.706325+0000","last_undegraded":"2024-07-29T00:34:50.706325+0000","last_fullsized":"2024-07-29T00:34:50.706325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.705213+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:34:50.705213+0000","last_peered":"2024-07-29T00:34:50.705213+0000","last_clean":"2024-07-29T00:34:50.705213+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:34:50.705213+0000","last_undegraded":"2024-07-29T00:34:50.705213+0000","last_fullsized":"2024-07-29T00:34:50.705213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720521+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:34:50.720521+0000","last_peered":"2024-07-29T00:34:50.720521+0000","last_clean":"2024-07-29T00:34:50.720521+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:34:50.720521+0000","last_undegraded":"2024-07-29T00:34:50.720521+0000","last_fullsized":"2024-07-29T00:34:50.720521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.720056+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:34:50.720056+0000","last_peered":"2024-07-29T00:34:50.720056+0000","last_clean":"2024-07-29T00:34:50.720056+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:34:50.720056+0000","last_undegraded":"2024-07-29T00:34:50.720056+0000","last_fullsized":"2024-07-29T00:34:50.720056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.711057+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:34:50.711057+0000","last_peered":"2024-07-29T00:34:50.711057+0000","last_clean":"2024-07-29T00:34:50.711057+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:34:50.711057+0000","last_undegraded":"2024-07-29T00:34:50.711057+0000","last_fullsized":"2024-07-29T00:34:50.711057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:34:50.716169+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:34:50.716169+0000","last_peered":"2024-07-29T00:34:50.716169+0000","last_clean":"2024-07-29T00:34:50.716169+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:34:50.716169+0000","last_undegraded":"2024-07-29T00:34:50.716169+0000","last_fullsized":"2024-07-29T00:34:50.716169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":109,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-29T00:35:20.835809+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:35:20.835809+0000","last_peered":"2024-07-29T00:35:20.835809+0000","last_clean":"2024-07-29T00:35:20.835809+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:35:20.835809+0000","last_undegraded":"2024-07-29T00:35:20.835809+0000","last_fullsized":"2024-07-29T00:35:20.835809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66573,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":676,"num_read_kb":423,"num_write":245,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":199719,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247654,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":475092,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1126272,"data_compressed":41617,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":475022,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":475400,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1067917,"data_compressed":41617,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":474739,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1067205,"data_compressed":41617,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":535921,"data_compressed":15352,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":474564,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":474702,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":535413,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":476078,"data_compressed":5769,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34207,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":62021,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:35:25.748 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-29T00:35:25.748 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-29T00:35:25.748 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-29T00:35:25.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph health --format=json 2024-07-29T00:35:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:25 smithi181 ceph-mon[36706]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:35:26.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:25 smithi175 ceph-mon[32138]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:35:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:25 smithi204 ceph-mon[37368]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:35:26.254 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:26.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 -- 172.21.15.175:0/4010170468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 msgr2=0x7ffa0c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 --2- 172.21.15.175:0/4010170468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 0x7ffa0c107a80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009930 tx=0x7ff9fc02a440 comp rx=0 tx=0).stop 2024-07-29T00:35:26.961 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:26 smithi175 ceph-mon[32138]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 -- 172.21.15.175:0/4010170468 shutdown_connections 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 --2- 172.21.15.175:0/4010170468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 0x7ffa0c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 --2- 172.21.15.175:0/4010170468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 --2- 172.21.15.175:0/4010170468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 0x7ffa0c107a80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 -- 172.21.15.175:0/4010170468 >> 172.21.15.175:0/4010170468 conn(0x7ffa0c0fd3a0 msgr2=0x7ffa0c0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 -- 172.21.15.175:0/4010170468 shutdown_connections 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.960+0000 7ffa0b200640 1 -- 172.21.15.175:0/4010170468 wait complete. 2024-07-29T00:35:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.961+0000 7ffa0b200640 1 Processor -- start 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.961+0000 7ffa0b200640 1 -- start start 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 0x7ffa0c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 0x7ffa0c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c111af0 con 0x7ffa0c1016a0 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c111970 con 0x7ffa0c102040 2024-07-29T00:35:26.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c111c70 con 0x7ffa0c1076a0 2024-07-29T00:35:26.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa09e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:26.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa09e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51174/0 (socket says 172.21.15.175:51174) 2024-07-29T00:35:26.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa09e00640 1 -- 172.21.15.175:0/2568535092 learned_addr learned my addr 172.21.15.175:0/2568535092 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa09400640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 0x7ffa0c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.962+0000 7ffa0a800640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 0x7ffa0c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09e00640 1 -- 172.21.15.175:0/2568535092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 msgr2=0x7ffa0c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09e00640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 0x7ffa0c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09e00640 1 -- 172.21.15.175:0/2568535092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 msgr2=0x7ffa0c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09e00640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 0x7ffa0c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09e00640 1 -- 172.21.15.175:0/2568535092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c19f180 con 0x7ffa0c1016a0 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09400640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 0x7ffa0c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa09e00640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c19a1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc02a960 tx=0x7ff9fc003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc03f070 con 0x7ffa0c1016a0 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.963+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c19f410 con 0x7ffa0c1016a0 2024-07-29T00:35:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.964+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c1a6d50 con 0x7ffa0c1016a0 2024-07-29T00:35:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.964+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc031c90 con 0x7ffa0c1016a0 2024-07-29T00:35:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.964+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc035030 con 0x7ffa0c1016a0 2024-07-29T00:35:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.965+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c101b00 con 0x7ffa0c1016a0 2024-07-29T00:35:26.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.968+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff9fc042070 con 0x7ffa0c1016a0 2024-07-29T00:35:26.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.969+0000 7ffa02a00640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff9f00a2c90 0x7ff9f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:26.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.970+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff9fc0efbc0 con 0x7ffa0c1016a0 2024-07-29T00:35:26.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.970+0000 7ffa09400640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff9f00a2c90 0x7ff9f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:26.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.971+0000 7ffa09400640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff9f00a2c90 0x7ff9f00a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c10e710 tx=0x7ff9f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:26.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:26.972+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff9fc0bc2f0 con 0x7ffa0c1016a0 2024-07-29T00:35:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:26 smithi181 ceph-mon[36706]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:26 smithi204 ceph-mon[37368]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:27.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.201+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffa0c1a6f20 con 0x7ffa0c1016a0 2024-07-29T00:35:27.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.202+0000 7ffa02a00640 1 -- 172.21.15.175:0/2568535092 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff9fc0c05f0 con 0x7ffa0c1016a0 2024-07-29T00:35:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:27.202 INFO:teuthology.orchestra.run.smithi175.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-29T00:35:27.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.204+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff9f00a2c90 msgr2=0x7ff9f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:27.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.204+0000 7ffa0b200640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff9f00a2c90 0x7ff9f00a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c10e710 tx=0x7ff9f8005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:27.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.204+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 msgr2=0x7ffa0c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:27.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.204+0000 7ffa0b200640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c19a1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc02a960 tx=0x7ff9fc003b20 comp rx=0 tx=0).stop 2024-07-29T00:35:27.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 shutdown_connections 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff9f00a2c90 0x7ff9f00a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa0c1076a0 0x7ffa0c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c102040 0x7ffa0c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 --2- 172.21.15.175:0/2568535092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0c1016a0 0x7ffa0c19a1d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 >> 172.21.15.175:0/2568535092 conn(0x7ffa0c0fd3a0 msgr2=0x7ffa0c10cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 shutdown_connections 2024-07-29T00:35:27.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:27.205+0000 7ffa0b200640 1 -- 172.21.15.175:0/2568535092 wait complete. 2024-07-29T00:35:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:27 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2568535092' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-29T00:35:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:27 smithi181 ceph-mon[36706]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:35:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:27 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2568535092' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-29T00:35:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:27 smithi175 ceph-mon[32138]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:35:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:27 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2568535092' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-29T00:35:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:27 smithi204 ceph-mon[37368]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:35:28.115 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-29T00:35:28.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:35:28.563 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:29.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 -- 172.21.15.175:0/274948058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217410d220 msgr2=0x7f2174114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 --2- 172.21.15.175:0/274948058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217410d220 0x7f2174114da0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f216c009900 tx=0x7f216c02a410 comp rx=0 tx=0).stop 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 -- 172.21.15.175:0/274948058 shutdown_connections 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 --2- 172.21.15.175:0/274948058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217410d220 0x7f2174114da0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 --2- 172.21.15.175:0/274948058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2174108af0 0x7f217410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 --2- 172.21.15.175:0/274948058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108140 0x7f2174108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 -- 172.21.15.175:0/274948058 >> 172.21.15.175:0/274948058 conn(0x7f2174077890 msgr2=0x7f2174079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 -- 172.21.15.175:0/274948058 shutdown_connections 2024-07-29T00:35:29.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.254+0000 7f2179000640 1 -- 172.21.15.175:0/274948058 wait complete. 2024-07-29T00:35:29.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.255+0000 7f2179000640 1 Processor -- start 2024-07-29T00:35:29.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.255+0000 7f2179000640 1 -- start start 2024-07-29T00:35:29.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2179000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2174108140 0x7f217419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:29.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2179000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108af0 0x7f217419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2179000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f217410d220 0x7f21741a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2179000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21741182a0 con 0x7f2174108140 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2179000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2174118120 con 0x7f217410d220 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2179000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2174118420 con 0x7f2174108af0 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2171e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2174108140 0x7f217419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2169400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108af0 0x7f217419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2171e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2174108140 0x7f217419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:52600/0 (socket says 172.21.15.175:52600) 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2171e00640 1 -- 172.21.15.175:0/2521064358 learned_addr learned my addr 172.21.15.175:0/2521064358 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2172800640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f217410d220 0x7f21741a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2169400640 1 -- 172.21.15.175:0/2521064358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f217410d220 msgr2=0x7f21741a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:29.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2169400640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f217410d220 0x7f21741a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2169400640 1 -- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2174108140 msgr2=0x7f217419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2169400640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2174108140 0x7f217419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.256+0000 7f2169400640 1 -- 172.21.15.175:0/2521064358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21741a36b0 con 0x7f2174108af0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.257+0000 7f2169400640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108af0 0x7f217419ec40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f215c005fd0 tx=0x7f215c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.257+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c009c30 con 0x7f2174108af0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.257+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215c009dd0 con 0x7f2174108af0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.257+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c010460 con 0x7f2174108af0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.257+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21741a39a0 con 0x7f2174108af0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.257+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21741ab280 con 0x7f2174108af0 2024-07-29T00:35:29.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.259+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2174108520 con 0x7f2174108af0 2024-07-29T00:35:29.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.260+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f215c003ef0 con 0x7f2174108af0 2024-07-29T00:35:29.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.262+0000 7f216b400640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f21580a2c10 0x7f21580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:29.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.263+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f215c0cece0 con 0x7f2174108af0 2024-07-29T00:35:29.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.263+0000 7f2171e00640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f21580a2c10 0x7f21580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:29.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.263+0000 7f2171e00640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f21580a2c10 0x7f21580a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21640049d0 tx=0x7f2164009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:29.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.266+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f215c017020 con 0x7f2174108af0 2024-07-29T00:35:29.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.493+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21741ab4e0 con 0x7f2174108af0 2024-07-29T00:35:29.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.494+0000 7f216b400640 1 -- 172.21.15.175:0/2521064358 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f215c09b410 con 0x7f2174108af0 2024-07-29T00:35:29.495 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:29.496 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":28,"btime":"2024-07-29T00:34:15:679894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:34:14.951643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:35:29.496 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:35:29.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f21580a2c10 msgr2=0x7f21580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:29.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f21580a2c10 0x7f21580a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21640049d0 tx=0x7f2164009500 comp rx=0 tx=0).stop 2024-07-29T00:35:29.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108af0 msgr2=0x7f217419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108af0 0x7f217419ec40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f215c005fd0 tx=0x7f215c00dde0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 shutdown_connections 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f217410d220 0x7f21741a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f21580a2c10 0x7f21580a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2174108af0 0x7f217419ec40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 --2- 172.21.15.175:0/2521064358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2174108140 0x7f217419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.498+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 >> 172.21.15.175:0/2521064358 conn(0x7f2174077890 msgr2=0x7f2174078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.499+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 shutdown_connections 2024-07-29T00:35:29.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:29.499+0000 7f2179000640 1 -- 172.21.15.175:0/2521064358 wait complete. 2024-07-29T00:35:29.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:29 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2521064358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:29.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:29 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2521064358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:29 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2521064358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:30.400 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:35:30.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:30 smithi175 ceph-mon[32138]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:30 smithi204 ceph-mon[37368]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:30.869 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:30 smithi181 ceph-mon[36706]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:35:31.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 -- 172.21.15.175:0/3631337447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f810dd50 msgr2=0x7f81f810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 --2- 172.21.15.175:0/3631337447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f810dd50 0x7f81f810e130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009860 tx=0x7f81e802a340 comp rx=0 tx=0).stop 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 -- 172.21.15.175:0/3631337447 shutdown_connections 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 --2- 172.21.15.175:0/3631337447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81f8108670 0x7f81f8114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 --2- 172.21.15.175:0/3631337447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f8107d10 0x7f81f8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 --2- 172.21.15.175:0/3631337447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f810dd50 0x7f81f810e130 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 -- 172.21.15.175:0/3631337447 >> 172.21.15.175:0/3631337447 conn(0x7f81f8077670 msgr2=0x7f81f8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 -- 172.21.15.175:0/3631337447 shutdown_connections 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.586+0000 7f81fce00640 1 -- 172.21.15.175:0/3631337447 wait complete. 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.587+0000 7f81fce00640 1 Processor -- start 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.587+0000 7f81fce00640 1 -- start start 2024-07-29T00:35:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.587+0000 7f81fce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 0x7f81f819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.587+0000 7f81fce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81f8108670 0x7f81f819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81fce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f810dd50 0x7f81f81a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81fce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8118070 con 0x7f81f810dd50 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81fce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8117ef0 con 0x7f81f8107d10 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81fce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81181f0 con 0x7f81f8108670 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 0x7f81f819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f810dd50 0x7f81f81a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 0x7f81f819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38314/0 (socket says 172.21.15.175:38314) 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 -- 172.21.15.175:0/2899290059 learned_addr learned my addr 172.21.15.175:0/2899290059 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f6a00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81f8108670 0x7f81f819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 -- 172.21.15.175:0/2899290059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81f8108670 msgr2=0x7f81f819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:31.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81f8108670 0x7f81f819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 -- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f810dd50 msgr2=0x7f81f81a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f810dd50 0x7f81f81a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7400640 1 -- 172.21.15.175:0/2899290059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f81a34c0 con 0x7f81f8107d10 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.588+0000 7f81f7e00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f810dd50 0x7f81f81a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.589+0000 7f81f7400640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 0x7f81f819e510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81e8035040 tx=0x7f81e8033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.589+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e803f020 con 0x7f81f8107d10 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.589+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81e8004040 con 0x7f81f8107d10 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.589+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f81a3750 con 0x7f81f8107d10 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.589+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8037040 con 0x7f81f8107d10 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.589+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f81ab090 con 0x7f81f8107d10 2024-07-29T00:35:31.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.590+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f8109400 con 0x7f81f8107d10 2024-07-29T00:35:31.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.592+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f81e8042020 con 0x7f81f8107d10 2024-07-29T00:35:31.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.593+0000 7f81efe00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f81dc0a2c10 0x7f81dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:31.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.594+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f81e80ef5b0 con 0x7f81f8107d10 2024-07-29T00:35:31.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.594+0000 7f81f6a00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f81dc0a2c10 0x7f81dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:31.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.594+0000 7f81f6a00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f81dc0a2c10 0x7f81dc0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f81f810d440 tx=0x7f81e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:31.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.596+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f81e8049020 con 0x7f81f8107d10 2024-07-29T00:35:31.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.824+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81f8108130 con 0x7f81f8107d10 2024-07-29T00:35:31.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.825+0000 7f81efe00640 1 -- 172.21.15.175:0/2899290059 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f81f8108130 con 0x7f81f8107d10 2024-07-29T00:35:31.826 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:31.826 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:35:31.827 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":28,"btime":"2024-07-29T00:34:15:679894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:34:14.951643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:35:31.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f81dc0a2c10 msgr2=0x7f81dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:31.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f81dc0a2c10 0x7f81dc0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f81f810d440 tx=0x7f81e4009260 comp rx=0 tx=0).stop 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 msgr2=0x7f81f819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 0x7f81f819e510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81e8035040 tx=0x7f81e8033b60 comp rx=0 tx=0).stop 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 shutdown_connections 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81f810dd50 0x7f81f81a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f81dc0a2c10 0x7f81dc0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81f8108670 0x7f81f819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 --2- 172.21.15.175:0/2899290059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f8107d10 0x7f81f819e510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.828+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 >> 172.21.15.175:0/2899290059 conn(0x7f81f8077670 msgr2=0x7f81f8078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:31.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.829+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 shutdown_connections 2024-07-29T00:35:31.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:31.829+0000 7f81fce00640 1 -- 172.21.15.175:0/2899290059 wait complete. 2024-07-29T00:35:32.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:35:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:32 smithi181 ceph-mon[36706]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2899290059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:32 smithi175 ceph-mon[32138]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2899290059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:32 smithi204 ceph-mon[37368]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:35:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2899290059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:33.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 -- 172.21.15.175:0/3661841584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168112560 msgr2=0x7fb168114950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:33.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 --2- 172.21.15.175:0/3661841584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168112560 0x7fb168114950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb160009ca0 tx=0x7fb16002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 -- 172.21.15.175:0/3661841584 shutdown_connections 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 --2- 172.21.15.175:0/3661841584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168112560 0x7fb168114950 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 --2- 172.21.15.175:0/3661841584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168076d30 0x7fb168112020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 --2- 172.21.15.175:0/3661841584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076410 0x7fb1680767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 -- 172.21.15.175:0/3661841584 >> 172.21.15.175:0/3661841584 conn(0x7fb1680ff5c0 msgr2=0x7fb1681019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 -- 172.21.15.175:0/3661841584 shutdown_connections 2024-07-29T00:35:33.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.875+0000 7fb16e200640 1 -- 172.21.15.175:0/3661841584 wait complete. 2024-07-29T00:35:33.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.876+0000 7fb16e200640 1 Processor -- start 2024-07-29T00:35:33.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.876+0000 7fb16e200640 1 -- start start 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.876+0000 7fb16e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168076410 0x7fb16819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.876+0000 7fb16e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076d30 0x7fb16819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.876+0000 7fb16e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168112560 0x7fb1681a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.876+0000 7fb16e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb168118100 con 0x7fb168076410 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb16e200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb168117f80 con 0x7fb168112560 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb16e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb168118280 con 0x7fb168076d30 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168076410 0x7fb16819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168076410 0x7fb16819e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:52638/0 (socket says 172.21.15.175:52638) 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb16ce00640 1 -- 172.21.15.175:0/1751929350 learned_addr learned my addr 172.21.15.175:0/1751929350 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb167e00640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076d30 0x7fb16819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb16d800640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168112560 0x7fb1681a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb167e00640 1 -- 172.21.15.175:0/1751929350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168112560 msgr2=0x7fb1681a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:33.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb167e00640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168112560 0x7fb1681a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:33.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb167e00640 1 -- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168076410 msgr2=0x7fb16819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:33.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb167e00640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168076410 0x7fb16819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:33.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.877+0000 7fb167e00640 1 -- 172.21.15.175:0/1751929350 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1681a3530 con 0x7fb168076d30 2024-07-29T00:35:33.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.878+0000 7fb167e00640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076d30 0x7fb16819eac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e9f0 tx=0x7fb15800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:33.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.878+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15800ccc0 con 0x7fb168076d30 2024-07-29T00:35:33.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.878+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1681a3820 con 0x7fb168076d30 2024-07-29T00:35:33.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.878+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb158010910 con 0x7fb168076d30 2024-07-29T00:35:33.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.878+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158018880 con 0x7fb168076d30 2024-07-29T00:35:33.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.878+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681ab100 con 0x7fb168076d30 2024-07-29T00:35:33.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.879+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1680767f0 con 0x7fb168076d30 2024-07-29T00:35:33.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.881+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb158010ac0 con 0x7fb168076d30 2024-07-29T00:35:33.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.883+0000 7fb165600640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb1500a2c90 0x7fb1500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:33.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.883+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fb1580cf4b0 con 0x7fb168076d30 2024-07-29T00:35:33.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.883+0000 7fb16ce00640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb1500a2c90 0x7fb1500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:33.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.883+0000 7fb16ce00640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb1500a2c90 0x7fb1500a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009cd0 tx=0x7fb15c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:33.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:33.887+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb15809bbe0 con 0x7fb168076d30 2024-07-29T00:35:34.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.080+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb16810d160 con 0x7fb168076d30 2024-07-29T00:35:34.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.082+0000 7fb165600640 1 -- 172.21.15.175:0/1751929350 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22789 (secure 0 0 0) 0x7fb15809fee0 con 0x7fb168076d30 2024-07-29T00:35:34.083 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:34.083 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":89,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:34:50.694810+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:35:34.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb1500a2c90 msgr2=0x7fb1500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:34.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb1500a2c90 0x7fb1500a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009cd0 tx=0x7fb15c005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:34.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076d30 msgr2=0x7fb16819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:34.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076d30 0x7fb16819eac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e9f0 tx=0x7fb15800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 shutdown_connections 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168112560 0x7fb1681a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fb1500a2c90 0x7fb1500a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb168076d30 0x7fb16819eac0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 --2- 172.21.15.175:0/1751929350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb168076410 0x7fb16819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 >> 172.21.15.175:0/1751929350 conn(0x7fb1680ff5c0 msgr2=0x7fb1681019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 shutdown_connections 2024-07-29T00:35:34.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:34.086+0000 7fb16e200640 1 -- 172.21.15.175:0/1751929350 wait complete. 2024-07-29T00:35:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:34 smithi175 ceph-mon[32138]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:35:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:34 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1751929350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:35:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:34 smithi181 ceph-mon[36706]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:35:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:34 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1751929350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:35:35.026 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:35:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:34 smithi204 ceph-mon[37368]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:35:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:34 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1751929350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:35:35.479 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:36.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 -- 172.21.15.175:0/1142656901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc10dd50 msgr2=0x7f70dc10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 --2- 172.21.15.175:0/1142656901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc10dd50 0x7f70dc10e130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009930 tx=0x7f70d002a420 comp rx=0 tx=0).stop 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 -- 172.21.15.175:0/1142656901 shutdown_connections 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 --2- 172.21.15.175:0/1142656901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc108670 0x7f70dc114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 --2- 172.21.15.175:0/1142656901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc107d10 0x7f70dc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 --2- 172.21.15.175:0/1142656901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc10dd50 0x7f70dc10e130 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 -- 172.21.15.175:0/1142656901 >> 172.21.15.175:0/1142656901 conn(0x7f70dc077670 msgr2=0x7f70dc079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 -- 172.21.15.175:0/1142656901 shutdown_connections 2024-07-29T00:35:36.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.188+0000 7f70e3000640 1 -- 172.21.15.175:0/1142656901 wait complete. 2024-07-29T00:35:36.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.189+0000 7f70e3000640 1 Processor -- start 2024-07-29T00:35:36.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.189+0000 7f70e3000640 1 -- start start 2024-07-29T00:35:36.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e3000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 0x7f70dc19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc108670 0x7f70dc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e3000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc10dd50 0x7f70dc1a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc118060 con 0x7f70dc108670 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e3000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc117ee0 con 0x7f70dc107d10 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e3000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1181e0 con 0x7f70dc10dd50 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc108670 0x7f70dc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 0x7f70dc19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 0x7f70dc19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38356/0 (socket says 172.21.15.175:38356) 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 -- 172.21.15.175:0/33530812 learned_addr learned my addr 172.21.15.175:0/33530812 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e2600640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc10dd50 0x7f70dc1a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 -- 172.21.15.175:0/33530812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc10dd50 msgr2=0x7f70dc1a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc10dd50 0x7f70dc1a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 -- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc108670 msgr2=0x7f70dc19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:36.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc108670 0x7f70dc19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1c00640 1 -- 172.21.15.175:0/33530812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc1a3510 con 0x7f70dc107d10 2024-07-29T00:35:36.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.190+0000 7f70e1200640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc108670 0x7f70dc19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.191+0000 7f70e1c00640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 0x7f70dc19e560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009230 tx=0x7f70d0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.191+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d003f030 con 0x7f70dc107d10 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.191+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d0004440 con 0x7f70dc107d10 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.191+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc1a37a0 con 0x7f70dc107d10 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.191+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0035070 con 0x7f70dc107d10 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.191+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc1ab0e0 con 0x7f70dc107d10 2024-07-29T00:35:36.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.193+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc108130 con 0x7f70dc107d10 2024-07-29T00:35:36.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.194+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f70d0042030 con 0x7f70dc107d10 2024-07-29T00:35:36.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.196+0000 7f70da000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f70c40a2c90 0x7f70c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:36.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.196+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f70d00ef490 con 0x7f70dc107d10 2024-07-29T00:35:36.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.196+0000 7f70e1200640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f70c40a2c90 0x7f70c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:36.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.197+0000 7f70e1200640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f70c40a2c90 0x7f70c40a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f70dc10ce40 tx=0x7f70cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:36.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.200+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f70d00bbbc0 con 0x7f70dc107d10 2024-07-29T00:35:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.424+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70dc1101e0 con 0x7f70dc107d10 2024-07-29T00:35:36.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.425+0000 7f70da000640 1 -- 172.21.15.175:0/33530812 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f70d00bfec0 con 0x7f70dc107d10 2024-07-29T00:35:36.426 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:35:36.427 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":28,"btime":"2024-07-29T00:34:15:679894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:34:14.951643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:35:36.427 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:35:36.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.429+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f70c40a2c90 msgr2=0x7f70c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:36.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.429+0000 7f70e3000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f70c40a2c90 0x7f70c40a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f70dc10ce40 tx=0x7f70cc005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:36.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.429+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 msgr2=0x7f70dc19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:36.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.429+0000 7f70e3000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 0x7f70dc19e560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009230 tx=0x7f70d0004040 comp rx=0 tx=0).stop 2024-07-29T00:35:36.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 shutdown_connections 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f70c40a2c90 0x7f70c40a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc10dd50 0x7f70dc1a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70dc108670 0x7f70dc19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 --2- 172.21.15.175:0/33530812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70dc107d10 0x7f70dc19e560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 >> 172.21.15.175:0/33530812 conn(0x7f70dc077670 msgr2=0x7f70dc078f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 shutdown_connections 2024-07-29T00:35:36.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:36.430+0000 7f70e3000640 1 -- 172.21.15.175:0/33530812 wait complete. 2024-07-29T00:35:36.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:36 smithi175 ceph-mon[32138]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:36.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:36 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/33530812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:36 smithi181 ceph-mon[36706]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:36.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:36 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/33530812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:36 smithi204 ceph-mon[37368]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:35:37.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:36 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/33530812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:35:37.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-29T00:33:58.041883+0000', 'modified': '2024-07-29T00:34:14.951643+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24515, 'mds_2': 24503, 'mds_3': 24499, 'mds_4': 24484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/1301583270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 1301583270}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 1301583270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/4123110499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 4123110499}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 4123110499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'j', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/284852041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 284852041}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 284852041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/568693704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 568693704}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 568693704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1977765890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1977765890}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1977765890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.204:6837/2487746199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2487746199}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2487746199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6839/2455267021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2455267021}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2455267021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.181:6837/753661139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 753661139}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 753661139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.175:6839/1522099567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1522099567}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1522099567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/4150501134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 4150501134}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 4150501134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24484, 24499, 24503, 24515, 24539]} 2024-07-29T00:35:37.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-29T00:35:37.476 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.c status 2024-07-29T00:35:37.926 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:38.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.571+0000 7f632fe00640 1 -- 172.21.15.175:0/4199387722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63300ffc30 msgr2=0x7f6330100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.571+0000 7f632fe00640 1 --2- 172.21.15.175:0/4199387722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63300ffc30 0x7f6330100090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6320009930 tx=0x7f632002a420 comp rx=0 tx=0).stop 2024-07-29T00:35:38.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 -- 172.21.15.175:0/4199387722 shutdown_connections 2024-07-29T00:35:38.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 --2- 172.21.15.175:0/4199387722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63301005d0 0x7f633010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 --2- 172.21.15.175:0/4199387722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63300ffc30 0x7f6330100090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 --2- 172.21.15.175:0/4199387722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6330107740 0x7f6330107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 -- 172.21.15.175:0/4199387722 >> 172.21.15.175:0/4199387722 conn(0x7f6330076d60 msgr2=0x7f63300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 -- 172.21.15.175:0/4199387722 shutdown_connections 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.572+0000 7f632fe00640 1 -- 172.21.15.175:0/4199387722 wait complete. 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 Processor -- start 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 -- start start 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 0x7f6330100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63301005d0 0x7f6330101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6330107740 0x7f6330101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6330111b90 con 0x7f63301005d0 2024-07-29T00:35:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6330111a10 con 0x7f6330107740 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6330111d10 con 0x7f63300ffc30 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 0x7f6330100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.573+0000 7f632e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63301005d0 0x7f6330101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 0x7f6330100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57260/0 (socket says 172.21.15.175:57260) 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 -- 172.21.15.175:0/3085634774 learned_addr learned my addr 172.21.15.175:0/3085634774 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6330107740 0x7f6330101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 -- 172.21.15.175:0/3085634774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6330107740 msgr2=0x7f6330101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6330107740 0x7f6330101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 -- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63301005d0 msgr2=0x7f6330101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63301005d0 0x7f6330101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 -- 172.21.15.175:0/3085634774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63301a64d0 con 0x7f63300ffc30 2024-07-29T00:35:38.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.574+0000 7f632ea00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 0x7f6330100e10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f631c00e9f0 tx=0x7f631c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.575+0000 7f6334a00640 1 -- 172.21.15.175:0/3085634774 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c00cd80 con 0x7f63300ffc30 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.575+0000 7f6334a00640 1 -- 172.21.15.175:0/3085634774 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631c010910 con 0x7f63300ffc30 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.575+0000 7f6334a00640 1 -- 172.21.15.175:0/3085634774 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c018990 con 0x7f63300ffc30 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.575+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63301a67c0 con 0x7f63300ffc30 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.575+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63301a6c80 con 0x7f63300ffc30 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.578+0000 7f6338a37740 1 Processor -- start 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.578+0000 7f6338a37740 1 -- start start 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.578+0000 7f6334a00640 1 -- 172.21.15.175:0/3085634774 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f631c010ac0 con 0x7f63300ffc30 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.578+0000 7f6338a37740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 0x55ccccc4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f6338a37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4eea0 0x55ccccc59220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f6338a37740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc5bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f6338a37740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccccbb4790 con 0x55ccccc4e580 2024-07-29T00:35:38.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f6338a37740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ccccb4c880 con 0x55ccccc59760 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f6338a37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ccccbb0750 con 0x55ccccc4eea0 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 0x55ccccc4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4eea0 0x55ccccc59220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc5bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 0x55ccccc4e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:52674/0 (socket says 172.21.15.175:52674) 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4eea0 0x55ccccc59220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57274/0 (socket says 172.21.15.175:57274) 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 -- 172.21.15.175:0/1355261413 learned_addr learned my addr 172.21.15.175:0/1355261413 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 -- 172.21.15.175:0/1355261413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4eea0 msgr2=0x55ccccc59220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4eea0 0x55ccccc59220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 -- 172.21.15.175:0/1355261413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 msgr2=0x55ccccc5bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc5bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.579+0000 7f632f400640 1 -- 172.21.15.175:0/1355261413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccc9262b0 con 0x55ccccc4e580 2024-07-29T00:35:38.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.580+0000 7f632f400640 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 0x55ccccc4e960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6328009fd0 tx=0x7f63280118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba97227f4673ea7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.580+0000 7f631be00640 1 -- 172.21.15.175:0/1355261413 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328004d20 con 0x55ccccc4e580 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.580+0000 7f631be00640 1 -- 172.21.15.175:0/1355261413 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6328004050 con 0x55ccccc4e580 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.580+0000 7f631be00640 1 -- 172.21.15.175:0/1355261413 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328005220 con 0x55ccccc4e580 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.580+0000 7f6334a00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63140a2c90 0x7f63140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6334a00640 1 -- 172.21.15.175:0/3085634774 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f631c0cf3c0 con 0x7f63300ffc30 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f632ea00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63140a2c90 0x7f63140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 -- 172.21.15.175:0/1355261413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 msgr2=0x55ccccc4e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 0x55ccccc4e960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6328009fd0 tx=0x7f63280118d0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 -- 172.21.15.175:0/1355261413 shutdown_connections 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc5bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4eea0 0x55ccccc59220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 --2- 172.21.15.175:0/1355261413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4e580 0x55ccccc4e960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 -- 172.21.15.175:0/1355261413 >> 172.21.15.175:0/1355261413 conn(0x55ccccc4ab10 msgr2=0x55ccccc4aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 -- 172.21.15.175:0/1355261413 shutdown_connections 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f6338a37740 1 -- 172.21.15.175:0/1355261413 wait complete. 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.581+0000 7f632ea00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63140a2c90 0x7f63140a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ccccc4b970 tx=0x7f631c0d2f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.582+0000 7f6338a37740 1 -- start start 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.582+0000 7f6338a37740 1 Processor -- start 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc5c330 con 0x55ccccc4eea0 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ccccb6f960 con 0x55ccccc59760 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc5c4b0 con 0x55ccccc4e580 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f6338a37740 10 client.0 _ll_register_callbacks cb 0x55cccc8c37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:38.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:52680/0 (socket says 172.21.15.175:52680) 2024-07-29T00:35:38.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 -- 172.21.15.175:0/2399062112 learned_addr learned my addr 172.21.15.175:0/2399062112 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:38.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632f400640 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 -- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 msgr2=0x55ccccc565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 -- 172.21.15.175:0/2399062112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 msgr2=0x55ccccc57060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.583+0000 7f632e000640 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc57060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f632e000640 1 -- 172.21.15.175:0/2399062112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccccb7ea90 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f632e000640 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56b20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6320009930 tx=0x7f63200043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577f48e76d1e3f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f6318c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f6318c00640 1 -- 172.21.15.175:0/2399062112 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320035040 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f6318c00640 1 -- 172.21.15.175:0/2399062112 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6320005450 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6328004050 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f6318c00640 1 -- 172.21.15.175:0/2399062112 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f632004c070 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.584+0000 7f6338a37740 10 client.14790 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccccc67ee0 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6318c00640 1 -- 172.21.15.175:0/2399062112 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6328004050 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6318c00640 1 -- 172.21.15.175:0/2399062112 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6320003760 con 0x55ccccc4eea0 2024-07-29T00:35:38.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 10 client.14790 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 10 client.14790 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 20 client.14790 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55ccccc6a530 0x55ccccc6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 4 client.14790 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.585+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccccbc1d00 con 0x55ccccc6a530 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.586+0000 7f632f400640 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55ccccc6a530 0x55ccccc6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.586+0000 7f632f400640 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55ccccc6a530 0x55ccccc6b9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9f8b774e7c9a8f4 server_cookie=e5fe84053924767f in_seq=0 out_seq=0 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.586+0000 7f6318c00640 10 client.14790 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6318c00640 1 -- 172.21.15.175:0/2399062112 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ccccbc1d00 con 0x55ccccc6a530 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6318c00640 10 client.14790 handle_command_reply: tid=0 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 client.14790 shutdown 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 msgr2=0x55ccccc56b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56b20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6320009930 tx=0x7f63200043b0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 shutdown_connections 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55ccccc6a530 0x55ccccc6b9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc59760 0x55ccccc57060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56b20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 --2- 172.21.15.175:0/2399062112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 >> 172.21.15.175:0/2399062112 conn(0x55ccccc4ab10 msgr2=0x55ccccc507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6318c00640 0 client.14790 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 shutdown_connections 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.587+0000 7f6338a37740 1 -- 172.21.15.175:0/2399062112 wait complete. 2024-07-29T00:35:38.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.588+0000 7f6338a37740 20 client.14790 trim_cache size 0 max 16384 2024-07-29T00:35:38.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 Processor -- start 2024-07-29T00:35:38.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 -- start start 2024-07-29T00:35:38.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4e580 0x55ccccc6a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc59760 0x55ccccc53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc69ce0 con 0x55ccccc4eea0 2024-07-29T00:35:38.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc6a3b0 con 0x55ccccc4e580 2024-07-29T00:35:38.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.607+0000 7f6338a37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc6aa80 con 0x55ccccc59760 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:52690/0 (socket says 172.21.15.175:52690) 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 -- 172.21.15.175:0/3896436506 learned_addr learned my addr 172.21.15.175:0/3896436506 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632f400640 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4e580 0x55ccccc6a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632f400640 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc59760 0x55ccccc53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 -- 172.21.15.175:0/3896436506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc59760 msgr2=0x55ccccc53840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc59760 0x55ccccc53840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 -- 172.21.15.175:0/3896436506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4e580 msgr2=0x55ccccc6a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4e580 0x55ccccc6a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 -- 172.21.15.175:0/3896436506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccccc4a940 con 0x55ccccc4eea0 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632f400640 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc59760 0x55ccccc53840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.608+0000 7f632e000640 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6320003d70 tx=0x7f6320005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363f0e3972824410 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f631b400640 1 -- 172.21.15.175:0/3896436506 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320035030 con 0x55ccccc4eea0 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f631b400640 1 -- 172.21.15.175:0/3896436506 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6320041070 con 0x55ccccc4eea0 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f631b400640 1 -- 172.21.15.175:0/3896436506 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320031880 con 0x55ccccc4eea0 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 -- 172.21.15.175:0/3896436506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 msgr2=0x55ccccc56eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6320003d70 tx=0x7f6320005d50 comp rx=0 tx=0).stop 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 -- 172.21.15.175:0/3896436506 shutdown_connections 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc59760 0x55ccccc53840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc4eea0 0x55ccccc56eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 --2- 172.21.15.175:0/3896436506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4e580 0x55ccccc6a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.609+0000 7f6338a37740 1 -- 172.21.15.175:0/3896436506 >> 172.21.15.175:0/3896436506 conn(0x55ccccc4ab10 msgr2=0x55ccccc51840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.610+0000 7f6338a37740 1 -- 172.21.15.175:0/3896436506 shutdown_connections 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.610+0000 7f6338a37740 1 -- 172.21.15.175:0/3896436506 wait complete. 2024-07-29T00:35:38.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.610+0000 7f6338a37740 1 -- start start 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.610+0000 7f6338a37740 1 Processor -- start 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 0x55ccccc74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc59760 0x55ccccc561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc6b820 con 0x55ccccc59760 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc6b150 con 0x55ccccc4eea0 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ccccc6bef0 con 0x55ccccc4e580 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f6338a37740 10 client.0 _ll_register_callbacks cb 0x7f6320009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 0x55ccccc74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 0x55ccccc74240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38398/0 (socket says 172.21.15.175:38398) 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 -- 172.21.15.175:0/3138237380 learned_addr learned my addr 172.21.15.175:0/3138237380 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:38.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632f400640 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc59760 0x55ccccc561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 -- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 msgr2=0x55ccccc73d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc73d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 -- 172.21.15.175:0/3138237380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc59760 msgr2=0x55ccccc561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc59760 0x55ccccc561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.611+0000 7f632e000640 1 -- 172.21.15.175:0/3138237380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccccc568e0 con 0x55ccccc4eea0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f632e000640 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 0x55ccccc74240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63200043f0 tx=0x7f6320005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0381b1f661c5bad server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6318c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6318c00640 1 -- 172.21.15.175:0/3138237380 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320060030 con 0x55ccccc4eea0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6318c00640 1 -- 172.21.15.175:0/3138237380 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6320003df0 con 0x55ccccc4eea0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6320041070 con 0x55ccccc4eea0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6318c00640 1 -- 172.21.15.175:0/3138237380 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6320035040 con 0x55ccccc4eea0 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6338a37740 10 client.24890 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:38.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.612+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccccc71830 con 0x55ccccc4eea0 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6318c00640 1 -- 172.21.15.175:0/3138237380 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6320041070 con 0x55ccccc4eea0 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6318c00640 1 -- 172.21.15.175:0/3138237380 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6320031910 con 0x55ccccc4eea0 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6338a37740 10 client.24890 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6338a37740 10 client.24890 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6338a37740 20 client.24890 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6338a37740 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55cccccd90a0 0x55cccccd7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6338a37740 4 client.24890 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:35:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "status"}) -- 0x55ccccbbc210 con 0x55cccccd90a0 2024-07-29T00:35:38.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.613+0000 7f632f400640 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55cccccd90a0 0x55cccccd7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:38.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f632f400640 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55cccccd90a0 0x55cccccd7db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e872f77b05e33b20 server_cookie=76bf6660c4b8fd17 in_seq=0 out_seq=0 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f6318c00640 10 client.24890 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f6318c00640 1 -- 172.21.15.175:0/3138237380 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ccccbbc210 con 0x55cccccd90a0 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f6318c00640 10 client.24890 handle_command_reply: tid=0 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f6338a37740 1 client.24890 shutdown 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 msgr2=0x55ccccc74240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.614+0000 7f6338a37740 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 0x55ccccc74240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63200043f0 tx=0x7f6320005250 comp rx=0 tx=0).stop 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 shutdown_connections 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccccc59760 0x55ccccc561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55cccccd90a0 0x55cccccd7db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccccc4eea0 0x55ccccc74240 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 --2- 172.21.15.175:0/3138237380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccccc4e580 0x55ccccc73d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6318c00640 0 client.24890 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 >> 172.21.15.175:0/3138237380 conn(0x55ccccc4ab10 msgr2=0x55ccccc64700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 shutdown_connections 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 1 -- 172.21.15.175:0/3138237380 wait complete. 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.615+0000 7f6338a37740 20 client.24890 trim_cache size 0 max 16384 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 4, 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24484, 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:35:38.619 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 85.940765096000007, 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 89, 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 132.99619653299999 2024-07-29T00:35:38.620 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63140a2c90 msgr2=0x7f63140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63140a2c90 0x7f63140a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ccccc4b970 tx=0x7f631c0d2f10 comp rx=0 tx=0).stop 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 msgr2=0x7f6330100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 0x7f6330100e10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f631c00e9f0 tx=0x7f631c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 shutdown_connections 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6330107740 0x7f6330101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63301005d0 0x7f6330101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f63140a2c90 0x7f63140a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 --2- 172.21.15.175:0/3085634774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63300ffc30 0x7f6330100e10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 >> 172.21.15.175:0/3085634774 conn(0x7f6330076d60 msgr2=0x7f6330104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:38.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 shutdown_connections 2024-07-29T00:35:38.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:38.616+0000 7f632fe00640 1 -- 172.21.15.175:0/3085634774 wait complete. 2024-07-29T00:35:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:38 smithi175 ceph-mon[32138]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-07-29T00:35:38.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:38 smithi181 ceph-mon[36706]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-07-29T00:35:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:38 smithi204 ceph-mon[37368]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-07-29T00:35:39.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.i status 2024-07-29T00:35:40.009 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 -- 172.21.15.175:0/1476341900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076d30 msgr2=0x7ff6b81121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1476341900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076d30 0x7ff6b81121e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009900 tx=0x7ff6a802a410 comp rx=0 tx=0).stop 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 -- 172.21.15.175:0/1476341900 shutdown_connections 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1476341900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b8112720 0x7ff6b8114b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1476341900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076d30 0x7ff6b81121e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1476341900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8076410 0x7ff6b80767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 -- 172.21.15.175:0/1476341900 >> 172.21.15.175:0/1476341900 conn(0x7ff6b80ff760 msgr2=0x7ff6b8101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:40.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 -- 172.21.15.175:0/1476341900 shutdown_connections 2024-07-29T00:35:40.678 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:40 smithi175 ceph-mon[32138]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.676+0000 7ff6c0600640 1 -- 172.21.15.175:0/1476341900 wait complete. 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.677+0000 7ff6c0600640 1 Processor -- start 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.677+0000 7ff6c0600640 1 -- start start 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6c0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 0x7ff6b81a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6c0600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b8076d30 0x7ff6b81a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6c0600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8112720 0x7ff6b819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6c0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81182e0 con 0x7ff6b8076410 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6c0600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8118160 con 0x7ff6b8076d30 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6c0600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8118460 con 0x7ff6b8112720 2024-07-29T00:35:40.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 0x7ff6b81a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 0x7ff6b81a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34974/0 (socket says 172.21.15.175:34974) 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6bdc00640 1 -- 172.21.15.175:0/1043452852 learned_addr learned my addr 172.21.15.175:0/1043452852 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6be600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8112720 0x7ff6b819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.678+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b8076d30 0x7ff6b81a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6be600640 1 -- 172.21.15.175:0/1043452852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b8076d30 msgr2=0x7ff6b81a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6be600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b8076d30 0x7ff6b81a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6be600640 1 -- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 msgr2=0x7ff6b81a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6be600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 0x7ff6b81a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6be600640 1 -- 172.21.15.175:0/1043452852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b819ef00 con 0x7ff6b8112720 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 0x7ff6b81a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6be600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8112720 0x7ff6b819e690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff6b807a700 tx=0x7ff6b000edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6b6000640 1 -- 172.21.15.175:0/1043452852 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0004d20 con 0x7ff6b8112720 2024-07-29T00:35:40.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b819f1f0 con 0x7ff6b8112720 2024-07-29T00:35:40.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6b6000640 1 -- 172.21.15.175:0/1043452852 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b0005020 con 0x7ff6b8112720 2024-07-29T00:35:40.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.679+0000 7ff6b6000640 1 -- 172.21.15.175:0/1043452852 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00116d0 con 0x7ff6b8112720 2024-07-29T00:35:40.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.680+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b81ab540 con 0x7ff6b8112720 2024-07-29T00:35:40.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.682+0000 7ff6b6000640 1 -- 172.21.15.175:0/1043452852 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff6b0005540 con 0x7ff6b8112720 2024-07-29T00:35:40.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 Processor -- start 2024-07-29T00:35:40.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 -- start start 2024-07-29T00:35:40.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617be409790 con 0x5617be4a3370 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5617be3a1880 con 0x5617be4ae550 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6c320c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5617be405750 con 0x5617be4a3c90 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.683+0000 7ff6bd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34980/0 (socket says 172.21.15.175:34980) 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bd200640 1 -- 172.21.15.175:0/1630420480 learned_addr learned my addr 172.21.15.175:0/1630420480 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 -- 172.21.15.175:0/1630420480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 msgr2=0x5617be4b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 -- 172.21.15.175:0/1630420480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 msgr2=0x5617be4a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 -- 172.21.15.175:0/1630420480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617be17b2b0 con 0x5617be4a3c90 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6b6000640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6a00a2c90 0x7ff6a00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.684+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4ae010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0114f0 tx=0x7ff6ac0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=747c409040f65b00 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.685+0000 7ff6a6a00640 1 -- 172.21.15.175:0/1630420480 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac011250 con 0x5617be4a3c90 2024-07-29T00:35:40.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.685+0000 7ff6a6a00640 1 -- 172.21.15.175:0/1630420480 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac0035f0 con 0x5617be4a3c90 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.685+0000 7ff6b6000640 1 -- 172.21.15.175:0/1043452852 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff6b00cf200 con 0x7ff6b8112720 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.685+0000 7ff6a6a00640 1 -- 172.21.15.175:0/1630420480 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00f7c0 con 0x5617be4a3c90 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.685+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6a00a2c90 0x7ff6a00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 -- 172.21.15.175:0/1630420480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 msgr2=0x5617be4ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6a00a2c90 0x7ff6a00a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8031550 tx=0x7ff6a8041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4ae010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0114f0 tx=0x7ff6ac0119b0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 -- 172.21.15.175:0/1630420480 shutdown_connections 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4ae010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1630420480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 -- 172.21.15.175:0/1630420480 >> 172.21.15.175:0/1630420480 conn(0x5617be49f900 msgr2=0x5617be49fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 -- 172.21.15.175:0/1630420480 shutdown_connections 2024-07-29T00:35:40.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 -- 172.21.15.175:0/1630420480 wait complete. 2024-07-29T00:35:40.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.686+0000 7ff6c320c740 1 -- start start 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 Processor -- start 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3c90 0x5617be4ab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617be4b1120 con 0x5617be4a3370 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5617be3c4960 con 0x5617be4a3c90 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5617be4b12a0 con 0x5617be4ae550 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6c320c740 10 client.0 _ll_register_callbacks cb 0x5617be1187b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34984/0 (socket says 172.21.15.175:34984) 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6bdc00640 1 -- 172.21.15.175:0/3275570160 learned_addr learned my addr 172.21.15.175:0/3275570160 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:40.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6be600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.687+0000 7ff6bd200640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3c90 0x5617be4ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6be600640 1 -- 172.21.15.175:0/3275570160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3c90 msgr2=0x5617be4ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6be600640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3c90 0x5617be4ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6be600640 1 -- 172.21.15.175:0/3275570160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 msgr2=0x5617be4ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6be600640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6be600640 1 -- 172.21.15.175:0/3275570160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617be3d3a90 con 0x5617be4ae550 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff6be600640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4abec0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00ccb30 tx=0x7ff6b009b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce28cf787a33b93 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff69b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.688+0000 7ff69b400640 1 -- 172.21.15.175:0/3275570160 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00b4060 con 0x5617be4ae550 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac0035f0 con 0x5617be4ae550 2024-07-29T00:35:40.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff69b400640 1 -- 172.21.15.175:0/3275570160 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff6b009cbb0 con 0x5617be4ae550 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff69b400640 1 -- 172.21.15.175:0/3275570160 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6b001f040 con 0x5617be4ae550 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff6c320c740 10 client.24895 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617be4bcd30 con 0x5617be4ae550 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff69b400640 1 -- 172.21.15.175:0/3275570160 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac0035f0 con 0x5617be4ae550 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff69b400640 1 -- 172.21.15.175:0/3275570160 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6b000f040 con 0x5617be4ae550 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.689+0000 7ff6c320c740 10 client.24895 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6c320c740 10 client.24895 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6c320c740 20 client.24895 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be4bf2c0 0x5617be4c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6c320c740 4 client.24895 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:35:40.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617be416d00 con 0x5617be4bf2c0 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be4bf2c0 0x5617be4c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.690+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be4bf2c0 0x5617be4c0730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81030be0d9f7a045 server_cookie=fbbf3eb32261cec1 in_seq=0 out_seq=0 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.691+0000 7ff69b400640 10 client.24895 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.691+0000 7ff69b400640 1 -- 172.21.15.175:0/3275570160 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617be416d00 con 0x5617be4bf2c0 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.691+0000 7ff69b400640 10 client.24895 handle_command_reply: tid=0 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 client.24895 shutdown 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 msgr2=0x5617be4abec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4abec0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00ccb30 tx=0x7ff6b009b9f0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 shutdown_connections 2024-07-29T00:35:40.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be4bf2c0 0x5617be4c0730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4abec0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3c90 0x5617be4ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3275570160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff69b400640 0 client.24895 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 >> 172.21.15.175:0/3275570160 conn(0x5617be49f900 msgr2=0x5617be49ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 shutdown_connections 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 1 -- 172.21.15.175:0/3275570160 wait complete. 2024-07-29T00:35:40.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.692+0000 7ff6c320c740 20 client.24895 trim_cache size 0 max 16384 2024-07-29T00:35:40.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.712+0000 7ff6c320c740 1 Processor -- start 2024-07-29T00:35:40.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.712+0000 7ff6c320c740 1 -- start start 2024-07-29T00:35:40.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 0x5617be4a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3c90 0x5617be49fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6c320c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617be4bea70 con 0x5617be4a3c90 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6c320c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5617be4bf140 con 0x5617be4a3370 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6c320c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5617be4bf810 con 0x5617be4ae550 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 0x5617be4a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 0x5617be4a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35058/0 (socket says 172.21.15.175:35058) 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6bdc00640 1 -- 172.21.15.175:0/3453209329 learned_addr learned my addr 172.21.15.175:0/3453209329 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6be600640 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.713+0000 7ff6bd200640 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3c90 0x5617be49fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6bdc00640 1 -- 172.21.15.175:0/3453209329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 msgr2=0x5617be4a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6bdc00640 1 -- 172.21.15.175:0/3453209329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3c90 msgr2=0x5617be49fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3c90 0x5617be49fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6bdc00640 1 -- 172.21.15.175:0/3453209329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617be4b1230 con 0x5617be4a3370 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6be600640 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 0x5617be4a5dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0097f0 tx=0x7ff6ac003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3619055ff0fc688a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6a6000640 1 -- 172.21.15.175:0/3453209329 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004290 con 0x5617be4a3370 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6a6000640 1 -- 172.21.15.175:0/3453209329 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac004430 con 0x5617be4a3370 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.714+0000 7ff6a6000640 1 -- 172.21.15.175:0/3453209329 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac003a60 con 0x5617be4a3370 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 -- 172.21.15.175:0/3453209329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 msgr2=0x5617be4a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 0x5617be4a5dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0097f0 tx=0x7ff6ac003de0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 -- 172.21.15.175:0/3453209329 shutdown_connections 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4ae550 0x5617be4a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3c90 0x5617be49fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 --2- 172.21.15.175:0/3453209329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4a3370 0x5617be4a5dd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 -- 172.21.15.175:0/3453209329 >> 172.21.15.175:0/3453209329 conn(0x5617be49f900 msgr2=0x5617be4a6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 -- 172.21.15.175:0/3453209329 shutdown_connections 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.715+0000 7ff6c320c740 1 -- 172.21.15.175:0/3453209329 wait complete. 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 -- start start 2024-07-29T00:35:40.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 Processor -- start 2024-07-29T00:35:40.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617be4c05b0 con 0x5617be4a3370 2024-07-29T00:35:40.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5617be4bfee0 con 0x5617be4ae550 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5617be4c0c80 con 0x5617be4a3c90 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.716+0000 7ff6c320c740 10 client.0 _ll_register_callbacks cb 0x7ff6b0004ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44086/0 (socket says 172.21.15.175:44086) 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6be600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 -- 172.21.15.175:0/1194547106 learned_addr learned my addr 172.21.15.175:0/1194547106 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 -- 172.21.15.175:0/1194547106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 msgr2=0x5617be4a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 -- 172.21.15.175:0/1194547106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 msgr2=0x5617be4a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 -- 172.21.15.175:0/1194547106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617be4c79a0 con 0x5617be4a3c90 2024-07-29T00:35:40.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff6bd200640 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4a7bf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000c00 tx=0x7ff6a8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b930e1018e0a9b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.717+0000 7ff69b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff69b400640 1 -- 172.21.15.175:0/1194547106 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8035040 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac004430 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff69b400640 1 -- 172.21.15.175:0/1194547106 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff6a8004dd0 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff69b400640 1 -- 172.21.15.175:0/1194547106 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6a805b070 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff6c320c740 10 client.24901 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617be4c8c90 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.718+0000 7ff69b400640 1 -- 172.21.15.175:0/1194547106 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac004430 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff69b400640 1 -- 172.21.15.175:0/1194547106 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6a8053020 con 0x5617be4a3c90 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6c320c740 10 client.24901 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6c320c740 10 client.24901 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-29T00:35:40.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6c320c740 20 client.24901 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:40.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be52da20 0x5617be52c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:40.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6c320c740 4 client.24901 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:35:40.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "status"}) -- 0x5617be411210 con 0x5617be52da20 2024-07-29T00:35:40.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.719+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be52da20 0x5617be52c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6bdc00640 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be52da20 0x5617be52c730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a3f2c97ea150883 server_cookie=d7b825361648d15f in_seq=0 out_seq=0 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff69b400640 10 client.24901 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff69b400640 1 -- 172.21.15.175:0/1194547106 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5617be411210 con 0x5617be52da20 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff69b400640 10 client.24901 handle_command_reply: tid=0 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 client.24901 shutdown 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 msgr2=0x5617be4a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4a7bf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000c00 tx=0x7ff6a8005b50 comp rx=0 tx=0).stop 2024-07-29T00:35:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 shutdown_connections 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5617be52da20 0x5617be52c730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617be4ae550 0x5617be4a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff69b400640 0 client.24901 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617be4a3c90 0x5617be4a7bf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 --2- 172.21.15.175:0/1194547106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617be4a3370 0x5617be4a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.720+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 >> 172.21.15.175:0/1194547106 conn(0x5617be49f900 msgr2=0x5617be49fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 shutdown_connections 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c320c740 1 -- 172.21.15.175:0/1194547106 wait complete. 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c320c740 20 client.24901 trim_cache size 0 max 16384 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:35:40.724 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 3, 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24499, 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 89.062309334000005, 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 89, 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 129.768940961 2024-07-29T00:35:40.725 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:35:40.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6a00a2c90 msgr2=0x7ff6a00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6a00a2c90 0x7ff6a00a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8031550 tx=0x7ff6a8041040 comp rx=0 tx=0).stop 2024-07-29T00:35:40.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8112720 msgr2=0x7ff6b819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:40.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.721+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8112720 0x7ff6b819e690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff6b807a700 tx=0x7ff6b000edd0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 shutdown_connections 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff6a00a2c90 0x7ff6a00a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6b8112720 0x7ff6b819e690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b8076d30 0x7ff6b81a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 --2- 172.21.15.175:0/1043452852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6b8076410 0x7ff6b81a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 >> 172.21.15.175:0/1043452852 conn(0x7ff6b80ff760 msgr2=0x7ff6b8101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 shutdown_connections 2024-07-29T00:35:40.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:40.722+0000 7ff6c0600640 1 -- 172.21.15.175:0/1043452852 wait complete. 2024-07-29T00:35:40.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:40 smithi181 ceph-mon[36706]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-29T00:35:41.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:40 smithi204 ceph-mon[37368]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-29T00:35:41.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.h status 2024-07-29T00:35:42.081 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:42.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 -- 172.21.15.175:0/4230955617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 msgr2=0x7fc94c10e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 --2- 172.21.15.175:0/4230955617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c10e740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc944009930 tx=0x7fc94402a420 comp rx=0 tx=0).stop 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 -- 172.21.15.175:0/4230955617 shutdown_connections 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 --2- 172.21.15.175:0/4230955617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c10e740 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 --2- 172.21.15.175:0/4230955617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c101870 0x7fc94c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 --2- 172.21.15.175:0/4230955617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c107870 0x7fc94c107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 -- 172.21.15.175:0/4230955617 >> 172.21.15.175:0/4230955617 conn(0x7fc94c0fd5b0 msgr2=0x7fc94c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 -- 172.21.15.175:0/4230955617 shutdown_connections 2024-07-29T00:35:42.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.738+0000 7fc954000640 1 -- 172.21.15.175:0/4230955617 wait complete. 2024-07-29T00:35:42.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.739+0000 7fc954000640 1 Processor -- start 2024-07-29T00:35:42.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.739+0000 7fc954000640 1 -- start start 2024-07-29T00:35:42.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc954000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c101870 0x7fc94c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc954000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc954000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 0x7fc94c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc954000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111c70 con 0x7fc94c102210 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc954000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111af0 con 0x7fc94c101870 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc954000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111df0 con 0x7fc94c107870 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc950c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc952000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 0x7fc94c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc952000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 0x7fc94c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44108/0 (socket says 172.21.15.175:44108) 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc952000640 1 -- 172.21.15.175:0/2487460619 learned_addr learned my addr 172.21.15.175:0/2487460619 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:42.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.740+0000 7fc951600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c101870 0x7fc94c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc950c00640 1 -- 172.21.15.175:0/2487460619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 msgr2=0x7fc94c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc950c00640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 0x7fc94c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc950c00640 1 -- 172.21.15.175:0/2487460619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c101870 msgr2=0x7fc94c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc950c00640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c101870 0x7fc94c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc950c00640 1 -- 172.21.15.175:0/2487460619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c19f2f0 con 0x7fc94c102210 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc951600640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c101870 0x7fc94c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc952000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 0x7fc94c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc950c00640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c19a880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc93800e9f0 tx=0x7fc93800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc94a000640 1 -- 172.21.15.175:0/2487460619 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93800ccc0 con 0x7fc94c102210 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc94a000640 1 -- 172.21.15.175:0/2487460619 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc938010910 con 0x7fc94c102210 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c19f5e0 con 0x7fc94c102210 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc94a000640 1 -- 172.21.15.175:0/2487460619 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc938018960 con 0x7fc94c102210 2024-07-29T00:35:42.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.741+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1a6ec0 con 0x7fc94c102210 2024-07-29T00:35:42.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc94a000640 1 -- 172.21.15.175:0/2487460619 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc938010ac0 con 0x7fc94c102210 2024-07-29T00:35:42.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 Processor -- start 2024-07-29T00:35:42.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 -- start start 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db91c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91cc90 0x5643db927010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db927550 0x5643db929940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5643db882790 con 0x5643db91cc90 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643db81a880 con 0x5643db927550 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.745+0000 7fc956a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643db87e750 con 0x5643db91c370 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db91c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db91c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44120/0 (socket says 172.21.15.175:44120) 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 -- 172.21.15.175:0/360125866 learned_addr learned my addr 172.21.15.175:0/360125866 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:42.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc951600640 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91cc90 0x5643db927010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc950c00640 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db927550 0x5643db929940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 -- 172.21.15.175:0/360125866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db927550 msgr2=0x5643db929940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db927550 0x5643db929940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 -- 172.21.15.175:0/360125866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91cc90 msgr2=0x5643db927010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91cc90 0x5643db927010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 -- 172.21.15.175:0/360125866 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643db5f42b0 con 0x5643db91c370 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc950c00640 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db927550 0x5643db929940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc952000640 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db91c750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc944005c00 tx=0x7fc944005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=561aa935084c673d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.746+0000 7fc94a000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9340a2c90 0x7fc9340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.747+0000 7fc94a000640 1 -- 172.21.15.175:0/2487460619 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc9380cf540 con 0x7fc94c102210 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.747+0000 7fc93e600640 1 -- 172.21.15.175:0/360125866 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944035040 con 0x5643db91c370 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.747+0000 7fc951600640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9340a2c90 0x7fc9340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.747+0000 7fc93e600640 1 -- 172.21.15.175:0/360125866 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9440318f0 con 0x5643db91c370 2024-07-29T00:35:42.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.747+0000 7fc93e600640 1 -- 172.21.15.175:0/360125866 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94403c440 con 0x5643db91c370 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 -- 172.21.15.175:0/360125866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 msgr2=0x5643db91c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db91c750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc944005c00 tx=0x7fc944005d50 comp rx=0 tx=0).stop 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc951600640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9340a2c90 0x7fc9340a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5643db919760 tx=0x7fc940009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 -- 172.21.15.175:0/360125866 shutdown_connections 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db927550 0x5643db929940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91cc90 0x5643db927010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 --2- 172.21.15.175:0/360125866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db91c750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 -- 172.21.15.175:0/360125866 >> 172.21.15.175:0/360125866 conn(0x5643db918900 msgr2=0x5643db918ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 -- 172.21.15.175:0/360125866 shutdown_connections 2024-07-29T00:35:42.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.748+0000 7fc956a76740 1 -- 172.21.15.175:0/360125866 wait complete. 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 -- start start 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 Processor -- start 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91c370 0x5643db924470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91cc90 0x5643db9249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db924ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5643db92a120 con 0x5643db927550 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643db83d960 con 0x5643db91c370 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643db92a2a0 con 0x5643db91cc90 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.749+0000 7fc956a76740 10 client.0 _ll_register_callbacks cb 0x5643db5917b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc952000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db924ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91cc90 0x5643db9249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc951600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91c370 0x5643db924470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc952000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db924ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35044/0 (socket says 172.21.15.175:35044) 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc952000640 1 -- 172.21.15.175:0/545654207 learned_addr learned my addr 172.21.15.175:0/545654207 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 -- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91c370 msgr2=0x5643db924470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91c370 0x5643db924470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 -- 172.21.15.175:0/545654207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 msgr2=0x5643db924ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db924ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 -- 172.21.15.175:0/545654207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643db84ca90 con 0x5643db91cc90 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc950c00640 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91cc90 0x5643db9249b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc9380cce70 tx=0x7fc93809ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51278707af873882 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.750+0000 7fc932a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc932a00640 1 -- 172.21.15.175:0/545654207 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380cf310 con 0x5643db91cc90 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc932a00640 1 -- 172.21.15.175:0/545654207 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc93809b0c0 con 0x5643db91cc90 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9440318f0 con 0x5643db91cc90 2024-07-29T00:35:42.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc932a00640 1 -- 172.21.15.175:0/545654207 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc93809bc70 con 0x5643db91cc90 2024-07-29T00:35:42.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc956a76740 10 client.24913 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:42.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643db935d60 con 0x5643db91cc90 2024-07-29T00:35:42.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc932a00640 1 -- 172.21.15.175:0/545654207 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9440318f0 con 0x5643db91cc90 2024-07-29T00:35:42.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc932a00640 1 -- 172.21.15.175:0/545654207 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc938018c20 con 0x5643db91cc90 2024-07-29T00:35:42.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc956a76740 10 client.24913 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc956a76740 10 client.24913 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.751+0000 7fc956a76740 20 client.24913 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.752+0000 7fc956a76740 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9382d0 0x5643db939740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.752+0000 7fc956a76740 4 client.24913 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.752+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643db88fd00 con 0x5643db9382d0 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.752+0000 7fc951600640 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9382d0 0x5643db939740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.752+0000 7fc951600640 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9382d0 0x5643db939740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=313eb9ebd5463808 server_cookie=d835e3e913378a9e in_seq=0 out_seq=0 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.752+0000 7fc932a00640 10 client.24913 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.753+0000 7fc932a00640 1 -- 172.21.15.175:0/545654207 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5643db88fd00 con 0x5643db9382d0 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.753+0000 7fc932a00640 10 client.24913 handle_command_reply: tid=0 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.753+0000 7fc956a76740 1 client.24913 shutdown 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.753+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91cc90 msgr2=0x5643db9249b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91cc90 0x5643db9249b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc9380cce70 tx=0x7fc93809ba40 comp rx=0 tx=0).stop 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 shutdown_connections 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db924ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91cc90 0x5643db9249b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9382d0 0x5643db939740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 --2- 172.21.15.175:0/545654207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91c370 0x5643db924470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 >> 172.21.15.175:0/545654207 conn(0x5643db918900 msgr2=0x5643db91e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc932a00640 0 client.24913 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 shutdown_connections 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 1 -- 172.21.15.175:0/545654207 wait complete. 2024-07-29T00:35:42.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.754+0000 7fc956a76740 20 client.24913 trim_cache size 0 max 16384 2024-07-29T00:35:42.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.773+0000 7fc956a76740 1 Processor -- start 2024-07-29T00:35:42.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 -- start start 2024-07-29T00:35:42.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 0x5643db939740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db91edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db927550 0x5643db91f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5643db937a80 con 0x5643db91c370 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643db938150 con 0x5643db91cc90 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc956a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643db938820 con 0x5643db927550 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc951600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 0x5643db939740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc951600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 0x5643db939740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35056/0 (socket says 172.21.15.175:35056) 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc951600640 1 -- 172.21.15.175:0/807667984 learned_addr learned my addr 172.21.15.175:0/807667984 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc950c00640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db91edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.774+0000 7fc952000640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db927550 0x5643db91f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc950c00640 1 -- 172.21.15.175:0/807667984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db927550 msgr2=0x5643db91f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc950c00640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db927550 0x5643db91f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc950c00640 1 -- 172.21.15.175:0/807667984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 msgr2=0x5643db939740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc950c00640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 0x5643db939740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc950c00640 1 -- 172.21.15.175:0/807667984 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643db91cac0 con 0x5643db91cc90 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc952000640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db927550 0x5643db91f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc951600640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 0x5643db939740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc950c00640 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db91edd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5643db925a20 tx=0x7fc93809bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15dd9485b50dd8d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc93dc00640 1 -- 172.21.15.175:0/807667984 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93809ff70 con 0x5643db91cc90 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc93dc00640 1 -- 172.21.15.175:0/807667984 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9380b2050 con 0x5643db91cc90 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.775+0000 7fc93dc00640 1 -- 172.21.15.175:0/807667984 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93809b0c0 con 0x5643db91cc90 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 -- 172.21.15.175:0/807667984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 msgr2=0x5643db91edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db91edd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5643db925a20 tx=0x7fc93809bb70 comp rx=0 tx=0).stop 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 -- 172.21.15.175:0/807667984 shutdown_connections 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db927550 0x5643db91f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db91edd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 --2- 172.21.15.175:0/807667984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db91c370 0x5643db939740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 -- 172.21.15.175:0/807667984 >> 172.21.15.175:0/807667984 conn(0x5643db918900 msgr2=0x5643db913670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 -- 172.21.15.175:0/807667984 shutdown_connections 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.776+0000 7fc956a76740 1 -- 172.21.15.175:0/807667984 wait complete. 2024-07-29T00:35:42.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.777+0000 7fc956a76740 1 -- start start 2024-07-29T00:35:42.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.777+0000 7fc956a76740 1 Processor -- start 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db920e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db921380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db940990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5643db9395c0 con 0x5643db927550 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643db938ef0 con 0x5643db91cc90 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643db939c90 con 0x5643db91c370 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc956a76740 10 client.0 _ll_register_callbacks cb 0x7fc93800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc952000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db940990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc951600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db920e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc952000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db940990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35060/0 (socket says 172.21.15.175:35060) 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc952000640 1 -- 172.21.15.175:0/1596226770 learned_addr learned my addr 172.21.15.175:0/1596226770 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:42.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc950c00640 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db921380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc951600640 1 -- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 msgr2=0x5643db921380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc951600640 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db921380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc951600640 1 -- 172.21.15.175:0/1596226770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 msgr2=0x5643db940990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc951600640 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db940990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.778+0000 7fc951600640 1 -- 172.21.15.175:0/1596226770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643db940fd0 con 0x5643db91c370 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc951600640 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db920e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5643db925860 tx=0x7fc940004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d651846daa711fb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc932a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc932a00640 1 -- 172.21.15.175:0/1596226770 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94002b440 con 0x5643db91c370 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9380b2050 con 0x5643db91c370 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc932a00640 1 -- 172.21.15.175:0/1596226770 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc94002b5e0 con 0x5643db91c370 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc932a00640 1 -- 172.21.15.175:0/1596226770 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc94002b7e0 con 0x5643db91c370 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc956a76740 10 client.24919 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:42.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643db942310 con 0x5643db91c370 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.779+0000 7fc932a00640 1 -- 172.21.15.175:0/1596226770 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9380b2050 con 0x5643db91c370 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc932a00640 1 -- 172.21.15.175:0/1596226770 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc94003dcb0 con 0x5643db91c370 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc956a76740 10 client.24919 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc956a76740 10 client.24919 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc956a76740 20 client.24919 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc956a76740 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9af880 0x5643db9ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc956a76740 4 client.24919 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "status"}) -- 0x5643db88a210 con 0x5643db9af880 2024-07-29T00:35:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.780+0000 7fc950c00640 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9af880 0x5643db9ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc950c00640 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9af880 0x5643db9ae590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb87364c65e2abc1 server_cookie=d52c8b436a26f59 in_seq=0 out_seq=0 2024-07-29T00:35:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc932a00640 10 client.24919 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:35:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc932a00640 1 -- 172.21.15.175:0/1596226770 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5643db88a210 con 0x5643db9af880 2024-07-29T00:35:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc932a00640 10 client.24919 handle_command_reply: tid=0 2024-07-29T00:35:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 client.24919 shutdown 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 msgr2=0x5643db920e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db920e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5643db925860 tx=0x7fc940004a40 comp rx=0 tx=0).stop 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 shutdown_connections 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643db927550 0x5643db940990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5643db9af880 0x5643db9ae590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643db91cc90 0x5643db921380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 --2- 172.21.15.175:0/1596226770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643db91c370 0x5643db920e40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc932a00640 0 client.24919 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 >> 172.21.15.175:0/1596226770 conn(0x5643db918900 msgr2=0x5643db91f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 shutdown_connections 2024-07-29T00:35:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 1 -- 172.21.15.175:0/1596226770 wait complete. 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.781+0000 7fc956a76740 20 client.24919 trim_cache size 0 max 16384 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 2, 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24503, 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:35:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 92.133661605, 2024-07-29T00:35:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:35:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 89, 2024-07-29T00:35:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:35:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 134.40727222199999 2024-07-29T00:35:42.787 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:35:42.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9340a2c90 msgr2=0x7fc9340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9340a2c90 0x7fc9340a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5643db919760 tx=0x7fc940009500 comp rx=0 tx=0).stop 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 msgr2=0x7fc94c19a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c19a880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc93800e9f0 tx=0x7fc93800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 shutdown_connections 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc9340a2c90 0x7fc9340a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c107870 0x7fc94c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c102210 0x7fc94c19a880 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 --2- 172.21.15.175:0/2487460619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc94c101870 0x7fc94c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 >> 172.21.15.175:0/2487460619 conn(0x7fc94c0fd5b0 msgr2=0x7fc94c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.782+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 shutdown_connections 2024-07-29T00:35:42.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:42.783+0000 7fc954000640 1 -- 172.21.15.175:0/2487460619 wait complete. 2024-07-29T00:35:42.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:42 smithi181 ceph-mon[36706]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-07-29T00:35:43.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:42 smithi175 ceph-mon[32138]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-07-29T00:35:43.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:42 smithi204 ceph-mon[37368]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-07-29T00:35:43.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.g status 2024-07-29T00:35:44.226 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:44.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 -- 172.21.15.175:0/3198316960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc254102050 msgr2=0x7fc25410e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 --2- 172.21.15.175:0/3198316960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc254102050 0x7fc25410e580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009cd0 tx=0x7fc24c02a8f0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 -- 172.21.15.175:0/3198316960 shutdown_connections 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 --2- 172.21.15.175:0/3198316960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc254102050 0x7fc25410e580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 --2- 172.21.15.175:0/3198316960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2541016b0 0x7fc254101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 --2- 172.21.15.175:0/3198316960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2541076b0 0x7fc254107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 -- 172.21.15.175:0/3198316960 >> 172.21.15.175:0/3198316960 conn(0x7fc2540fd3d0 msgr2=0x7fc2540ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 -- 172.21.15.175:0/3198316960 shutdown_connections 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.880+0000 7fc253e00640 1 -- 172.21.15.175:0/3198316960 wait complete. 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.881+0000 7fc253e00640 1 Processor -- start 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.881+0000 7fc253e00640 1 -- start start 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2541016b0 0x7fc254197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc254102050 0x7fc254198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 0x7fc25419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc254111b00 con 0x7fc2541076b0 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc254111980 con 0x7fc254102050 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc254111c80 con 0x7fc2541016b0 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 0x7fc25419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 0x7fc25419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35074/0 (socket says 172.21.15.175:35074) 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 -- 172.21.15.175:0/4290235589 learned_addr learned my addr 172.21.15.175:0/4290235589 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc252000640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc254102050 0x7fc254198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc252a00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2541016b0 0x7fc254197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 -- 172.21.15.175:0/4290235589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2541016b0 msgr2=0x7fc254197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2541016b0 0x7fc254197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 -- 172.21.15.175:0/4290235589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc254102050 msgr2=0x7fc254198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc254102050 0x7fc254198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.882+0000 7fc253400640 1 -- 172.21.15.175:0/4290235589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25419ce90 con 0x7fc2541076b0 2024-07-29T00:35:44.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.883+0000 7fc253400640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 0x7fc25419c7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc24c005e30 tx=0x7fc24c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:44.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.883+0000 7fc24b400640 1 -- 172.21.15.175:0/4290235589 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c030dc0 con 0x7fc2541076b0 2024-07-29T00:35:44.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.883+0000 7fc24b400640 1 -- 172.21.15.175:0/4290235589 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24c040070 con 0x7fc2541076b0 2024-07-29T00:35:44.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.883+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25419d120 con 0x7fc2541076b0 2024-07-29T00:35:44.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.883+0000 7fc24b400640 1 -- 172.21.15.175:0/4290235589 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c045ad0 con 0x7fc2541076b0 2024-07-29T00:35:44.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.883+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541a4a60 con 0x7fc2541076b0 2024-07-29T00:35:44.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 Processor -- start 2024-07-29T00:35:44.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc24b400640 1 -- 172.21.15.175:0/4290235589 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc24c004700 con 0x7fc2541076b0 2024-07-29T00:35:44.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 -- start start 2024-07-29T00:35:44.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36713960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 0x55ac3671e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36720b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac36679790 con 0x55ac36713580 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac36611880 con 0x55ac3671e760 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.887+0000 7fc25c859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ac36675750 con 0x55ac36713ea0 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 0x55ac3671e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36713960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 0x55ac3671e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44198/0 (socket says 172.21.15.175:44198) 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 -- 172.21.15.175:0/1394176849 learned_addr learned my addr 172.21.15.175:0/1394176849 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252000640 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36720b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 -- 172.21.15.175:0/1394176849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 msgr2=0x55ac36720b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36720b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 -- 172.21.15.175:0/1394176849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 msgr2=0x55ac36713960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36713960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252a00640 1 -- 172.21.15.175:0/1394176849 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac363eb2b0 con 0x55ac36713ea0 2024-07-29T00:35:44.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc252000640 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36713960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:44.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.888+0000 7fc24b400640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2380a2c10 0x7fc2380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.889+0000 7fc252a00640 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 0x55ac3671e220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc244009c70 tx=0x7fc244011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f7b533a75670c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:44.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.889+0000 7fc24b400640 1 -- 172.21.15.175:0/4290235589 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc24c0f0090 con 0x7fc2541076b0 2024-07-29T00:35:44.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.889+0000 7fc252000640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2380a2c10 0x7fc2380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.889+0000 7fc23fe00640 1 -- 172.21.15.175:0/1394176849 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244004230 con 0x55ac36713ea0 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.889+0000 7fc23fe00640 1 -- 172.21.15.175:0/1394176849 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2440043d0 con 0x55ac36713ea0 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.889+0000 7fc23fe00640 1 -- 172.21.15.175:0/1394176849 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244019790 con 0x55ac36713ea0 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 -- 172.21.15.175:0/1394176849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 msgr2=0x55ac3671e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 0x55ac3671e220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc244009c70 tx=0x7fc244011810 comp rx=0 tx=0).stop 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc252000640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2380a2c10 0x7fc2380a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc240009730 tx=0x7fc2400096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 -- 172.21.15.175:0/1394176849 shutdown_connections 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36720b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713ea0 0x55ac3671e220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 --2- 172.21.15.175:0/1394176849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36713960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 -- 172.21.15.175:0/1394176849 >> 172.21.15.175:0/1394176849 conn(0x55ac3670fb10 msgr2=0x55ac3670fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 -- 172.21.15.175:0/1394176849 shutdown_connections 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.890+0000 7fc25c859740 1 -- 172.21.15.175:0/1394176849 wait complete. 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 -- start start 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 Processor -- start 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac3671b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac3671bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac3671e760 0x55ac3671c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac36721330 con 0x55ac3671e760 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac36634960 con 0x55ac36713ea0 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ac367214b0 con 0x55ac36713580 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.891+0000 7fc25c859740 10 client.0 _ll_register_callbacks cb 0x55ac363887b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac3671bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac3671b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac3671b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44200/0 (socket says 172.21.15.175:44200) 2024-07-29T00:35:44.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252a00640 1 -- 172.21.15.175:0/3975897087 learned_addr learned my addr 172.21.15.175:0/3975897087 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:44.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc253400640 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac3671e760 0x55ac3671c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 -- 172.21.15.175:0/3975897087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 msgr2=0x55ac3671b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac3671b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 -- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac3671e760 msgr2=0x55ac3671c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac3671e760 0x55ac3671c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 -- 172.21.15.175:0/3975897087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac36643a90 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc252000640 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac3671bbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc254106c80 tx=0x7fc240004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5807d5757480355c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.892+0000 7fc23cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc23cc00640 1 -- 172.21.15.175:0/3975897087 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240022850 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2440043d0 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc23cc00640 1 -- 172.21.15.175:0/3975897087 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc2400229f0 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc23cc00640 1 -- 172.21.15.175:0/3975897087 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc240022bf0 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc25c859740 10 client.24938 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac36731420 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.893+0000 7fc23cc00640 1 -- 172.21.15.175:0/3975897087 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2440043d0 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc23cc00640 1 -- 172.21.15.175:0/3975897087 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc240035420 con 0x55ac36713ea0 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc25c859740 10 client.24938 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:44.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc25c859740 10 client.24938 resolve_mds: resolved name 'g' to daemon mds.g 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc25c859740 20 client.24938 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc25c859740 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac36733a90 0x55ac36734f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc25c859740 4 client.24938 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac36686d00 con 0x55ac36733a90 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.894+0000 7fc252a00640 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac36733a90 0x55ac36734f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.895+0000 7fc252a00640 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac36733a90 0x55ac36734f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf0acdd940789e86 server_cookie=935f771e738f6d07 in_seq=0 out_seq=0 2024-07-29T00:35:44.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.895+0000 7fc23cc00640 10 client.24938 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc23cc00640 1 -- 172.21.15.175:0/3975897087 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ac36686d00 con 0x55ac36733a90 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc23cc00640 10 client.24938 handle_command_reply: tid=0 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 client.24938 shutdown 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 msgr2=0x55ac3671bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac3671bbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc254106c80 tx=0x7fc240004730 comp rx=0 tx=0).stop 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 shutdown_connections 2024-07-29T00:35:44.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac36733a90 0x55ac36734f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac3671e760 0x55ac3671c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac3671bbb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 --2- 172.21.15.175:0/3975897087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac3671b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc23cc00640 0 client.24938 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 >> 172.21.15.175:0/3975897087 conn(0x55ac3670fb10 msgr2=0x55ac36715840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 shutdown_connections 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.896+0000 7fc25c859740 1 -- 172.21.15.175:0/3975897087 wait complete. 2024-07-29T00:35:44.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.897+0000 7fc25c859740 20 client.24938 trim_cache size 0 max 16384 2024-07-29T00:35:44.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.916+0000 7fc25c859740 1 Processor -- start 2024-07-29T00:35:44.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.916+0000 7fc25c859740 1 -- start start 2024-07-29T00:35:44.917 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:44 smithi175 ceph-mon[32138]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc25c859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac36715fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc25c859740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713ea0 0x55ac3671bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc25c859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36719f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc25c859740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac36733240 con 0x55ac36713ea0 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc25c859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac36733910 con 0x55ac3671e760 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc25c859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ac36733fe0 con 0x55ac36713580 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36719f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc252000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713ea0 0x55ac3671bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36719f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35166/0 (socket says 172.21.15.175:35166) 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 -- 172.21.15.175:0/1455519994 learned_addr learned my addr 172.21.15.175:0/1455519994 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc252a00640 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac36715fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 -- 172.21.15.175:0/1455519994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 msgr2=0x55ac36715fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac36715fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 -- 172.21.15.175:0/1455519994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713ea0 msgr2=0x55ac3671bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713ea0 0x55ac3671bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc253400640 1 -- 172.21.15.175:0/1455519994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac36721440 con 0x55ac3671e760 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.917+0000 7fc252000640 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713ea0 0x55ac3671bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.918+0000 7fc253400640 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36719f40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006010 tx=0x7fc24c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33804516066ef27c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.918+0000 7fc23f400640 1 -- 172.21.15.175:0/1455519994 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0bc910 con 0x55ac3671e760 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.918+0000 7fc23f400640 1 -- 172.21.15.175:0/1455519994 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24c036040 con 0x55ac3671e760 2024-07-29T00:35:44.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.918+0000 7fc23f400640 1 -- 172.21.15.175:0/1455519994 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c045c70 con 0x55ac3671e760 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 -- 172.21.15.175:0/1455519994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 msgr2=0x55ac36719f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36719f40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006010 tx=0x7fc24c005ef0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 -- 172.21.15.175:0/1455519994 shutdown_connections 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac3671e760 0x55ac36719f40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713ea0 0x55ac3671bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 --2- 172.21.15.175:0/1455519994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac36713580 0x55ac36715fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 -- 172.21.15.175:0/1455519994 >> 172.21.15.175:0/1455519994 conn(0x55ac3670fb10 msgr2=0x55ac3670fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 -- 172.21.15.175:0/1455519994 shutdown_connections 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.919+0000 7fc25c859740 1 -- 172.21.15.175:0/1455519994 wait complete. 2024-07-29T00:35:44.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.920+0000 7fc25c859740 1 -- start start 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.920+0000 7fc25c859740 1 Processor -- start 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36736e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac367373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac3671e760 0x55ac367378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac36734d80 con 0x55ac36713580 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac367346b0 con 0x55ac36713ea0 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ac36735450 con 0x55ac3671e760 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc25c859740 10 client.0 _ll_register_callbacks cb 0x7fc240010f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:44.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac367373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36736e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac367373a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35176/0 (socket says 172.21.15.175:35176) 2024-07-29T00:35:44.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 -- 172.21.15.175:0/487983301 learned_addr learned my addr 172.21.15.175:0/487983301 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:44.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc253400640 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac3671e760 0x55ac367378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 -- 172.21.15.175:0/487983301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac3671e760 msgr2=0x55ac367378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac3671e760 0x55ac367378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 -- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 msgr2=0x55ac36736e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36736e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.921+0000 7fc252000640 1 -- 172.21.15.175:0/487983301 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3673b650 con 0x55ac36713ea0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.922+0000 7fc252000640 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac367373a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc240006010 tx=0x7fc240035650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb78d18c042b0a8d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.922+0000 7fc23cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.922+0000 7fc23cc00640 1 -- 172.21.15.175:0/487983301 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24003a040 con 0x55ac36713ea0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.922+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc24c036040 con 0x55ac36713ea0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.922+0000 7fc23cc00640 1 -- 172.21.15.175:0/487983301 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc240022850 con 0x55ac36713ea0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc23cc00640 1 -- 172.21.15.175:0/487983301 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc240035e60 con 0x55ac36713ea0 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 10 client.24944 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:44.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac3679e3e0 con 0x55ac36713ea0 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc23cc00640 1 -- 172.21.15.175:0/487983301 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24c036040 con 0x55ac36713ea0 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc23cc00640 1 -- 172.21.15.175:0/487983301 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc24003ecc0 con 0x55ac36713ea0 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 10 client.24944 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 10 client.24944 resolve_mds: resolved name 'g' to daemon mds.g 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 20 client.24944 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac367a2fd0 0x55ac367a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 4 client.24944 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.923+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "status"}) -- 0x55ac36681210 con 0x55ac367a2fd0 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.924+0000 7fc252a00640 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac367a2fd0 0x55ac367a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.924+0000 7fc23cc00640 10 client.24944 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:35:44.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.924+0000 7fc252a00640 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac367a2fd0 0x55ac367a1ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8729be6f6ec16c1 server_cookie=e89fe6a391421683 in_seq=0 out_seq=0 2024-07-29T00:35:44.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.924+0000 7fc23cc00640 1 -- 172.21.15.175:0/487983301 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ac36681210 con 0x55ac367a2fd0 2024-07-29T00:35:44.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.924+0000 7fc23cc00640 10 client.24944 handle_command_reply: tid=0 2024-07-29T00:35:44.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 client.24944 shutdown 2024-07-29T00:35:44.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 msgr2=0x55ac367373a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac367373a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc240006010 tx=0x7fc240035650 comp rx=0 tx=0).stop 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 shutdown_connections 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ac367a2fd0 0x55ac367a1ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac3671e760 0x55ac367378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac36713ea0 0x55ac367373a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 --2- 172.21.15.175:0/487983301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac36713580 0x55ac36736e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc23cc00640 0 client.24944 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 >> 172.21.15.175:0/487983301 conn(0x55ac3670fb10 msgr2=0x55ac3672db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 shutdown_connections 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 1 -- 172.21.15.175:0/487983301 wait complete. 2024-07-29T00:35:44.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.925+0000 7fc25c859740 20 client.24944 trim_cache size 0 max 16384 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 1, 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24515, 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 95.285960782000004, 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 89, 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 129.279525767 2024-07-29T00:35:44.929 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.926+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2380a2c10 msgr2=0x7fc2380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.926+0000 7fc253e00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2380a2c10 0x7fc2380a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc240009730 tx=0x7fc2400096c0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.926+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 msgr2=0x7fc25419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.926+0000 7fc253e00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 0x7fc25419c7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc24c005e30 tx=0x7fc24c003f10 comp rx=0 tx=0).stop 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 shutdown_connections 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2541076b0 0x7fc25419c7b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc254102050 0x7fc254198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2380a2c10 0x7fc2380a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 --2- 172.21.15.175:0/4290235589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2541016b0 0x7fc254197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:44.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 >> 172.21.15.175:0/4290235589 conn(0x7fc2540fd3d0 msgr2=0x7fc2540fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:44.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 shutdown_connections 2024-07-29T00:35:44.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:44.927+0000 7fc253e00640 1 -- 172.21.15.175:0/4290235589 wait complete. 2024-07-29T00:35:44.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:44 smithi181 ceph-mon[36706]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-07-29T00:35:45.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:44 smithi204 ceph-mon[37368]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-07-29T00:35:45.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.b status 2024-07-29T00:35:46.372 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:46.755 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:46 smithi175 ceph-mon[32138]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-07-29T00:35:46.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:46 smithi204 ceph-mon[37368]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-07-29T00:35:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:46 smithi181 ceph-mon[36706]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-07-29T00:35:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 -- 172.21.15.175:0/1112151006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 msgr2=0x7fc29c10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 --2- 172.21.15.175:0/1112151006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c10e130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009930 tx=0x7fc28c02a420 comp rx=0 tx=0).stop 2024-07-29T00:35:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 -- 172.21.15.175:0/1112151006 shutdown_connections 2024-07-29T00:35:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 --2- 172.21.15.175:0/1112151006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c108670 0x7fc29c114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 --2- 172.21.15.175:0/1112151006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c107d10 0x7fc29c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 --2- 172.21.15.175:0/1112151006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c10e130 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 -- 172.21.15.175:0/1112151006 >> 172.21.15.175:0/1112151006 conn(0x7fc29c077670 msgr2=0x7fc29c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 -- 172.21.15.175:0/1112151006 shutdown_connections 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.018+0000 7fc2a1800640 1 -- 172.21.15.175:0/1112151006 wait complete. 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.019+0000 7fc2a1800640 1 Processor -- start 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.019+0000 7fc2a1800640 1 -- start start 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.019+0000 7fc2a1800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 0x7fc29c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c108670 0x7fc29c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c1a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1180b0 con 0x7fc29c108670 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a1800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c117f30 con 0x7fc29c107d10 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c118230 con 0x7fc29c10dd50 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c108670 0x7fc29c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 0x7fc29c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 0x7fc29c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35192/0 (socket says 172.21.15.175:35192) 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 -- 172.21.15.175:0/3140812974 learned_addr learned my addr 172.21.15.175:0/3140812974 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c1a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 -- 172.21.15.175:0/3140812974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 msgr2=0x7fc29c1a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c1a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 -- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c108670 msgr2=0x7fc29c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c108670 0x7fc29c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29be00640 1 -- 172.21.15.175:0/3140812974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c1a3510 con 0x7fc29c107d10 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c1a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.020+0000 7fc29b400640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c108670 0x7fc29c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.021+0000 7fc29be00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 0x7fc29c19e560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc28c02a940 tx=0x7fc28c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.021+0000 7fc298c00640 1 -- 172.21.15.175:0/3140812974 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c03f070 con 0x7fc29c107d10 2024-07-29T00:35:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.021+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c1a37a0 con 0x7fc29c107d10 2024-07-29T00:35:47.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.021+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c1ab0e0 con 0x7fc29c107d10 2024-07-29T00:35:47.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.021+0000 7fc298c00640 1 -- 172.21.15.175:0/3140812974 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc28c031c90 con 0x7fc29c107d10 2024-07-29T00:35:47.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.021+0000 7fc298c00640 1 -- 172.21.15.175:0/3140812974 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c049060 con 0x7fc29c107d10 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.024+0000 7fc298c00640 1 -- 172.21.15.175:0/3140812974 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc28c042070 con 0x7fc29c107d10 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 Processor -- start 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 -- start start 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f98b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 0x55c80f996220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f996760 0x55c80f998b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c80f8f1790 con 0x55c80f98b580 2024-07-29T00:35:47.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c80f889880 con 0x55c80f98bea0 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.025+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c80f8ed750 con 0x55c80f996760 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 0x55c80f996220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc2a0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f98b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 0x55c80f996220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35206/0 (socket says 172.21.15.175:35206) 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 -- 172.21.15.175:0/1938922791 learned_addr learned my addr 172.21.15.175:0/1938922791 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29b400640 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f996760 0x55c80f998b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 -- 172.21.15.175:0/1938922791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f996760 msgr2=0x55c80f998b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f996760 0x55c80f998b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 -- 172.21.15.175:0/1938922791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 msgr2=0x55c80f98b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f98b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 -- 172.21.15.175:0/1938922791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80f6632b0 con 0x55c80f98bea0 2024-07-29T00:35:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc298c00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2800a2c90 0x7fc2800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.026+0000 7fc29be00640 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 0x55c80f996220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0f5730 tx=0x7fc28c0f5820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9087ac5cb46d8808 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.027+0000 7fc298c00640 1 -- 172.21.15.175:0/3140812974 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc28c0ef5f0 con 0x7fc29c107d10 2024-07-29T00:35:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.027+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2800a2c90 0x7fc2800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.027+0000 7fc291600640 1 -- 172.21.15.175:0/1938922791 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0b7f10 con 0x55c80f98bea0 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.027+0000 7fc291600640 1 -- 172.21.15.175:0/1938922791 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc28c031e40 con 0x55c80f98bea0 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.027+0000 7fc291600640 1 -- 172.21.15.175:0/1938922791 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0f5200 con 0x55c80f98bea0 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1938922791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 msgr2=0x55c80f996220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2800a2c90 0x7fc2800a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc294005fd0 tx=0x7fc294009770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 0x55c80f996220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0f5730 tx=0x7fc28c0f5820 comp rx=0 tx=0).stop 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1938922791 shutdown_connections 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f996760 0x55c80f998b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f98bea0 0x55c80f996220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1938922791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f98b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1938922791 >> 172.21.15.175:0/1938922791 conn(0x55c80f987b10 msgr2=0x55c80f987ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1938922791 shutdown_connections 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1938922791 wait complete. 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.028+0000 7fc2a59cc740 1 -- start start 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 Processor -- start 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f993600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f993b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f994080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9990f0 con 0x55c80f98b580 2024-07-29T00:35:47.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c80f8ac960 con 0x55c80f996760 2024-07-29T00:35:47.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c80f999270 con 0x55c80f98bea0 2024-07-29T00:35:47.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc2a59cc740 10 client.0 _ll_register_callbacks cb 0x55c80f6007b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.029+0000 7fc29b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f993b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f993b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44264/0 (socket says 172.21.15.175:44264) 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc2a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f994080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 -- 172.21.15.175:0/1345126980 learned_addr learned my addr 172.21.15.175:0/1345126980 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29be00640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f993600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 -- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 msgr2=0x55c80f994080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f994080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 -- 172.21.15.175:0/1345126980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 msgr2=0x55c80f993600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f993600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 -- 172.21.15.175:0/1345126980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80f8bba90 con 0x55c80f98bea0 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f994080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29be00640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f993600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc29b400640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f993b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc288012b10 tx=0x7fc28800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e402a79cc48d44c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.030+0000 7fc286000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:35:47.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc286000640 1 -- 172.21.15.175:0/1345126980 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28800fb40 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc286000640 1 -- 172.21.15.175:0/1345126980 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc28800fce0 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c80f8fed00 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc286000640 1 -- 172.21.15.175:0/1345126980 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc288022070 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc2a59cc740 10 client.24949 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c80f9a4c90 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc286000640 1 -- 172.21.15.175:0/1345126980 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c80f8fed00 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc286000640 1 -- 172.21.15.175:0/1345126980 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc288010430 con 0x55c80f98bea0 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.031+0000 7fc2a59cc740 10 client.24949 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.032+0000 7fc2a59cc740 10 client.24949 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.032+0000 7fc2a59cc740 20 client.24949 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.032+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80f9a72e0 0x55c80f9a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.032+0000 7fc2a59cc740 4 client.24949 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.032+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c80f8f9210 con 0x55c80f9a72e0 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.032+0000 7fc29be00640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80f9a72e0 0x55c80f9a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.033+0000 7fc29be00640 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80f9a72e0 0x55c80f9a8750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1304f8500a810c49 server_cookie=5d8cc1239186a6f7 in_seq=0 out_seq=0 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.033+0000 7fc286000640 10 client.24949 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.033+0000 7fc286000640 1 -- 172.21.15.175:0/1345126980 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c80f8f9210 con 0x55c80f9a72e0 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.033+0000 7fc286000640 10 client.24949 handle_command_reply: tid=0 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.033+0000 7fc2a59cc740 1 client.24949 shutdown 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 msgr2=0x55c80f993b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f993b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc288012b10 tx=0x7fc28800f3c0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 shutdown_connections 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80f9a72e0 0x55c80f9a8750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f994080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f993b40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc286000640 0 client.24949 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:35:47.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1345126980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f993600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 >> 172.21.15.175:0/1345126980 conn(0x55c80f987b10 msgr2=0x55c80f98d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:47.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 shutdown_connections 2024-07-29T00:35:47.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1345126980 wait complete. 2024-07-29T00:35:47.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.034+0000 7fc2a59cc740 20 client.24949 trim_cache size 0 max 16384 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.053+0000 7fc2a59cc740 1 Processor -- start 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 -- start start 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f9a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f98e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f993ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9a6a90 con 0x55c80f98b580 2024-07-29T00:35:47.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9a7160 con 0x55c80f996760 2024-07-29T00:35:47.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9a7830 con 0x55c80f98bea0 2024-07-29T00:35:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f993ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc29be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f9a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f993ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35226/0 (socket says 172.21.15.175:35226) 2024-07-29T00:35:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc2a0e00640 1 -- 172.21.15.175:0/3446542416 learned_addr learned my addr 172.21.15.175:0/3446542416 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.054+0000 7fc29b400640 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f98e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc2a0e00640 1 -- 172.21.15.175:0/3446542416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 msgr2=0x55c80f98e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f98e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc2a0e00640 1 -- 172.21.15.175:0/3446542416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 msgr2=0x55c80f9a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f9a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc2a0e00640 1 -- 172.21.15.175:0/3446542416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80f998e90 con 0x55c80f996760 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f993ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc294011a50 tx=0x7fc294016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ac571ef6685d8b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc290c00640 1 -- 172.21.15.175:0/3446542416 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29402d070 con 0x55c80f996760 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc290c00640 1 -- 172.21.15.175:0/3446542416 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29401fa70 con 0x55c80f996760 2024-07-29T00:35:47.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.055+0000 7fc290c00640 1 -- 172.21.15.175:0/3446542416 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294028b40 con 0x55c80f996760 2024-07-29T00:35:47.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 -- 172.21.15.175:0/3446542416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 msgr2=0x55c80f993ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f993ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc294011a50 tx=0x7fc294016000 comp rx=0 tx=0).stop 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 -- 172.21.15.175:0/3446542416 shutdown_connections 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f993ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98bea0 0x55c80f98e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/3446542416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98b580 0x55c80f9a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 -- 172.21.15.175:0/3446542416 >> 172.21.15.175:0/3446542416 conn(0x55c80f987b10 msgr2=0x55c80f98e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 -- 172.21.15.175:0/3446542416 shutdown_connections 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.056+0000 7fc2a59cc740 1 -- 172.21.15.175:0/3446542416 wait complete. 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.057+0000 7fc2a59cc740 1 -- start start 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.057+0000 7fc2a59cc740 1 Processor -- start 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98b580 0x55c80f9a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98bea0 0x55c80f9a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f9aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9a85d0 con 0x55c80f98bea0 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9a7f00 con 0x55c80f996760 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c80f9a8ca0 con 0x55c80f98b580 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a59cc740 10 client.0 _ll_register_callbacks cb 0x7fc288009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc29b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98bea0 0x55c80f9a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f9aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc29be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98b580 0x55c80f9a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc29b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98bea0 0x55c80f9a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35152/0 (socket says 172.21.15.175:35152) 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc29b400640 1 -- 172.21.15.175:0/1311775546 learned_addr learned my addr 172.21.15.175:0/1311775546 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a0e00640 1 -- 172.21.15.175:0/1311775546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98b580 msgr2=0x55c80f9a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98b580 0x55c80f9a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a0e00640 1 -- 172.21.15.175:0/1311775546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98bea0 msgr2=0x55c80f9a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98bea0 0x55c80f9a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc2a0e00640 1 -- 172.21.15.175:0/1311775546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80fa0fbe0 con 0x55c80f996760 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.058+0000 7fc29be00640 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98b580 0x55c80f9a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc2a0e00640 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f9aa3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc294013cc0 tx=0x7fc294028640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c643266b2c1537c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc286000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc286000640 1 -- 172.21.15.175:0/1311775546 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29402d040 con 0x55c80f996760 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc29401fa70 con 0x55c80f996760 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc286000640 1 -- 172.21.15.175:0/1311775546 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc29401f7d0 con 0x55c80f996760 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc286000640 1 -- 172.21.15.175:0/1311775546 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc294004a40 con 0x55c80f996760 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc2a59cc740 10 client.24962 fetch_fsmap learned FSMap version 28 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.059+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c80fa10ee0 con 0x55c80f996760 2024-07-29T00:35:47.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc286000640 1 -- 172.21.15.175:0/1311775546 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29401fa70 con 0x55c80f996760 2024-07-29T00:35:47.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc286000640 1 -- 172.21.15.175:0/1311775546 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc29403b030 con 0x55c80f996760 2024-07-29T00:35:47.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc2a59cc740 10 client.24962 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:35:47.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc2a59cc740 10 client.24962 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-29T00:35:47.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc2a59cc740 20 client.24962 populate_metadata read hostname 'smithi175' 2024-07-29T00:35:47.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80fa15720 0x55c80fa14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:47.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc2a59cc740 4 client.24962 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "status"}) -- 0x55c80f875f80 con 0x55c80fa15720 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.060+0000 7fc29be00640 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80fa15720 0x55c80fa14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc29be00640 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80fa15720 0x55c80fa14430 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3038bddb798ee8b6 server_cookie=76ef0f9f361ae4d9 in_seq=0 out_seq=0 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc286000640 10 client.24962 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc286000640 1 -- 172.21.15.175:0/1311775546 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c80f875f80 con 0x55c80fa15720 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc286000640 10 client.24962 handle_command_reply: tid=0 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 client.24962 shutdown 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 msgr2=0x55c80f9aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f9aa3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc294013cc0 tx=0x7fc294028640 comp rx=0 tx=0).stop 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 shutdown_connections 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c80fa15720 0x55c80fa14430 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c80f996760 0x55c80f9aa3d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80f98bea0 0x55c80f9a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 --2- 172.21.15.175:0/1311775546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c80f98b580 0x55c80f9a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 >> 172.21.15.175:0/1311775546 conn(0x55c80f987b10 msgr2=0x55c80f991ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.061+0000 7fc286000640 0 client.24962 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 shutdown_connections 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a59cc740 1 -- 172.21.15.175:0/1311775546 wait complete. 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a59cc740 20 client.24962 trim_cache size 0 max 16384 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:35:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 0, 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24539, 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 108.32377974800001, 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 89, 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 124.639772216 2024-07-29T00:35:47.067 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2800a2c90 msgr2=0x7fc2800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2800a2c90 0x7fc2800a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc294005fd0 tx=0x7fc294009770 comp rx=0 tx=0).stop 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 msgr2=0x7fc29c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 0x7fc29c19e560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc28c02a940 tx=0x7fc28c003b90 comp rx=0 tx=0).stop 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 shutdown_connections 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc2800a2c90 0x7fc2800a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc29c10dd50 0x7fc29c1a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29c108670 0x7fc29c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 --2- 172.21.15.175:0/3140812974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c107d10 0x7fc29c19e560 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:47.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.062+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 >> 172.21.15.175:0/3140812974 conn(0x7fc29c077670 msgr2=0x7fc29c078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:47.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.063+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 shutdown_connections 2024-07-29T00:35:47.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:47.063+0000 7fc2a1800640 1 -- 172.21.15.175:0/3140812974 wait complete. 2024-07-29T00:35:48.033 INFO:tasks.cephadm:Setup complete, yielding 2024-07-29T00:35:48.034 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-29T00:35:48.044 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:35:48.044 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph orch status' 2024-07-29T00:35:48.501 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:48.902 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:48 smithi175 ceph-mon[32138]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-07-29T00:35:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:48 smithi181 ceph-mon[36706]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-07-29T00:35:49.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:48 smithi204 ceph-mon[37368]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-07-29T00:35:49.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 -- 172.21.15.175:0/2036048199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 msgr2=0x7f4194102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 --2- 172.21.15.175:0/2036048199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 0x7f4194102da0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4180009900 tx=0x7f418002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 -- 172.21.15.175:0/2036048199 shutdown_connections 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 --2- 172.21.15.175:0/2036048199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41941032e0 0x7f4194109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 --2- 172.21.15.175:0/2036048199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 0x7f4194102da0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 --2- 172.21.15.175:0/2036048199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4194101720 0x7f4194101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 -- 172.21.15.175:0/2036048199 >> 172.21.15.175:0/2036048199 conn(0x7f41940fceb0 msgr2=0x7f41940ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 -- 172.21.15.175:0/2036048199 shutdown_connections 2024-07-29T00:35:49.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.187+0000 7f4193200640 1 -- 172.21.15.175:0/2036048199 wait complete. 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.188+0000 7f4193200640 1 Processor -- start 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.188+0000 7f4193200640 1 -- start start 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4193200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 0x7f41941979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4193200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 0x7f4194197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4193200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41941032e0 0x7f419419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4193200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f419410d3d0 con 0x7f41941032e0 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4193200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f419410d250 con 0x7f4194102920 2024-07-29T00:35:49.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4193200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f419410d550 con 0x7f4194101720 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4191e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 0x7f41941979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4192800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41941032e0 0x7f419419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4191e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 0x7f41941979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57388/0 (socket says 172.21.15.175:57388) 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4191e00640 1 -- 172.21.15.175:0/3199497029 learned_addr learned my addr 172.21.15.175:0/3199497029 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.189+0000 7f4191400640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 0x7f4194197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4191e00640 1 -- 172.21.15.175:0/3199497029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 msgr2=0x7f4194197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4191e00640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 0x7f4194197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4191e00640 1 -- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41941032e0 msgr2=0x7f419419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4191e00640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41941032e0 0x7f419419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4191e00640 1 -- 172.21.15.175:0/3199497029 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419419f620 con 0x7f4194101720 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4191e00640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 0x7f41941979f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f418400e9c0 tx=0x7f418400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418400cd80 con 0x7f4194101720 2024-07-29T00:35:49.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41840035f0 con 0x7f4194101720 2024-07-29T00:35:49.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419419f910 con 0x7f4194101720 2024-07-29T00:35:49.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184010690 con 0x7f4194101720 2024-07-29T00:35:49.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.190+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419419fe20 con 0x7f4194101720 2024-07-29T00:35:49.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.192+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4194101b20 con 0x7f4194101720 2024-07-29T00:35:49.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.194+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f41840037a0 con 0x7f4194101720 2024-07-29T00:35:49.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.195+0000 7f418aa00640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f41780a2c40 0x7f41780a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:49.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.195+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f41840cf610 con 0x7f4194101720 2024-07-29T00:35:49.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.195+0000 7f4191400640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f41780a2c40 0x7f41780a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:49.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.196+0000 7f4191400640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f41780a2c40 0x7f41780a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4180031440 tx=0x7f41800315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:49.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.199+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f418409c390 con 0x7f4194101720 2024-07-29T00:35:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.403+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4194062320 con 0x7f41780a2c40 2024-07-29T00:35:49.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.406+0000 7f418aa00640 1 -- 172.21.15.175:0/3199497029 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4194062320 con 0x7f41780a2c40 2024-07-29T00:35:49.406 INFO:teuthology.orchestra.run.smithi175.stdout:Backend: cephadm 2024-07-29T00:35:49.407 INFO:teuthology.orchestra.run.smithi175.stdout:Available: Yes 2024-07-29T00:35:49.407 INFO:teuthology.orchestra.run.smithi175.stdout:Paused: No 2024-07-29T00:35:49.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f41780a2c40 msgr2=0x7f41780a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:49.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f41780a2c40 0x7f41780a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4180031440 tx=0x7f41800315b0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 msgr2=0x7f41941979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:49.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 0x7f41941979f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f418400e9c0 tx=0x7f418400ee80 comp rx=0 tx=0).stop 2024-07-29T00:35:49.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 shutdown_connections 2024-07-29T00:35:49.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41941032e0 0x7f419419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4194102920 0x7f4194197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f41780a2c40 0x7f41780a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 --2- 172.21.15.175:0/3199497029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4194101720 0x7f41941979f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:49.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.409+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 >> 172.21.15.175:0/3199497029 conn(0x7f41940fceb0 msgr2=0x7f4194105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:49.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.410+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 shutdown_connections 2024-07-29T00:35:49.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:49.410+0000 7f4193200640 1 -- 172.21.15.175:0/3199497029 wait complete. 2024-07-29T00:35:50.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-29T00:35:50.751 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:50.902 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:50 smithi175 ceph-mon[32138]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:50.902 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:50 smithi175 ceph-mon[32138]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-29T00:35:50.902 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:50 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-07-29T00:35:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:50 smithi181 ceph-mon[36706]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:50 smithi181 ceph-mon[36706]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-29T00:35:50.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:50 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-07-29T00:35:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:50 smithi204 ceph-mon[37368]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:51.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:50 smithi204 ceph-mon[37368]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-29T00:35:51.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:50 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-07-29T00:35:51.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 -- 172.21.15.175:0/706086445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90078480 msgr2=0x7fdc90078d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 --2- 172.21.15.175:0/706086445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90078480 0x7fdc90078d20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009ca0 tx=0x7fdc8802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 -- 172.21.15.175:0/706086445 shutdown_connections 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 --2- 172.21.15.175:0/706086445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90078480 0x7fdc90078d20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 --2- 172.21.15.175:0/706086445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90077ae0 0x7fdc90077f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 --2- 172.21.15.175:0/706086445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc90076890 0x7fdc90076c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 -- 172.21.15.175:0/706086445 >> 172.21.15.175:0/706086445 conn(0x7fdc900ff0d0 msgr2=0x7fdc90101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 -- 172.21.15.175:0/706086445 shutdown_connections 2024-07-29T00:35:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.457+0000 7fdc96e00640 1 -- 172.21.15.175:0/706086445 wait complete. 2024-07-29T00:35:51.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.458+0000 7fdc96e00640 1 Processor -- start 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.458+0000 7fdc96e00640 1 -- start start 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 0x7fdc9019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc90077ae0 0x7fdc9019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90078480 0x7fdc901a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90113a50 con 0x7fdc90076890 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901138d0 con 0x7fdc90078480 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90113bd0 con 0x7fdc90077ae0 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc95a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 0x7fdc9019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc95a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 0x7fdc9019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43176/0 (socket says 172.21.15.175:43176) 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc95a00640 1 -- 172.21.15.175:0/2707031910 learned_addr learned my addr 172.21.15.175:0/2707031910 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc95000640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc90077ae0 0x7fdc9019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.459+0000 7fdc96400640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90078480 0x7fdc901a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:51.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc95a00640 1 -- 172.21.15.175:0/2707031910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc90077ae0 msgr2=0x7fdc9019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc95a00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc90077ae0 0x7fdc9019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc95a00640 1 -- 172.21.15.175:0/2707031910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90078480 msgr2=0x7fdc901a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc95a00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90078480 0x7fdc901a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc95a00640 1 -- 172.21.15.175:0/2707031910 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc901a5c00 con 0x7fdc90076890 2024-07-29T00:35:51.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc96400640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90078480 0x7fdc901a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc95a00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 0x7fdc9019dfd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400e9f0 tx=0x7fdc8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8400cd80 con 0x7fdc90076890 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc84010910 con 0x7fdc90076890 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc901a5ef0 con 0x7fdc90076890 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84018990 con 0x7fdc90076890 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.460+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc901a63a0 con 0x7fdc90076890 2024-07-29T00:35:51.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.462+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc901102c0 con 0x7fdc90076890 2024-07-29T00:35:51.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.463+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdc84010ac0 con 0x7fdc90076890 2024-07-29T00:35:51.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.465+0000 7fdc8e000640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdc780a2c30 0x7fdc780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:51.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.465+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8298+0+0 (secure 0 0 0) 0x7fdc840cf3a0 con 0x7fdc90076890 2024-07-29T00:35:51.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.465+0000 7fdc95000640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdc780a2c30 0x7fdc780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:51.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.466+0000 7fdc95000640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdc780a2c30 0x7fdc780a50f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc9019f4f0 tx=0x7fdc7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:51.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.469+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc8409c260 con 0x7fdc90076890 2024-07-29T00:35:51.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.671+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdc90062320 con 0x7fdc780a2c30 2024-07-29T00:35:51.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.706+0000 7fdc8e000640 1 -- 172.21.15.175:0/2707031910 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7fdc90062320 con 0x7fdc780a2c30 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.a smithi175 running (2m) 33s ago 2m 12.7M - 19.1.0-1150-gba82afec 07a7c1195fa9 c4911c99062a 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.b smithi181 running (2m) 59s ago 2m 28.9M - 19.1.0-1150-gba82afec 07a7c1195fa9 84ff2af4a35a 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.c smithi204 running (2m) 59s ago 2m 16.9M - 19.1.0-1150-gba82afec 07a7c1195fa9 48119f7fae9d 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.d smithi175 running (2m) 33s ago 2m 12.8M - 19.1.0-1150-gba82afec 07a7c1195fa9 067d1842d197 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.e smithi181 running (2m) 59s ago 2m 16.6M - 19.1.0-1150-gba82afec 07a7c1195fa9 14b08d6e5b52 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.f smithi204 running (2m) 59s ago 2m 19.8M - 19.1.0-1150-gba82afec 07a7c1195fa9 f0a1c0c1381d 2024-07-29T00:35:51.707 INFO:teuthology.orchestra.run.smithi175.stdout:mds.g smithi175 running (2m) 33s ago 2m 16.7M - 19.1.0-1150-gba82afec 07a7c1195fa9 e048746e0637 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mds.h smithi181 running (2m) 59s ago 2m 19.7M - 19.1.0-1150-gba82afec 07a7c1195fa9 8e376d404a9d 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mds.i smithi204 running (2m) 59s ago 2m 17.7M - 19.1.0-1150-gba82afec 07a7c1195fa9 6b310ef961e9 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mds.j smithi175 running (2m) 33s ago 2m 15.5M - 19.1.0-1150-gba82afec 07a7c1195fa9 ade123c4916a 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mds.k smithi181 running (2m) 59s ago 2m 18.0M - 19.1.0-1150-gba82afec 07a7c1195fa9 c9c81f1f1284 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mds.l smithi204 running (2m) 59s ago 2m 16.6M - 19.1.0-1150-gba82afec 07a7c1195fa9 cce0a62972a3 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mgr.x smithi175 *:9283,8765 running (9m) 33s ago 9m 507M - 19.1.0-1150-gba82afec 07a7c1195fa9 8df856417f2b 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mgr.y smithi181 *:8443,8765 running (7m) 59s ago 7m 442M - 19.1.0-1150-gba82afec 07a7c1195fa9 613bd09f3e12 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mgr.z smithi204 *:8443,8765 running (7m) 59s ago 7m 441M - 19.1.0-1150-gba82afec 07a7c1195fa9 71ba458fb5a2 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mon.a smithi175 running (9m) 33s ago 9m 47.1M 2048M 19.1.0-1150-gba82afec 07a7c1195fa9 50d2691b7e35 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mon.b smithi181 running (8m) 59s ago 8m 44.9M 2048M 19.1.0-1150-gba82afec 07a7c1195fa9 6ebc6a9cf208 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:mon.c smithi204 running (8m) 59s ago 8m 44.9M 2048M 19.1.0-1150-gba82afec 07a7c1195fa9 0a892ddf45e6 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:osd.0 smithi175 running (7m) 33s ago 7m 55.9M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 fc26bc4272fd 2024-07-29T00:35:51.708 INFO:teuthology.orchestra.run.smithi175.stdout:osd.1 smithi181 running (7m) 59s ago 7m 45.8M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 201e808b1afe 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.2 smithi204 running (6m) 59s ago 6m 46.9M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 220f060e895f 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.3 smithi175 running (6m) 33s ago 6m 52.7M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 c2cbfa08dc0e 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.4 smithi181 running (6m) 59s ago 6m 43.0M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 c58385a8f539 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.5 smithi204 running (5m) 59s ago 5m 53.3M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 ac54ffbaaabd 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.6 smithi175 running (5m) 33s ago 5m 41.1M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 249bc4729b24 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.7 smithi181 running (4m) 59s ago 4m 42.0M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 dd035fa9de7d 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.8 smithi204 running (4m) 59s ago 4m 53.7M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 db0391529679 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.9 smithi175 running (3m) 33s ago 3m 40.7M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 6f1cd63a744d 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.10 smithi181 running (3m) 59s ago 3m 42.4M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 705d762df384 2024-07-29T00:35:51.709 INFO:teuthology.orchestra.run.smithi175.stdout:osd.11 smithi204 running (3m) 59s ago 3m 56.2M 4272M 19.1.0-1150-gba82afec 07a7c1195fa9 203cedcef0dd 2024-07-29T00:35:51.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdc780a2c30 msgr2=0x7fdc780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:51.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdc780a2c30 0x7fdc780a50f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc9019f4f0 tx=0x7fdc7c005f90 comp rx=0 tx=0).stop 2024-07-29T00:35:51.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 msgr2=0x7fdc9019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:51.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 0x7fdc9019dfd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400e9f0 tx=0x7fdc8400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 shutdown_connections 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc90078480 0x7fdc901a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fdc780a2c30 0x7fdc780a50f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc90077ae0 0x7fdc9019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 --2- 172.21.15.175:0/2707031910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc90076890 0x7fdc9019dfd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.709+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 >> 172.21.15.175:0/2707031910 conn(0x7fdc900ff0d0 msgr2=0x7fdc90100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.710+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 shutdown_connections 2024-07-29T00:35:51.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:51.710+0000 7fdc96e00640 1 -- 172.21.15.175:0/2707031910 wait complete. 2024-07-29T00:35:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:51 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-07-29T00:35:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:51 smithi181 ceph-mon[36706]: osdmap e90: 12 total, 12 up, 12 in 2024-07-29T00:35:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:51 smithi181 ceph-mon[36706]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-07-29T00:35:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:51 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-07-29T00:35:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:51 smithi175 ceph-mon[32138]: osdmap e90: 12 total, 12 up, 12 in 2024-07-29T00:35:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:51 smithi175 ceph-mon[32138]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-07-29T00:35:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:51 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-07-29T00:35:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:51 smithi204 ceph-mon[37368]: osdmap e90: 12 total, 12 up, 12 in 2024-07-29T00:35:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:51 smithi204 ceph-mon[37368]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-07-29T00:35:52.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-29T00:35:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:52 smithi181 ceph-mon[36706]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-07-29T00:35:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:52 smithi181 ceph-mon[36706]: osdmap e91: 12 total, 12 up, 12 in 2024-07-29T00:35:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:52 smithi181 ceph-mon[36706]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:53.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:52 smithi175 ceph-mon[32138]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-07-29T00:35:53.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:52 smithi175 ceph-mon[32138]: osdmap e91: 12 total, 12 up, 12 in 2024-07-29T00:35:53.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:52 smithi175 ceph-mon[32138]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:52 smithi204 ceph-mon[37368]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-07-29T00:35:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:52 smithi204 ceph-mon[37368]: osdmap e91: 12 total, 12 up, 12 in 2024-07-29T00:35:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:52 smithi204 ceph-mon[37368]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:53.254 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:53.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 -- 172.21.15.175:0/3623312058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8101480 msgr2=0x7ff1f8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:53.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 --2- 172.21.15.175:0/3623312058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8101480 0x7ff1f8101930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009be0 tx=0x7ff1f002a6c0 comp rx=0 tx=0).stop 2024-07-29T00:35:53.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 -- 172.21.15.175:0/3623312058 shutdown_connections 2024-07-29T00:35:53.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 --2- 172.21.15.175:0/3623312058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8101480 0x7ff1f8101930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:53.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 --2- 172.21.15.175:0/3623312058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1f8100ae0 0x7ff1f8100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:53.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 --2- 172.21.15.175:0/3623312058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f81001a0 0x7ff1f81005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:53.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 -- 172.21.15.175:0/3623312058 >> 172.21.15.175:0/3623312058 conn(0x7ff1f80fbf70 msgr2=0x7ff1f80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:53.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 -- 172.21.15.175:0/3623312058 shutdown_connections 2024-07-29T00:35:53.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.935+0000 7ff1ff200640 1 -- 172.21.15.175:0/3623312058 wait complete. 2024-07-29T00:35:53.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 Processor -- start 2024-07-29T00:35:53.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 -- start start 2024-07-29T00:35:53.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1f81001a0 0x7ff1f8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:53.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 0x7ff1f819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f8101480 0x7ff1f81a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f810f730 con 0x7ff1f8101480 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f810f5b0 con 0x7ff1f8100ae0 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.936+0000 7ff1ff200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f810f8b0 con 0x7ff1f81001a0 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 0x7ff1f819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 0x7ff1f819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53568/0 (socket says 172.21.15.175:53568) 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 -- 172.21.15.175:0/4243036068 learned_addr learned my addr 172.21.15.175:0/4243036068 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7e00640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1f81001a0 0x7ff1f8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1fd200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f8101480 0x7ff1f81a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:53.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 -- 172.21.15.175:0/4243036068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1f81001a0 msgr2=0x7ff1f8199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1f81001a0 0x7ff1f8199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 -- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f8101480 msgr2=0x7ff1f81a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f8101480 0x7ff1f81a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.937+0000 7ff1f7400640 1 -- 172.21.15.175:0/4243036068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a19c0 con 0x7ff1f8100ae0 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.938+0000 7ff1f7400640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 0x7ff1f819a2e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e9f0 tx=0x7ff1e400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.938+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400cd80 con 0x7ff1f8100ae0 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.938+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e4005500 con 0x7ff1f8100ae0 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.938+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a1cb0 con 0x7ff1f8100ae0 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.938+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4010690 con 0x7ff1f8100ae0 2024-07-29T00:35:53.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.938+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a21a0 con 0x7ff1f8100ae0 2024-07-29T00:35:53.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.939+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f810aef0 con 0x7ff1f8100ae0 2024-07-29T00:35:53.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.941+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff1e40046c0 con 0x7ff1f8100ae0 2024-07-29T00:35:53.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.943+0000 7ff1f4c00640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1dc0a2c00 0x7ff1dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:53.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.943+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7ff1e40cf450 con 0x7ff1f8100ae0 2024-07-29T00:35:53.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.943+0000 7ff1f7e00640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1dc0a2c00 0x7ff1dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:53.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.944+0000 7ff1f7e00640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1dc0a2c00 0x7ff1dc0a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80096c0 tx=0x7ff1e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:53.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:53.945+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff1e409c310 con 0x7ff1f8100ae0 2024-07-29T00:35:54.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.144+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff1f8062320 con 0x7ff1dc0a2c00 2024-07-29T00:35:54.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.154+0000 7ff1f4c00640 1 -- 172.21.15.175:0/4243036068 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff1f8062320 con 0x7ff1dc0a2c00 2024-07-29T00:35:54.153 INFO:teuthology.orchestra.run.smithi175.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-29T00:35:54.153 INFO:teuthology.orchestra.run.smithi175.stdout:mds.all 12/12 61s ago 2m smithi175=a;smithi175=d;smithi175=g;smithi175=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-07-29T00:35:54.153 INFO:teuthology.orchestra.run.smithi175.stdout:mgr 3/3 61s ago 8m smithi175=x;smithi181=y;smithi204=z;count:3 2024-07-29T00:35:54.153 INFO:teuthology.orchestra.run.smithi175.stdout:mon 3/3 61s ago 8m smithi175:172.21.15.175=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c;count:3 2024-07-29T00:35:54.154 INFO:teuthology.orchestra.run.smithi175.stdout:osd 12 61s ago - 2024-07-29T00:35:54.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.156+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1dc0a2c00 msgr2=0x7ff1dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:54.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.156+0000 7ff1ff200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1dc0a2c00 0x7ff1dc0a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80096c0 tx=0x7ff1e8009260 comp rx=0 tx=0).stop 2024-07-29T00:35:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.156+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 msgr2=0x7ff1f819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.156+0000 7ff1ff200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 0x7ff1f819a2e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e9f0 tx=0x7ff1e400edf0 comp rx=0 tx=0).stop 2024-07-29T00:35:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 shutdown_connections 2024-07-29T00:35:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f8101480 0x7ff1f81a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f8100ae0 0x7ff1f819a2e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7ff1dc0a2c00 0x7ff1dc0a50c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:54.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 --2- 172.21.15.175:0/4243036068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1f81001a0 0x7ff1f8199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:54.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 >> 172.21.15.175:0/4243036068 conn(0x7ff1f80fbf70 msgr2=0x7ff1f80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:54.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 shutdown_connections 2024-07-29T00:35:54.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:54.157+0000 7ff1ff200640 1 -- 172.21.15.175:0/4243036068 wait complete. 2024-07-29T00:35:54.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:54 smithi181 ceph-mon[36706]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2024-07-29T00:35:55.006 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:54 smithi175 ceph-mon[32138]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2024-07-29T00:35:55.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-29T00:35:55.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:54 smithi204 ceph-mon[37368]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 12 op/s 2024-07-29T00:35:55.525 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:55.934 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:55 smithi175 ceph-mon[32138]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:55.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:55 smithi181 ceph-mon[36706]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:56.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:55 smithi204 ceph-mon[37368]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 -- 172.21.15.175:0/3750443636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b4107d10 msgr2=0x7f87b4108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 --2- 172.21.15.175:0/3750443636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b4107d10 0x7f87b4108190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009900 tx=0x7f87a002a480 comp rx=0 tx=0).stop 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 -- 172.21.15.175:0/3750443636 shutdown_connections 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 --2- 172.21.15.175:0/3750443636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b41086d0 0x7f87b4110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 --2- 172.21.15.175:0/3750443636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b4107d10 0x7f87b4108190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 --2- 172.21.15.175:0/3750443636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4109440 0x7f87b4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 -- 172.21.15.175:0/3750443636 >> 172.21.15.175:0/3750443636 conn(0x7f87b40770c0 msgr2=0x7f87b4079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:56.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 -- 172.21.15.175:0/3750443636 shutdown_connections 2024-07-29T00:35:56.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.187+0000 7f87bc000640 1 -- 172.21.15.175:0/3750443636 wait complete. 2024-07-29T00:35:56.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 Processor -- start 2024-07-29T00:35:56.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 -- start start 2024-07-29T00:35:56.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 0x7f87b419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:56.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b41086d0 0x7f87b419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:56.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4109440 0x7f87b41a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:56.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113be0 con 0x7f87b41086d0 2024-07-29T00:35:56.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113a60 con 0x7f87b4107d10 2024-07-29T00:35:56.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.188+0000 7f87bc000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113d60 con 0x7f87b4109440 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 0x7f87b419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b41086d0 0x7f87b419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 0x7f87b419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53592/0 (socket says 172.21.15.175:53592) 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 -- 172.21.15.175:0/1952598095 learned_addr learned my addr 172.21.15.175:0/1952598095 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87ba000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4109440 0x7f87b41a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 -- 172.21.15.175:0/1952598095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4109440 msgr2=0x7f87b41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4109440 0x7f87b41a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 -- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b41086d0 msgr2=0x7f87b419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b41086d0 0x7f87b419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 -- 172.21.15.175:0/1952598095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b41a5e50 con 0x7f87b4107d10 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.189+0000 7f87b9600640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 0x7f87b419e220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:56.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.190+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a800cd80 con 0x7f87b4107d10 2024-07-29T00:35:56.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.190+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a8010910 con 0x7f87b4107d10 2024-07-29T00:35:56.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.190+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8018990 con 0x7f87b4107d10 2024-07-29T00:35:56.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.190+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b41a6140 con 0x7f87b4107d10 2024-07-29T00:35:56.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.190+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a6600 con 0x7f87b4107d10 2024-07-29T00:35:56.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.191+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b410f2a0 con 0x7f87b4107d10 2024-07-29T00:35:56.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.193+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f87a8010ac0 con 0x7f87b4107d10 2024-07-29T00:35:56.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.195+0000 7f87b2000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f879c0a2c30 0x7f879c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:56.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.195+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f87a80cf730 con 0x7f87b4107d10 2024-07-29T00:35:56.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.195+0000 7f87b8c00640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f879c0a2c30 0x7f879c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:56.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.196+0000 7f87b8c00640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f879c0a2c30 0x7f879c0a50f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87a002aa70 tx=0x7f87a00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:56.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.197+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f87a809bfa0 con 0x7f87b4107d10 2024-07-29T00:35:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.392+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f87b4062320 con 0x7f879c0a2c30 2024-07-29T00:35:56.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.395+0000 7f87b2000640 1 -- 172.21.15.175:0/1952598095 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f87b4062320 con 0x7f879c0a2c30 2024-07-29T00:35:56.395 INFO:teuthology.orchestra.run.smithi175.stdout:HOST ADDR LABELS STATUS 2024-07-29T00:35:56.395 INFO:teuthology.orchestra.run.smithi175.stdout:smithi175 172.21.15.175 2024-07-29T00:35:56.395 INFO:teuthology.orchestra.run.smithi175.stdout:smithi181 172.21.15.181 2024-07-29T00:35:56.395 INFO:teuthology.orchestra.run.smithi175.stdout:smithi204 172.21.15.204 2024-07-29T00:35:56.395 INFO:teuthology.orchestra.run.smithi175.stdout:3 hosts in cluster 2024-07-29T00:35:56.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f879c0a2c30 msgr2=0x7f879c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:56.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f879c0a2c30 0x7f879c0a50f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87a002aa70 tx=0x7f87a00315b0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 msgr2=0x7f87b419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:56.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 0x7f87b419e220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 shutdown_connections 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f879c0a2c30 0x7f879c0a50f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4109440 0x7f87b41a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87b41086d0 0x7f87b419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 --2- 172.21.15.175:0/1952598095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4107d10 0x7f87b419e220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.398+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 >> 172.21.15.175:0/1952598095 conn(0x7f87b40770c0 msgr2=0x7f87b410c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.399+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 shutdown_connections 2024-07-29T00:35:56.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:56.399+0000 7f87bc000640 1 -- 172.21.15.175:0/1952598095 wait complete. 2024-07-29T00:35:56.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:56 smithi181 ceph-mon[36706]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:35:57.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:56 smithi175 ceph-mon[32138]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:35:57.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:56 smithi204 ceph-mon[37368]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:35:57.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-29T00:35:57.690 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:57 smithi181 ceph-mon[36706]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:58.027 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:57 smithi175 ceph-mon[32138]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:57 smithi204 ceph-mon[37368]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 -- 172.21.15.175:0/1138304432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 msgr2=0x7fc490108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 --2- 172.21.15.175:0/1138304432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc490108190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc478009860 tx=0x7fc47802a3d0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 -- 172.21.15.175:0/1138304432 shutdown_connections 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 --2- 172.21.15.175:0/1138304432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 0x7fc490110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 --2- 172.21.15.175:0/1138304432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc490108190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 --2- 172.21.15.175:0/1138304432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc490109440 0x7fc490109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 -- 172.21.15.175:0/1138304432 >> 172.21.15.175:0/1138304432 conn(0x7fc4900770c0 msgr2=0x7fc490079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:58.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.382+0000 7fc494a00640 1 -- 172.21.15.175:0/1138304432 shutdown_connections 2024-07-29T00:35:58.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.383+0000 7fc494a00640 1 -- 172.21.15.175:0/1138304432 wait complete. 2024-07-29T00:35:58.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 Processor -- start 2024-07-29T00:35:58.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 -- start start 2024-07-29T00:35:58.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc49019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:58.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 0x7fc49019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc490109440 0x7fc4901a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc490113b90 con 0x7fc490109440 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc490113a10 con 0x7fc4901086d0 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.384+0000 7fc494a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc490113d10 con 0x7fc490107d10 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc485400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 0x7fc49019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc49019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc49019e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57442/0 (socket says 172.21.15.175:57442) 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc485400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 0x7fc49019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53604/0 (socket says 172.21.15.175:53604) 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 -- 172.21.15.175:0/3358154102 learned_addr learned my addr 172.21.15.175:0/3358154102 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48e800640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc490109440 0x7fc4901a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 -- 172.21.15.175:0/3358154102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 msgr2=0x7fc49019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:58.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 0x7fc49019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 -- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc490109440 msgr2=0x7fc4901a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:58.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc490109440 0x7fc4901a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.385+0000 7fc48de00640 1 -- 172.21.15.175:0/3358154102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4901a5e50 con 0x7fc490107d10 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.386+0000 7fc48de00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc49019e220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e9c0 tx=0x7fc48000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.386+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48000cde0 con 0x7fc490107d10 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.386+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4800035f0 con 0x7fc490107d10 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.386+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480010690 con 0x7fc490107d10 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.386+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4901a6140 con 0x7fc490107d10 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.386+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4901a6600 con 0x7fc490107d10 2024-07-29T00:35:58.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.387+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49010f440 con 0x7fc490107d10 2024-07-29T00:35:58.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.389+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc4800037a0 con 0x7fc490107d10 2024-07-29T00:35:58.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.391+0000 7fc487400640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4700a2c30 0x7fc4700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:35:58.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.391+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7fc4800cf310 con 0x7fc490107d10 2024-07-29T00:35:58.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.391+0000 7fc485400640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4700a2c30 0x7fc4700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:35:58.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.392+0000 7fc485400640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4700a2c30 0x7fc4700a50f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4780309e0 tx=0x7fc478005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:35:58.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.395+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc48009c1d0 con 0x7fc490107d10 2024-07-29T00:35:58.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.591+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 --> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc490062320 con 0x7fc4700a2c30 2024-07-29T00:35:58.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.596+0000 7fc487400640 1 -- 172.21.15.175:0/3358154102 <== mgr.14150 v2:172.21.15.175:6800/3246788131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc490062320 con 0x7fc4700a2c30 2024-07-29T00:35:58.595 INFO:teuthology.orchestra.run.smithi175.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-29T00:35:58.595 INFO:teuthology.orchestra.run.smithi175.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-07-29T00:35:58.595 INFO:teuthology.orchestra.run.smithi175.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-07-29T00:35:58.595 INFO:teuthology.orchestra.run.smithi175.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-07-29T00:35:58.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4700a2c30 msgr2=0x7fc4700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:58.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4700a2c30 0x7fc4700a50f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4780309e0 tx=0x7fc478005d50 comp rx=0 tx=0).stop 2024-07-29T00:35:58.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 msgr2=0x7fc49019e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc49019e220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e9c0 tx=0x7fc48000ee80 comp rx=0 tx=0).stop 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 shutdown_connections 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc490109440 0x7fc4901a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4901086d0 0x7fc49019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fc4700a2c30 0x7fc4700a50f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 --2- 172.21.15.175:0/3358154102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490107d10 0x7fc49019e220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 >> 172.21.15.175:0/3358154102 conn(0x7fc4900770c0 msgr2=0x7fc49010c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 shutdown_connections 2024-07-29T00:35:58.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:35:58.599+0000 7fc494a00640 1 -- 172.21.15.175:0/3358154102 wait complete. 2024-07-29T00:35:58.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:58 smithi181 ceph-mon[36706]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:35:59.018 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:58 smithi175 ceph-mon[32138]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:35:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:58 smithi204 ceph-mon[37368]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:35:59.481 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-29T00:35:59.493 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:35:59.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-29T00:35:59.949 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:35:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:35:59 smithi181 ceph-mon[36706]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:35:59 smithi175 ceph-mon[32138]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:35:59 smithi204 ceph-mon[37368]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 -- 172.21.15.175:0/3764870730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 msgr2=0x7f25f810bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 --2- 172.21.15.175:0/3764870730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f810bf40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009ca0 tx=0x7f25f002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 -- 172.21.15.175:0/3764870730 shutdown_connections 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 --2- 172.21.15.175:0/3764870730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f810bf40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 --2- 172.21.15.175:0/3764870730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f8104cf0 0x7f25f8105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 --2- 172.21.15.175:0/3764870730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8103af0 0x7f25f8103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 -- 172.21.15.175:0/3764870730 >> 172.21.15.175:0/3764870730 conn(0x7f25f80ff2a0 msgr2=0x7f25f81016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 -- 172.21.15.175:0/3764870730 shutdown_connections 2024-07-29T00:36:00.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.638+0000 7f25fda00640 1 -- 172.21.15.175:0/3764870730 wait complete. 2024-07-29T00:36:00.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.639+0000 7f25fda00640 1 Processor -- start 2024-07-29T00:36:00.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 -- start start 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f8103af0 0x7f25f8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8104cf0 0x7f25f819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f81a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810f7a0 con 0x7f25f81056b0 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810f620 con 0x7f25f8104cf0 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25fda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810f920 con 0x7f25f8103af0 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25f7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f81a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25f6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f8103af0 0x7f25f8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25f7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f81a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57588/0 (socket says 172.21.15.175:57588) 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25f7400640 1 -- 172.21.15.175:0/377092192 learned_addr learned my addr 172.21.15.175:0/377092192 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:00.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.640+0000 7f25ee000640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8104cf0 0x7f25f819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:00.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25f7400640 1 -- 172.21.15.175:0/377092192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f8103af0 msgr2=0x7f25f8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:00.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25f7400640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f8103af0 0x7f25f8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25f7400640 1 -- 172.21.15.175:0/377092192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8104cf0 msgr2=0x7f25f819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25f7400640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8104cf0 0x7f25f819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25f7400640 1 -- 172.21.15.175:0/377092192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f81a1970 con 0x7f25f81056b0 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25ee000640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8104cf0 0x7f25f819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25f7400640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f81a1300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f25f002f540 tx=0x7f25f0031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0003b70 con 0x7f25f81056b0 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f0003d10 con 0x7f25f81056b0 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.641+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f002fb50 con 0x7f25f81056b0 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.642+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f81a1c00 con 0x7f25f81056b0 2024-07-29T00:36:00.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.642+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81a20f0 con 0x7f25f81056b0 2024-07-29T00:36:00.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.643+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f810ae20 con 0x7f25f81056b0 2024-07-29T00:36:00.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.645+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f25f002fcf0 con 0x7f25f81056b0 2024-07-29T00:36:00.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.646+0000 7f25efe00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25dc0a2c30 0x7f25dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:00.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.646+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f25f00f03f0 con 0x7f25f81056b0 2024-07-29T00:36:00.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.646+0000 7f25f6a00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25dc0a2c30 0x7f25dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:00.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.647+0000 7f25f6a00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25dc0a2c30 0x7f25dc0a50f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25f8104b50 tx=0x7f25e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:00.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.650+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f25f00bd3d0 con 0x7f25f81056b0 2024-07-29T00:36:00.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.880+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f25f8062b80 con 0x7f25f81056b0 2024-07-29T00:36:00.881 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:00 smithi175 ceph-mon[32138]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 114 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:00.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.881+0000 7f25efe00640 1 -- 172.21.15.175:0/377092192 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3142 (secure 0 0 0) 0x7f25f00bccd0 con 0x7f25f81056b0 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:e28 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:btime 2024-07-29T00:34:15:679894+0000 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:legacy client fscid: 1 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:Filesystem 'cephfs' (1) 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:fs_name cephfs 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:epoch 28 2024-07-29T00:36:00.883 INFO:teuthology.orchestra.run.smithi175.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:created 2024-07-29T00:33:58.041883+0000 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:modified 2024-07-29T00:34:14.951643+0000 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:tableserver 0 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:root 0 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:session_timeout 300 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:session_autoclose 300 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:max_file_size 1099511627776 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:max_xattr_size 65536 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:required_client_features {} 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:last_failure 0 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:last_failure_osd_epoch 0 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-29T00:36:00.884 INFO:teuthology.orchestra.run.smithi175.stdout:max_mds 5 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:in 0,1,2,3,4 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:up {0=24539,1=24515,2=24503,3=24499,4=24484} 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:failed 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:damaged 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:stopped 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:data_pools [3] 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:metadata_pool 2 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:inline_data disabled 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:balancer 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:bal_rank_mask -1 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:standby_count_wanted 0 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:qdb_cluster leader: 24539 members: 24484,24499,24503,24515,24539 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b{0:24539} state up:active seq 6 addr [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.885 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.f{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/1977765890,v1:172.21.15.204:6835/1977765890] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.g{1:24515} state up:active seq 10 addr [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.e{1:24476} state up:standby-replay seq 1 addr [v2:172.21.15.181:6834/568693704,v1:172.21.15.181:6835/568693704] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.h{2:24503} state up:active seq 12 addr [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.j{2:24452} state up:standby-replay seq 1 addr [v2:172.21.15.175:6834/284852041,v1:172.21.15.175:6835/284852041] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.i{3:24499} state up:active seq 12 addr [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.k{3:24440} state up:standby-replay seq 1 addr [v2:172.21.15.181:6832/4123110499,v1:172.21.15.181:6833/4123110499] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c{4:24484} state up:active seq 13 addr [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.l{4:24436} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/1301583270,v1:172.21.15.204:6833/1301583270] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:Standby daemons: 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:00.886 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d{-1:14394} state up:standby seq 1 addr [v2:172.21.15.175:6836/1855710459,v1:172.21.15.175:6837/1855710459] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.887 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a{-1:14436} state up:standby seq 1 addr [v2:172.21.15.175:6840/1735377936,v1:172.21.15.175:6841/1735377936] compat {c=[1],r=[1],i=[1fff]}] 2024-07-29T00:36:00.887 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:36:00.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.884+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25dc0a2c30 msgr2=0x7f25dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.884+0000 7f25fda00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25dc0a2c30 0x7f25dc0a50f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25f8104b50 tx=0x7f25e8005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.884+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 msgr2=0x7f25f81a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.884+0000 7f25fda00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f81a1300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f25f002f540 tx=0x7f25f0031750 comp rx=0 tx=0).stop 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 shutdown_connections 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f81056b0 0x7f25f81a1300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f8104cf0 0x7f25f819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f25dc0a2c30 0x7f25dc0a50f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 --2- 172.21.15.175:0/377092192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f8103af0 0x7f25f8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 >> 172.21.15.175:0/377092192 conn(0x7f25f80ff2a0 msgr2=0x7f25f8107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 shutdown_connections 2024-07-29T00:36:00.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:00.885+0000 7f25fda00640 1 -- 172.21.15.175:0/377092192 wait complete. 2024-07-29T00:36:00.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:00 smithi181 ceph-mon[36706]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 114 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:01.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:00 smithi204 ceph-mon[37368]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 114 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:01.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- bash -c 'ceph osd dump' 2024-07-29T00:36:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:01 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/377092192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-29T00:36:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:01 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/377092192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-29T00:36:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:01 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/377092192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-29T00:36:02.294 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 -- 172.21.15.175:0/761110794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b3810ded0 msgr2=0x7f9b381102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/761110794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b3810ded0 0x7f9b381102c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009be0 tx=0x7f9b3002a6c0 comp rx=0 tx=0).stop 2024-07-29T00:36:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 -- 172.21.15.175:0/761110794 shutdown_connections 2024-07-29T00:36:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/761110794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b3810ded0 0x7f9b381102c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/761110794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 0x7f9b38074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/761110794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b38075f10 0x7f9b38074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:02.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 -- 172.21.15.175:0/761110794 >> 172.21.15.175:0/761110794 conn(0x7f9b380fd080 msgr2=0x7f9b380ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:02.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 -- 172.21.15.175:0/761110794 shutdown_connections 2024-07-29T00:36:02.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.976+0000 7f9b3ec00640 1 -- 172.21.15.175:0/761110794 wait complete. 2024-07-29T00:36:02.978 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:02 smithi175 ceph-mon[32138]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:02.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.977+0000 7f9b3ec00640 1 Processor -- start 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.977+0000 7f9b3ec00640 1 -- start start 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 0x7f9b3819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 0x7f9b3819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b3810ded0 0x7f9b381a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38113b90 con 0x7f9b38075f10 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38113a10 con 0x7f9b38074940 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38113d10 con 0x7f9b3810ded0 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b37400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 0x7f9b3819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b37400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 0x7f9b3819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57598/0 (socket says 172.21.15.175:57598) 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b37400640 1 -- 172.21.15.175:0/1940505691 learned_addr learned my addr 172.21.15.175:0/1940505691 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b37e00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 0x7f9b3819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:02.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.978+0000 7f9b3cc00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b3810ded0 0x7f9b381a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37400640 1 -- 172.21.15.175:0/1940505691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b3810ded0 msgr2=0x7f9b381a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37400640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b3810ded0 0x7f9b381a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37400640 1 -- 172.21.15.175:0/1940505691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 msgr2=0x7f9b3819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37400640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 0x7f9b3819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37400640 1 -- 172.21.15.175:0/1940505691 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381a5dc0 con 0x7f9b38075f10 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b3cc00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b3810ded0 0x7f9b381a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37e00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 0x7f9b3819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b37400640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 0x7f9b3819e6d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000e9c0 tx=0x7f9b2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000cde0 con 0x7f9b38075f10 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b200035f0 con 0x7f9b38075f10 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20010690 con 0x7f9b38075f10 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a60b0 con 0x7f9b38075f10 2024-07-29T00:36:02.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.979+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a65a0 con 0x7f9b38075f10 2024-07-29T00:36:02.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.981+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b381040a0 con 0x7f9b38075f10 2024-07-29T00:36:02.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.982+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9b200037a0 con 0x7f9b38075f10 2024-07-29T00:36:02.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.986+0000 7f9b34c00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9b1c0a2c30 0x7f9b1c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:02.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.986+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f9b200cf3c0 con 0x7f9b38075f10 2024-07-29T00:36:02.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.986+0000 7f9b37e00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9b1c0a2c30 0x7f9b1c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:02.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.987+0000 7f9b37e00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9b1c0a2c30 0x7f9b1c0a50f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b380762f0 tx=0x7f9b28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:02.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:02.989+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9b2009c280 con 0x7f9b38075f10 2024-07-29T00:36:02.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:02 smithi181 ceph-mon[36706]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:03.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:02 smithi204 ceph-mon[37368]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:03.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.179+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9b3802d5a0 con 0x7f9b38075f10 2024-07-29T00:36:03.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.182+0000 7f9b34c00640 1 -- 172.21.15.175:0/1940505691 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5380 (secure 0 0 0) 0x7f9b2009bb80 con 0x7f9b38075f10 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:epoch 91 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:fsid f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:created 2024-07-29T00:25:53.732185+0000 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:modified 2024-07-29T00:35:51.651385+0000 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:crush_version 26 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:full_ratio 0.95 2024-07-29T00:36:03.183 INFO:teuthology.orchestra.run.smithi175.stdout:backfillfull_ratio 0.9 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:nearfull_ratio 0.85 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:require_min_compat_client luminous 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:min_compat_client luminous 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:require_osd_release squid 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:stretch_mode_enabled false 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:max_osd 12 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6802/311033878,v1:172.21.15.175:6803/311033878] [v2:172.21.15.175:6804/311033878,v1:172.21.15.175:6805/311033878] exists,up 563e75c0-0a80-4c6d-948c-6a7d756fa193 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/2146535950,v1:172.21.15.181:6801/2146535950] [v2:172.21.15.181:6802/2146535950,v1:172.21.15.181:6803/2146535950] exists,up f644309b-2ac6-416d-bb60-a115681712b5 2024-07-29T00:36:03.184 INFO:teuthology.orchestra.run.smithi175.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/3684418621,v1:172.21.15.204:6801/3684418621] [v2:172.21.15.204:6802/3684418621,v1:172.21.15.204:6803/3684418621] exists,up 77a7c751-1c7b-45ae-87fc-91eb99bdced4 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6810/2700469757,v1:172.21.15.175:6811/2700469757] [v2:172.21.15.175:6812/2700469757,v1:172.21.15.175:6813/2700469757] exists,up ec28d659-e23c-4abf-bc28-9dae71c447c5 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/1605621704,v1:172.21.15.181:6809/1605621704] [v2:172.21.15.181:6810/1605621704,v1:172.21.15.181:6811/1605621704] exists,up 708e65da-400e-40c5-8991-52201ddf57b1 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.5 up in weight 1 up_from 38 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/377725660,v1:172.21.15.204:6809/377725660] [v2:172.21.15.204:6810/377725660,v1:172.21.15.204:6811/377725660] exists,up f6258db1-4eca-4859-bca9-9291b0c40d4e 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6818/3660975891,v1:172.21.15.175:6819/3660975891] [v2:172.21.15.175:6820/3660975891,v1:172.21.15.175:6821/3660975891] exists,up 58d77f5c-49a7-4727-99a5-c86d7bf62e86 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.7 up in weight 1 up_from 51 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/591220240,v1:172.21.15.181:6817/591220240] [v2:172.21.15.181:6818/591220240,v1:172.21.15.181:6819/591220240] exists,up 393b8cdf-a78f-4d38-b132-f45237f8f088 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.8 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/1532855581,v1:172.21.15.204:6817/1532855581] [v2:172.21.15.204:6818/1532855581,v1:172.21.15.204:6819/1532855581] exists,up 6f7f413a-1907-4fd0-8482-293e00ba2c96 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.9 up in weight 1 up_from 66 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6826/2466426097,v1:172.21.15.175:6827/2466426097] [v2:172.21.15.175:6828/2466426097,v1:172.21.15.175:6829/2466426097] exists,up 56c85017-3133-4b51-90da-455c226a0d6f 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.10 up in weight 1 up_from 71 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/2035344536,v1:172.21.15.181:6825/2035344536] [v2:172.21.15.181:6826/2035344536,v1:172.21.15.181:6827/2035344536] exists,up d33f82ef-0e35-45ca-ba0c-27da92a81a15 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:osd.11 up in weight 1 up_from 79 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/945517137,v1:172.21.15.204:6825/945517137] [v2:172.21.15.204:6826/945517137,v1:172.21.15.204:6827/945517137] exists,up a28bd9a6-023c-4dd3-8acd-441f4bc106b2 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 2.0 [7,11] 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 2.c [2,4] 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 2.14 [2,4] 2024-07-29T00:36:03.185 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 2.1d [8,11] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.0 [4,6] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.e [5,9] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.1e [3,9] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.27 [4,7] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.31 [4,8] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.36 [4,9] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:pg_upmap_items 3.39 [5,8] 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:0/3344352688 expires 2024-07-30T00:26:49.412212+0000 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:6801/61876509 expires 2024-07-30T00:26:49.412212+0000 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:6800/61876509 expires 2024-07-30T00:26:49.412212+0000 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:0/1674524668 expires 2024-07-30T00:26:49.412212+0000 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:0/1820809193 expires 2024-07-30T00:26:20.140387+0000 2024-07-29T00:36:03.186 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:6801/2801743500 expires 2024-07-30T00:26:20.140387+0000 2024-07-29T00:36:03.187 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:6800/2801743500 expires 2024-07-30T00:26:20.140387+0000 2024-07-29T00:36:03.187 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:0/1808586297 expires 2024-07-30T00:26:49.412212+0000 2024-07-29T00:36:03.187 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:0/1779316541 expires 2024-07-30T00:26:20.140387+0000 2024-07-29T00:36:03.187 INFO:teuthology.orchestra.run.smithi175.stdout:blocklist 172.21.15.175:0/2775747625 expires 2024-07-30T00:26:20.140387+0000 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9b1c0a2c30 msgr2=0x7f9b1c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9b1c0a2c30 0x7f9b1c0a50f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b380762f0 tx=0x7f9b28005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 msgr2=0x7f9b3819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 0x7f9b3819e6d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000e9c0 tx=0x7f9b2000ee80 comp rx=0 tx=0).stop 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 shutdown_connections 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f9b1c0a2c30 0x7f9b1c0a50f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b3810ded0 0x7f9b381a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.185+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38075f10 0x7f9b3819e6d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.186+0000 7f9b3ec00640 1 --2- 172.21.15.175:0/1940505691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b38074940 0x7f9b3819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.186+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 >> 172.21.15.175:0/1940505691 conn(0x7f9b380fd080 msgr2=0x7f9b38101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.186+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 shutdown_connections 2024-07-29T00:36:03.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:03.186+0000 7f9b3ec00640 1 -- 172.21.15.175:0/1940505691 wait complete. 2024-07-29T00:36:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:03 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1940505691' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-29T00:36:04.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:03 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1940505691' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-29T00:36:04.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:03 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1940505691' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-29T00:36:04.133 INFO:teuthology.run_tasks:Running task fs.ready... 2024-07-29T00:36:04.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:36:04.591 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:04.986 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:04 smithi175 ceph-mon[32138]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:04.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:04 smithi181 ceph-mon[36706]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:05.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:04 smithi204 ceph-mon[37368]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 -- 172.21.15.175:0/408588196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 msgr2=0x7f80cc10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 --2- 172.21.15.175:0/408588196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc10e130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009930 tx=0x7f80bc02a420 comp rx=0 tx=0).stop 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 -- 172.21.15.175:0/408588196 shutdown_connections 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 --2- 172.21.15.175:0/408588196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80cc108670 0x7f80cc114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 --2- 172.21.15.175:0/408588196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 0x7f80cc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 --2- 172.21.15.175:0/408588196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc10e130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 -- 172.21.15.175:0/408588196 >> 172.21.15.175:0/408588196 conn(0x7f80cc077670 msgr2=0x7f80cc079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 -- 172.21.15.175:0/408588196 shutdown_connections 2024-07-29T00:36:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.271+0000 7f80d1600640 1 -- 172.21.15.175:0/408588196 wait complete. 2024-07-29T00:36:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 Processor -- start 2024-07-29T00:36:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 -- start start 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 0x7f80cc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80cc108670 0x7f80cc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc1a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc118080 con 0x7f80cc10dd50 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc117f00 con 0x7f80cc108670 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.273+0000 7f80d1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc118200 con 0x7f80cc107d10 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80d0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc1a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80d0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc1a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57624/0 (socket says 172.21.15.175:57624) 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80d0c00640 1 -- 172.21.15.175:0/3614686325 learned_addr learned my addr 172.21.15.175:0/3614686325 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80cbe00640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 0x7f80cc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80c3400640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80cc108670 0x7f80cc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:05.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80cbe00640 1 -- 172.21.15.175:0/3614686325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80cc108670 msgr2=0x7f80cc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:05.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80cbe00640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80cc108670 0x7f80cc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80cbe00640 1 -- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 msgr2=0x7f80cc1a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80cbe00640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc1a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80cbe00640 1 -- 172.21.15.175:0/3614686325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1a3560 con 0x7f80cc107d10 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.274+0000 7f80d0c00640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc1a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.275+0000 7f80cbe00640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 0x7f80cc19e5b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005940 tx=0x7f80bc0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.275+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc03f070 con 0x7f80cc107d10 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.275+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1a37f0 con 0x7f80cc107d10 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.275+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80bc0044c0 con 0x7f80cc107d10 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.275+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc035070 con 0x7f80cc107d10 2024-07-29T00:36:05.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.275+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1ab130 con 0x7f80cc107d10 2024-07-29T00:36:05.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.276+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc108130 con 0x7f80cc107d10 2024-07-29T00:36:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.278+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f80bc03b460 con 0x7f80cc107d10 2024-07-29T00:36:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.280+0000 7f80c9600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80b00a2b80 0x7f80b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.280+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f80bc0ef800 con 0x7f80cc107d10 2024-07-29T00:36:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.280+0000 7f80c3400640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80b00a2b80 0x7f80b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.281+0000 7f80c3400640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80b00a2b80 0x7f80b00a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80cc10d4d0 tx=0x7f80b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.282+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f80bc0bc070 con 0x7f80cc107d10 2024-07-29T00:36:05.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.510+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80cc19f910 con 0x7f80cc107d10 2024-07-29T00:36:05.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.511+0000 7f80c9600640 1 -- 172.21.15.175:0/3614686325 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f80bc0c0370 con 0x7f80cc107d10 2024-07-29T00:36:05.512 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:05.512 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":28,"btime":"2024-07-29T00:34:15:679894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:34:14.951643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:36:05.512 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:36:05.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80b00a2b80 msgr2=0x7f80b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:05.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80b00a2b80 0x7f80b00a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80cc10d4d0 tx=0x7f80b4005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:05.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 msgr2=0x7f80cc19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:05.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 0x7f80cc19e5b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005940 tx=0x7f80bc0040c0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 shutdown_connections 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc10dd50 0x7f80cc1a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80cc108670 0x7f80cc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f80b00a2b80 0x7f80b00a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 --2- 172.21.15.175:0/3614686325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc107d10 0x7f80cc19e5b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 >> 172.21.15.175:0/3614686325 conn(0x7f80cc077670 msgr2=0x7f80cc078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 shutdown_connections 2024-07-29T00:36:05.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:05.514+0000 7f80d1600640 1 -- 172.21.15.175:0/3614686325 wait complete. 2024-07-29T00:36:05.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:05 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3614686325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:05 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3614686325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:05 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3614686325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:06.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:36:06.853 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:06.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:06 smithi181 ceph-mon[36706]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:07.001 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:06 smithi175 ceph-mon[32138]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:07.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:06 smithi204 ceph-mon[37368]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-07-29T00:36:07.575 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 -- 172.21.15.175:0/2372484659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f329010dd50 msgr2=0x7f329010e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 --2- 172.21.15.175:0/2372484659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f329010dd50 0x7f329010e130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3284009900 tx=0x7f328402a400 comp rx=0 tx=0).stop 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 -- 172.21.15.175:0/2372484659 shutdown_connections 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 --2- 172.21.15.175:0/2372484659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3290108670 0x7f3290114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 --2- 172.21.15.175:0/2372484659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3290107d10 0x7f3290108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 --2- 172.21.15.175:0/2372484659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f329010dd50 0x7f329010e130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 -- 172.21.15.175:0/2372484659 >> 172.21.15.175:0/2372484659 conn(0x7f3290077670 msgr2=0x7f3290079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 -- 172.21.15.175:0/2372484659 shutdown_connections 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.574+0000 7f3297200640 1 -- 172.21.15.175:0/2372484659 wait complete. 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.575+0000 7f3297200640 1 Processor -- start 2024-07-29T00:36:07.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.575+0000 7f3297200640 1 -- start start 2024-07-29T00:36:07.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3297200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 0x7f329019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:07.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3297200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3290108670 0x7f329019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:07.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3297200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329010dd50 0x7f32901a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:07.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3297200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32901180b0 con 0x7f329010dd50 2024-07-29T00:36:07.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3297200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3290117f30 con 0x7f3290107d10 2024-07-29T00:36:07.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3297200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3290118230 con 0x7f3290108670 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3295e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 0x7f329019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3296800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329010dd50 0x7f32901a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3295e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 0x7f329019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57072/0 (socket says 172.21.15.175:57072) 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3295e00640 1 -- 172.21.15.175:0/1883839836 learned_addr learned my addr 172.21.15.175:0/1883839836 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3296800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329010dd50 0x7f32901a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57650/0 (socket says 172.21.15.175:57650) 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.576+0000 7f3295400640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3290108670 0x7f329019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3295e00640 1 -- 172.21.15.175:0/1883839836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3290108670 msgr2=0x7f329019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3295e00640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3290108670 0x7f329019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3295e00640 1 -- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329010dd50 msgr2=0x7f32901a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3295e00640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329010dd50 0x7f32901a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3295e00640 1 -- 172.21.15.175:0/1883839836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32901a3600 con 0x7f3290107d10 2024-07-29T00:36:07.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3295e00640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 0x7f329019e650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f328402a9f0 tx=0x7f3284003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:07.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328403f070 con 0x7f3290107d10 2024-07-29T00:36:07.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32901a3890 con 0x7f3290107d10 2024-07-29T00:36:07.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.577+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3284031c30 con 0x7f3290107d10 2024-07-29T00:36:07.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.578+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284035030 con 0x7f3290107d10 2024-07-29T00:36:07.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.578+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32901ab1d0 con 0x7f3290107d10 2024-07-29T00:36:07.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.579+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3290109400 con 0x7f3290107d10 2024-07-29T00:36:07.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.581+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3284042070 con 0x7f3290107d10 2024-07-29T00:36:07.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.582+0000 7f328aa00640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32780a2b80 0x7f32780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:07.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.582+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f32840ef360 con 0x7f3290107d10 2024-07-29T00:36:07.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.582+0000 7f3295400640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32780a2b80 0x7f32780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:07.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.583+0000 7f3295400640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32780a2b80 0x7f32780a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f329010d490 tx=0x7f327c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:07.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.587+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3284031de0 con 0x7f3290107d10 2024-07-29T00:36:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.817+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32901101e0 con 0x7f3290107d10 2024-07-29T00:36:07.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.818+0000 7f328aa00640 1 -- 172.21.15.175:0/1883839836 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f32840bbbd0 con 0x7f3290107d10 2024-07-29T00:36:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:07.819 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":28,"btime":"2024-07-29T00:34:15:679894+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:34:14.951643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:36:07.819 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 28 2024-07-29T00:36:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32780a2b80 msgr2=0x7f32780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32780a2b80 0x7f32780a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f329010d490 tx=0x7f327c005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 msgr2=0x7f329019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 0x7f329019e650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f328402a9f0 tx=0x7f3284003c10 comp rx=0 tx=0).stop 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 shutdown_connections 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329010dd50 0x7f32901a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f32780a2b80 0x7f32780a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3290108670 0x7f329019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 --2- 172.21.15.175:0/1883839836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3290107d10 0x7f329019e650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.821+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 >> 172.21.15.175:0/1883839836 conn(0x7f3290077670 msgr2=0x7f3290078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.822+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 shutdown_connections 2024-07-29T00:36:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:07.822+0000 7f3297200640 1 -- 172.21.15.175:0/1883839836 wait complete. 2024-07-29T00:36:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:07 smithi181 ceph-mon[36706]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:36:08.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:07 smithi175 ceph-mon[32138]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:36:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:07 smithi204 ceph-mon[37368]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:36:08.752 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:36:08.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:08 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1883839836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:09.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:08 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1883839836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:09.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:08 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1883839836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:09.217 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:09.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.914+0000 7fe23a200640 1 -- 172.21.15.175:0/1929697082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe234107d10 msgr2=0x7fe234108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.914+0000 7fe23a200640 1 --2- 172.21.15.175:0/1929697082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe234107d10 0x7fe234108130 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009890 tx=0x7fe21c02a360 comp rx=0 tx=0).stop 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 -- 172.21.15.175:0/1929697082 shutdown_connections 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 --2- 172.21.15.175:0/1929697082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 0x7fe234114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 --2- 172.21.15.175:0/1929697082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe234107d10 0x7fe234108130 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 --2- 172.21.15.175:0/1929697082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23410dd50 0x7fe23410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 -- 172.21.15.175:0/1929697082 >> 172.21.15.175:0/1929697082 conn(0x7fe234077670 msgr2=0x7fe234079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 -- 172.21.15.175:0/1929697082 shutdown_connections 2024-07-29T00:36:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.915+0000 7fe23a200640 1 -- 172.21.15.175:0/1929697082 wait complete. 2024-07-29T00:36:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.916+0000 7fe23a200640 1 Processor -- start 2024-07-29T00:36:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.916+0000 7fe23a200640 1 -- start start 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe23a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 0x7fe23419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe23a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 0x7fe23419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe23a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe23410dd50 0x7fe2341a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe23a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe234118070 con 0x7fe234108670 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe23a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe234117ef0 con 0x7fe234107d10 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe23a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2341181f0 con 0x7fe23410dd50 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe238e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 0x7fe23419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe22be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 0x7fe23419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe238e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 0x7fe23419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56112/0 (socket says 172.21.15.175:56112) 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe238e00640 1 -- 172.21.15.175:0/1873067045 learned_addr learned my addr 172.21.15.175:0/1873067045 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe239800640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe23410dd50 0x7fe2341a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.917+0000 7fe238e00640 1 -- 172.21.15.175:0/1873067045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe23410dd50 msgr2=0x7fe2341a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe238e00640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe23410dd50 0x7fe2341a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe238e00640 1 -- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 msgr2=0x7fe23419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe238e00640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 0x7fe23419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe238e00640 1 -- 172.21.15.175:0/1873067045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341a3580 con 0x7fe234107d10 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe22be00640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 0x7fe23419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:09.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe239800640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe23410dd50 0x7fe2341a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe238e00640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 0x7fe23419e5d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9f0 tx=0x7fe22400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22400cd80 con 0x7fe234107d10 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2341a3870 con 0x7fe234107d10 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe224005500 con 0x7fe234107d10 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224010690 con 0x7fe234107d10 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.918+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2341ab150 con 0x7fe234107d10 2024-07-29T00:36:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.919+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe234108130 con 0x7fe234107d10 2024-07-29T00:36:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.922+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe2240046c0 con 0x7fe234107d10 2024-07-29T00:36:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.923+0000 7fe232000640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe2180a2b80 0x7fe2180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.923+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7fe2240cf530 con 0x7fe234107d10 2024-07-29T00:36:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.923+0000 7fe22be00640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe2180a2b80 0x7fe2180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:09.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.924+0000 7fe22be00640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe2180a2b80 0x7fe2180a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe21c02a950 tx=0x7fe21c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:09.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:09.925+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe22409c3f0 con 0x7fe234107d10 2024-07-29T00:36:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:09 smithi175 ceph-mon[32138]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:36:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:09 smithi204 ceph-mon[37368]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:36:10.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.116+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2341101e0 con 0x7fe234107d10 2024-07-29T00:36:10.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.118+0000 7fe232000640 1 -- 172.21.15.175:0/1873067045 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22836 (secure 0 0 0) 0x7fe22409bcf0 con 0x7fe234107d10 2024-07-29T00:36:10.119 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:10.119 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":91,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:35:51.651385+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:36:10.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe2180a2b80 msgr2=0x7fe2180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:10.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe2180a2b80 0x7fe2180a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe21c02a950 tx=0x7fe21c0056d0 comp rx=0 tx=0).stop 2024-07-29T00:36:10.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 msgr2=0x7fe23419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:10.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 0x7fe23419e5d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9f0 tx=0x7fe22400edf0 comp rx=0 tx=0).stop 2024-07-29T00:36:10.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 shutdown_connections 2024-07-29T00:36:10.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7fe2180a2b80 0x7fe2180a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:10.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe23410dd50 0x7fe2341a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:10.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe234108670 0x7fe23419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:10.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 --2- 172.21.15.175:0/1873067045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234107d10 0x7fe23419e5d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:10.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 >> 172.21.15.175:0/1873067045 conn(0x7fe234077670 msgr2=0x7fe234078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:10.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 shutdown_connections 2024-07-29T00:36:10.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:10.121+0000 7fe23a200640 1 -- 172.21.15.175:0/1873067045 wait complete. 2024-07-29T00:36:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:09 smithi181 ceph-mon[36706]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:36:11.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:10 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1873067045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:11.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:10 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1873067045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:11.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-29T00:33:58.041883+0000', 'modified': '2024-07-29T00:34:14.951643+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24515, 'mds_2': 24503, 'mds_3': 24499, 'mds_4': 24484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/1301583270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 1301583270}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 1301583270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/4123110499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 4123110499}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 4123110499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'j', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/284852041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 284852041}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 284852041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/568693704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 568693704}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 568693704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1977765890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1977765890}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1977765890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.204:6837/2487746199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2487746199}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2487746199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6839/2455267021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2455267021}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2455267021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.181:6837/753661139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 753661139}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 753661139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.175:6839/1522099567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1522099567}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1522099567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/4150501134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 4150501134}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 4150501134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24484, 24499, 24503, 24515, 24539]} 2024-07-29T00:36:11.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-29T00:36:11.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.c status 2024-07-29T00:36:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:10 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1873067045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:11.661 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:11 smithi175 ceph-mon[32138]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:36:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:11 smithi204 ceph-mon[37368]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:36:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:11 smithi181 ceph-mon[36706]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:36:12.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 -- 172.21.15.175:0/1311878722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873810bcb0 msgr2=0x7f873810e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 --2- 172.21.15.175:0/1311878722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873810bcb0 0x7f873810e120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8730009ca0 tx=0x7f873002a950 comp rx=0 tx=0).stop 2024-07-29T00:36:12.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 -- 172.21.15.175:0/1311878722 shutdown_connections 2024-07-29T00:36:12.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 --2- 172.21.15.175:0/1311878722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873810bcb0 0x7f873810e120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 --2- 172.21.15.175:0/1311878722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87381031f0 0x7f873810b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 --2- 172.21.15.175:0/1311878722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381028d0 0x7f8738102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 -- 172.21.15.175:0/1311878722 >> 172.21.15.175:0/1311878722 conn(0x7f87380fc2b0 msgr2=0x7f87380fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:12.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 -- 172.21.15.175:0/1311878722 shutdown_connections 2024-07-29T00:36:12.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.316+0000 7f873d400640 1 -- 172.21.15.175:0/1311878722 wait complete. 2024-07-29T00:36:12.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.317+0000 7f873d400640 1 Processor -- start 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.317+0000 7f873d400640 1 -- start start 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 0x7f8738100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381031f0 0x7f8738100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873810bcb0 0x7f8738100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8738111ab0 con 0x7f87381028d0 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873d400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8738111930 con 0x7f87381031f0 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8738111c30 con 0x7f873810bcb0 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 0x7f8738100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381031f0 0x7f8738100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 0x7f8738100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35722/0 (socket says 172.21.15.175:35722) 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737e00640 1 -- 172.21.15.175:0/2423180112 learned_addr learned my addr 172.21.15.175:0/2423180112 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f873ca00640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873810bcb0 0x7f8738100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737400640 1 -- 172.21.15.175:0/2423180112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873810bcb0 msgr2=0x7f8738100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873810bcb0 0x7f8738100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737400640 1 -- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 msgr2=0x7f8738100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 0x7f8738100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737400640 1 -- 172.21.15.175:0/2423180112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87381a6470 con 0x7f87381031f0 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.318+0000 7f8737e00640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 0x7f8738100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.319+0000 7f8737400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381031f0 0x7f8738100a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f872400e9f0 tx=0x7f872400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:12.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.319+0000 7f8734c00640 1 -- 172.21.15.175:0/2423180112 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872400ccc0 con 0x7f87381031f0 2024-07-29T00:36:12.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.319+0000 7f8734c00640 1 -- 172.21.15.175:0/2423180112 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8724005020 con 0x7f87381031f0 2024-07-29T00:36:12.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.319+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87381a6760 con 0x7f87381031f0 2024-07-29T00:36:12.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.319+0000 7f8734c00640 1 -- 172.21.15.175:0/2423180112 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87240049e0 con 0x7f87381031f0 2024-07-29T00:36:12.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.319+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381a6c50 con 0x7f87381031f0 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.322+0000 7f874144d740 1 Processor -- start 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 -- start start 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f8734c00640 1 -- 172.21.15.175:0/2423180112 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8724005560 con 0x7f87381031f0 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c1b80c790 con 0x557c1b8a6ea0 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557c1b7a4880 con 0x557c1b8a6580 2024-07-29T00:36:12.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f874144d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557c1b808750 con 0x557c1b8b1760 2024-07-29T00:36:12.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f8737e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f873ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f873ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56148/0 (socket says 172.21.15.175:56148) 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.323+0000 7f873ca00640 1 -- 172.21.15.175:0/1768712951 learned_addr learned my addr 172.21.15.175:0/1768712951 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737400640 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737e00640 1 -- 172.21.15.175:0/1768712951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 msgr2=0x557c1b8b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737e00640 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737e00640 1 -- 172.21.15.175:0/1768712951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 msgr2=0x557c1b8a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737e00640 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737e00640 1 -- 172.21.15.175:0/1768712951 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1b57e2b0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8737e00640 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8b1220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87280114f0 tx=0x7f87280119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15af14bedc3a55d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.324+0000 7f8734c00640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f872d200640 1 -- 172.21.15.175:0/1768712951 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872801e070 con 0x557c1b8a6ea0 2024-07-29T00:36:12.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f872d200640 1 -- 172.21.15.175:0/1768712951 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87280035f0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f8734c00640 1 -- 172.21.15.175:0/2423180112 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f87240cefd0 con 0x7f87381031f0 2024-07-29T00:36:12.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f872d200640 1 -- 172.21.15.175:0/1768712951 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872800f820 con 0x557c1b8a6ea0 2024-07-29T00:36:12.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f873ca00640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f874144d740 1 -- 172.21.15.175:0/1768712951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 msgr2=0x557c1b8b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f874144d740 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8b1220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87280114f0 tx=0x7f87280119b0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.325+0000 7f873ca00640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f871c0a2b80 0x7f871c0a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8730009a60 tx=0x7f8730005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 -- 172.21.15.175:0/1768712951 shutdown_connections 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8b1220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 --2- 172.21.15.175:0/1768712951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 -- 172.21.15.175:0/1768712951 >> 172.21.15.175:0/1768712951 conn(0x557c1b8a2b10 msgr2=0x557c1b8a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 -- 172.21.15.175:0/1768712951 shutdown_connections 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 -- 172.21.15.175:0/1768712951 wait complete. 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 -- start start 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.326+0000 7f874144d740 1 Processor -- start 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 0x557c1b8aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8b1760 0x557c1b8af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8b4330 con 0x557c1b8b1760 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557c1b7c7960 con 0x557c1b8a6580 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8b44b0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f874144d740 10 client.0 _ll_register_callbacks cb 0x557c1b51b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f873ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8b1760 0x557c1b8af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f8737e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f8737400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 0x557c1b8aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f873ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8b1760 0x557c1b8af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35740/0 (socket says 172.21.15.175:35740) 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f8737400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 0x557c1b8aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34868/0 (socket says 172.21.15.175:34868) 2024-07-29T00:36:12.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.327+0000 7f873ca00640 1 -- 172.21.15.175:0/4126702859 learned_addr learned my addr 172.21.15.175:0/4126702859 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8737400640 1 -- 172.21.15.175:0/4126702859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 msgr2=0x557c1b8ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8737400640 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8737400640 1 -- 172.21.15.175:0/4126702859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8b1760 msgr2=0x557c1b8af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8737400640 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8b1760 0x557c1b8af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8737400640 1 -- 172.21.15.175:0/4126702859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1b7d6a90 con 0x557c1b8a6ea0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8737400640 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 0x557c1b8aeb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x557c1b8a3e90 tx=0x7f8724097b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492838703a82f203 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8721600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8721600640 1 -- 172.21.15.175:0/4126702859 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872401c070 con 0x557c1b8a6ea0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87280035f0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.328+0000 7f8721600640 1 -- 172.21.15.175:0/4126702859 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f8724004ca0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f8721600640 1 -- 172.21.15.175:0/4126702859 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87240051d0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 10 client.25027 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c1b8bff70 con 0x557c1b8a6ea0 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f8721600640 1 -- 172.21.15.175:0/4126702859 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87280035f0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f8721600640 1 -- 172.21.15.175:0/4126702859 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87240a3f70 con 0x557c1b8a6ea0 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 10 client.25027 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 10 client.25027 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 20 client.25027 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b8c2520 0x557c1b8c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 4 client.25027 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.329+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c1b819d00 con 0x557c1b8c2520 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.330+0000 7f8737e00640 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b8c2520 0x557c1b8c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.330+0000 7f8737e00640 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b8c2520 0x557c1b8c3990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=789b32b947d35a14 server_cookie=1be31720c47af0cc in_seq=0 out_seq=0 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.330+0000 7f8721600640 10 client.25027 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:36:12.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f8721600640 1 -- 172.21.15.175:0/4126702859 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557c1b819d00 con 0x557c1b8c2520 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f8721600640 10 client.25027 handle_command_reply: tid=0 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 client.25027 shutdown 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 msgr2=0x557c1b8aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 0x557c1b8aeb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x557c1b8a3e90 tx=0x7f8724097b20 comp rx=0 tx=0).stop 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 shutdown_connections 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8b1760 0x557c1b8af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6ea0 0x557c1b8aeb10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b8c2520 0x557c1b8c3990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 --2- 172.21.15.175:0/4126702859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f8721600640 0 client.25027 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 >> 172.21.15.175:0/4126702859 conn(0x557c1b8a2b10 msgr2=0x557c1b8a8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 shutdown_connections 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.331+0000 7f874144d740 1 -- 172.21.15.175:0/4126702859 wait complete. 2024-07-29T00:36:12.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.332+0000 7f874144d740 20 client.25027 trim_cache size 0 max 16384 2024-07-29T00:36:12.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.351+0000 7f874144d740 1 Processor -- start 2024-07-29T00:36:12.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.351+0000 7f874144d740 1 -- start start 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f874144d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f874144d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f874144d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f874144d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8c1cd0 con 0x557c1b8a6ea0 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f874144d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8c23a0 con 0x557c1b8a6580 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f874144d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8c2a70 con 0x557c1b8b1760 2024-07-29T00:36:12.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f8737400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f873ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f8737e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f8737400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35752/0 (socket says 172.21.15.175:35752) 2024-07-29T00:36:12.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f8737400640 1 -- 172.21.15.175:0/142166061 learned_addr learned my addr 172.21.15.175:0/142166061 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:12.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.352+0000 7f873ca00640 1 -- 172.21.15.175:0/142166061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 msgr2=0x557c1b8a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f873ca00640 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f873ca00640 1 -- 172.21.15.175:0/142166061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 msgr2=0x557c1b8c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f873ca00640 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f873ca00640 1 -- 172.21.15.175:0/142166061 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1b8a37a0 con 0x557c1b8b1760 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f8737400640 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f8737e00640 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f873ca00640 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8ad5d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f873002ae70 tx=0x7f8730042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13786d9ec0e86296 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f8723e00640 1 -- 172.21.15.175:0/142166061 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730034040 con 0x557c1b8b1760 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f8723e00640 1 -- 172.21.15.175:0/142166061 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f873002fc00 con 0x557c1b8b1760 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.353+0000 7f8723e00640 1 -- 172.21.15.175:0/142166061 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730030880 con 0x557c1b8b1760 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 -- 172.21.15.175:0/142166061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 msgr2=0x557c1b8ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8ad5d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f873002ae70 tx=0x7f8730042000 comp rx=0 tx=0).stop 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 -- 172.21.15.175:0/142166061 shutdown_connections 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8b1760 0x557c1b8ad5d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 --2- 172.21.15.175:0/142166061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8a6580 0x557c1b8a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 -- 172.21.15.175:0/142166061 >> 172.21.15.175:0/142166061 conn(0x557c1b8a2b10 msgr2=0x557c1b8acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:12.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 -- 172.21.15.175:0/142166061 shutdown_connections 2024-07-29T00:36:12.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 -- 172.21.15.175:0/142166061 wait complete. 2024-07-29T00:36:12.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.354+0000 7f874144d740 1 -- start start 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 Processor -- start 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 0x557c1b8cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8b1760 0x557c1b8ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8c3810 con 0x557c1b8a6ea0 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8c3140 con 0x557c1b8b1760 2024-07-29T00:36:12.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557c1b8c3ee0 con 0x557c1b8a6580 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f874144d740 10 client.0 _ll_register_callbacks cb 0x7f87240046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.355+0000 7f8737400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 0x557c1b8cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35768/0 (socket says 172.21.15.175:35768) 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737400640 1 -- 172.21.15.175:0/3899600755 learned_addr learned my addr 172.21.15.175:0/3899600755 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 0x557c1b8cc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34874/0 (socket says 172.21.15.175:34874) 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f873ca00640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8b1760 0x557c1b8ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 -- 172.21.15.175:0/3899600755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8b1760 msgr2=0x557c1b8ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8b1760 0x557c1b8ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 -- 172.21.15.175:0/3899600755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 msgr2=0x557c1b8ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 -- 172.21.15.175:0/3899600755 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1b8abc50 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737400640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8737e00640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 0x557c1b8cc0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8728003ef0 tx=0x7f8728005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae738047882d8e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.356+0000 7f8721600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f8721600640 1 -- 172.21.15.175:0/3899600755 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872801e040 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873002fc00 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f8721600640 1 -- 172.21.15.175:0/3899600755 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f87280035f0 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f8721600640 1 -- 172.21.15.175:0/3899600755 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8728004120 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f874144d740 10 client.25033 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c1b8ca8b0 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f8721600640 1 -- 172.21.15.175:0/3899600755 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f873002fc00 con 0x557c1b8a6580 2024-07-29T00:36:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f8721600640 1 -- 172.21.15.175:0/3899600755 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f872800fad0 con 0x557c1b8a6580 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.357+0000 7f874144d740 10 client.25033 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f874144d740 10 client.25033 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f874144d740 20 client.25033 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f874144d740 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b931310 0x557c1b930020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f874144d740 4 client.25033 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "status"}) -- 0x557c1b814210 con 0x557c1b931310 2024-07-29T00:36:12.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f8737400640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b931310 0x557c1b930020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:12.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f8737400640 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b931310 0x557c1b930020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=593f83694c9c5b24 server_cookie=a50df30d596a9f40 in_seq=0 out_seq=0 2024-07-29T00:36:12.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.358+0000 7f8721600640 10 client.25033 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:36:12.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f8721600640 1 -- 172.21.15.175:0/3899600755 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557c1b814210 con 0x557c1b931310 2024-07-29T00:36:12.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f8721600640 10 client.25033 handle_command_reply: tid=0 2024-07-29T00:36:12.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 client.25033 shutdown 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 msgr2=0x557c1b8cc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 0x557c1b8cc0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8728003ef0 tx=0x7f8728005f10 comp rx=0 tx=0).stop 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 shutdown_connections 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x557c1b931310 0x557c1b930020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c1b8b1760 0x557c1b8ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c1b8a6ea0 0x557c1b8ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 --2- 172.21.15.175:0/3899600755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1b8a6580 0x557c1b8cc0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 >> 172.21.15.175:0/3899600755 conn(0x557c1b8a2b10 msgr2=0x557c1b8ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f8721600640 0 client.25033 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 shutdown_connections 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 1 -- 172.21.15.175:0/3899600755 wait complete. 2024-07-29T00:36:12.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.359+0000 7f874144d740 20 client.25033 trim_cache size 0 max 16384 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 4, 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24484, 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 119.685205153, 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 91, 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 166.74063739900001 2024-07-29T00:36:12.364 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:12.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f871c0a2b80 msgr2=0x7f871c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f871c0a2b80 0x7f871c0a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8730009a60 tx=0x7f8730005d80 comp rx=0 tx=0).stop 2024-07-29T00:36:12.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381031f0 msgr2=0x7f8738100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:12.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381031f0 0x7f8738100a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f872400e9f0 tx=0x7f872400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 shutdown_connections 2024-07-29T00:36:12.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873810bcb0 0x7f8738100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87381031f0 0x7f8738100a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 --2- 172.21.15.175:0/2423180112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87381028d0 0x7f8738100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:12.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 >> 172.21.15.175:0/2423180112 conn(0x7f87380fc2b0 msgr2=0x7f87380fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:12.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 shutdown_connections 2024-07-29T00:36:12.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:12.360+0000 7f873d400640 1 -- 172.21.15.175:0/2423180112 wait complete. 2024-07-29T00:36:13.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.i status 2024-07-29T00:36:13.723 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:14.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 -- 172.21.15.175:0/2740744897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81031a0 msgr2=0x7f6dc810b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 --2- 172.21.15.175:0/2740744897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81031a0 0x7f6dc810b690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009900 tx=0x7f6db002a480 comp rx=0 tx=0).stop 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 -- 172.21.15.175:0/2740744897 shutdown_connections 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 --2- 172.21.15.175:0/2740744897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc810bcb0 0x7f6dc810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 --2- 172.21.15.175:0/2740744897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81031a0 0x7f6dc810b690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 --2- 172.21.15.175:0/2740744897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc8102880 0x7f6dc8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 -- 172.21.15.175:0/2740744897 >> 172.21.15.175:0/2740744897 conn(0x7f6dc80fc300 msgr2=0x7f6dc80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 -- 172.21.15.175:0/2740744897 shutdown_connections 2024-07-29T00:36:14.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.412+0000 7f6dce200640 1 -- 172.21.15.175:0/2740744897 wait complete. 2024-07-29T00:36:14.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.413+0000 7f6dce200640 1 Processor -- start 2024-07-29T00:36:14.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.413+0000 7f6dce200640 1 -- start start 2024-07-29T00:36:14.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dce200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc8102880 0x7f6dc8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dce200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc81031a0 0x7f6dc81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dce200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 0x7f6dc819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dce200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8111b00 con 0x7f6dc810bcb0 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dce200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8111980 con 0x7f6dc8102880 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dce200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8111c80 con 0x7f6dc81031a0 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dcd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 0x7f6dc819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dcd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 0x7f6dc819c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35798/0 (socket says 172.21.15.175:35798) 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dcd800640 1 -- 172.21.15.175:0/3619073003 learned_addr learned my addr 172.21.15.175:0/3619073003 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc81031a0 0x7f6dc81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.414+0000 7f6dcce00640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc8102880 0x7f6dc8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dcd800640 1 -- 172.21.15.175:0/3619073003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc81031a0 msgr2=0x7f6dc81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dcd800640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc81031a0 0x7f6dc81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dcd800640 1 -- 172.21.15.175:0/3619073003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc8102880 msgr2=0x7f6dc8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dcd800640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc8102880 0x7f6dc8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dcd800640 1 -- 172.21.15.175:0/3619073003 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc819cf50 con 0x7f6dc810bcb0 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc81031a0 0x7f6dc81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:14.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dcd800640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 0x7f6dc819c870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000eb10 tx=0x7f6dc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:14.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dc6000640 1 -- 172.21.15.175:0/3619073003 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004b20 con 0x7f6dc810bcb0 2024-07-29T00:36:14.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dc6000640 1 -- 172.21.15.175:0/3619073003 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0004cc0 con 0x7f6dc810bcb0 2024-07-29T00:36:14.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc819d240 con 0x7f6dc810bcb0 2024-07-29T00:36:14.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dc6000640 1 -- 172.21.15.175:0/3619073003 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0010760 con 0x7f6dc810bcb0 2024-07-29T00:36:14.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.415+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81a4b20 con 0x7f6dc810bcb0 2024-07-29T00:36:14.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dc6000640 1 -- 172.21.15.175:0/3619073003 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6dc0010900 con 0x7f6dc810bcb0 2024-07-29T00:36:14.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 Processor -- start 2024-07-29T00:36:14.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 -- start start 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 0x563af7848960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848ea0 0x563af7853220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7853760 0x563af7855b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563af77ae790 con 0x563af7848ea0 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563af7746880 con 0x563af7853760 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.419+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563af77aa750 con 0x563af7848580 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 0x563af7848960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 0x563af7848960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34904/0 (socket says 172.21.15.175:34904) 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dcce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848ea0 0x563af7853220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 -- 172.21.15.175:0/2683682412 learned_addr learned my addr 172.21.15.175:0/2683682412 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7853760 0x563af7855b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 -- 172.21.15.175:0/2683682412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7853760 msgr2=0x563af7855b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7853760 0x563af7855b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 -- 172.21.15.175:0/2683682412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848ea0 msgr2=0x563af7853220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848ea0 0x563af7853220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dbfe00640 1 -- 172.21.15.175:0/2683682412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af75202b0 con 0x563af7848580 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dcce00640 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848ea0 0x563af7853220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.420+0000 7f6dc6000640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6dac0a2c00 0x7f6dac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.421+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 0x563af7848960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6db00096c0 tx=0x7f6db00056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88723ea2e715db9f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.421+0000 7f6dc6000640 1 -- 172.21.15.175:0/3619073003 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f6dc00cf740 con 0x7f6dc810bcb0 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.421+0000 7f6dcce00640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6dac0a2c00 0x7f6dac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.421+0000 7f6dbe000640 1 -- 172.21.15.175:0/2683682412 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0035040 con 0x563af7848580 2024-07-29T00:36:14.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.421+0000 7f6dbe000640 1 -- 172.21.15.175:0/2683682412 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0031c90 con 0x563af7848580 2024-07-29T00:36:14.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.421+0000 7f6dbe000640 1 -- 172.21.15.175:0/2683682412 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db003dc80 con 0x563af7848580 2024-07-29T00:36:14.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 -- 172.21.15.175:0/2683682412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 msgr2=0x563af7848960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dcce00640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6dac0a2c00 0x7f6dac0a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x563af7845970 tx=0x7f6db8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:14.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 0x563af7848960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6db00096c0 tx=0x7f6db00056d0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 -- 172.21.15.175:0/2683682412 shutdown_connections 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7853760 0x563af7855b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848ea0 0x563af7853220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/2683682412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848580 0x563af7848960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 -- 172.21.15.175:0/2683682412 >> 172.21.15.175:0/2683682412 conn(0x563af7844b10 msgr2=0x563af7844ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 -- 172.21.15.175:0/2683682412 shutdown_connections 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.422+0000 7f6dd22d1740 1 -- 172.21.15.175:0/2683682412 wait complete. 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 -- start start 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 Processor -- start 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848580 0x563af7857cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848ea0 0x563af7858210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7853760 0x563af785c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563af7856330 con 0x563af7848580 2024-07-29T00:36:14.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563af7769960 con 0x563af7848ea0 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563af78564b0 con 0x563af7853760 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.423+0000 7f6dd22d1740 10 client.0 _ll_register_callbacks cb 0x563af74bd7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848ea0 0x563af7858210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848580 0x563af7857cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848ea0 0x563af7858210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56192/0 (socket says 172.21.15.175:56192) 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dbfe00640 1 -- 172.21.15.175:0/1433085553 learned_addr learned my addr 172.21.15.175:0/1433085553 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcd800640 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7853760 0x563af785c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 -- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7853760 msgr2=0x563af785c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7853760 0x563af785c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 -- 172.21.15.175:0/1433085553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848ea0 msgr2=0x563af7858210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848ea0 0x563af7858210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 -- 172.21.15.175:0/1433085553 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af7778a90 con 0x563af7848580 2024-07-29T00:36:14.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6dcce00640 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848580 0x563af7857cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6db8012cb0 tx=0x7f6db8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6543da91fe779452 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:14.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.424+0000 7f6db6a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:36:14.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6db6a00640 1 -- 172.21.15.175:0/1433085553 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db802e060 con 0x563af7848580 2024-07-29T00:36:14.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db0031c90 con 0x563af7848580 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6db6a00640 1 -- 172.21.15.175:0/1433085553 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f6db8018030 con 0x563af7848580 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6db6a00640 1 -- 172.21.15.175:0/1433085553 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6db801ede0 con 0x563af7848580 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6dd22d1740 10 client.14958 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563af7866490 con 0x563af7848580 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.425+0000 7f6db6a00640 1 -- 172.21.15.175:0/1433085553 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0031c90 con 0x563af7848580 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6db6a00640 1 -- 172.21.15.175:0/1433085553 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6db800f580 con 0x563af7848580 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dd22d1740 10 client.14958 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dd22d1740 10 client.14958 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dd22d1740 20 client.14958 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af7868aa0 0x563af7869f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dd22d1740 4 client.14958 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af77bbd00 con 0x563af7868aa0 2024-07-29T00:36:14.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.426+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af7868aa0 0x563af7869f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.427+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af7868aa0 0x563af7869f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc1f6cd0f5c73592 server_cookie=ad58b27b7efb8933 in_seq=0 out_seq=0 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.427+0000 7f6db6a00640 10 client.14958 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6db6a00640 1 -- 172.21.15.175:0/1433085553 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563af77bbd00 con 0x563af7868aa0 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6db6a00640 10 client.14958 handle_command_reply: tid=0 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 client.14958 shutdown 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848580 msgr2=0x563af7857cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848580 0x563af7857cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6db8012cb0 tx=0x7f6db8004730 comp rx=0 tx=0).stop 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 shutdown_connections 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af7868aa0 0x563af7869f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7853760 0x563af785c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848ea0 0x563af7858210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1433085553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7848580 0x563af7857cd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6db6a00640 0 client.14958 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:36:14.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 >> 172.21.15.175:0/1433085553 conn(0x563af7844b10 msgr2=0x563af7843c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:14.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 shutdown_connections 2024-07-29T00:36:14.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1433085553 wait complete. 2024-07-29T00:36:14.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.428+0000 7f6dd22d1740 20 client.14958 trim_cache size 0 max 16384 2024-07-29T00:36:14.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 Processor -- start 2024-07-29T00:36:14.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 -- start start 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af785c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af78566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563af7868250 con 0x563af7853760 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563af7868920 con 0x563af7848580 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.448+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563af7868ff0 con 0x563af7848ea0 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dcd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af78566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af785c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dcce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dcce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56198/0 (socket says 172.21.15.175:56198) 2024-07-29T00:36:14.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dcce00640 1 -- 172.21.15.175:0/3069050184 learned_addr learned my addr 172.21.15.175:0/3069050184 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 -- 172.21.15.175:0/3069050184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 msgr2=0x563af784f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 -- 172.21.15.175:0/3069050184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 msgr2=0x563af78566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af78566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 -- 172.21.15.175:0/3069050184 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af78430d0 con 0x563af7848ea0 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dcce00640 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.449+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af785c260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x563af7857380 tx=0x7f6db002a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d695b39aa23cf0f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.450+0000 7f6dbd600640 1 -- 172.21.15.175:0/3069050184 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0035070 con 0x563af7848ea0 2024-07-29T00:36:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.450+0000 7f6dbd600640 1 -- 172.21.15.175:0/3069050184 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0005520 con 0x563af7848ea0 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.450+0000 7f6dbd600640 1 -- 172.21.15.175:0/3069050184 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0031880 con 0x563af7848ea0 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.450+0000 7f6dd22d1740 1 -- 172.21.15.175:0/3069050184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 msgr2=0x563af785c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.450+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af785c260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x563af7857380 tx=0x7f6db002a480 comp rx=0 tx=0).stop 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 -- 172.21.15.175:0/3069050184 shutdown_connections 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af78566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af785c260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/3069050184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 -- 172.21.15.175:0/3069050184 >> 172.21.15.175:0/3069050184 conn(0x563af7844b10 msgr2=0x563af784fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 -- 172.21.15.175:0/3069050184 shutdown_connections 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 -- 172.21.15.175:0/3069050184 wait complete. 2024-07-29T00:36:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 -- start start 2024-07-29T00:36:14.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.451+0000 7f6dd22d1740 1 Processor -- start 2024-07-29T00:36:14.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af784c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af784cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563af7869d90 con 0x563af7853760 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563af78696c0 con 0x563af7848580 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563af786a460 con 0x563af7848ea0 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dd22d1740 10 client.0 _ll_register_callbacks cb 0x7f6db8009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dcd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af784cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dcce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dcd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af784cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35818/0 (socket says 172.21.15.175:35818) 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dcce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56214/0 (socket says 172.21.15.175:56214) 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dcd800640 1 -- 172.21.15.175:0/1463117680 learned_addr learned my addr 172.21.15.175:0/1463117680 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.452+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af784c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcce00640 1 -- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 msgr2=0x563af784c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcce00640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af784c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcce00640 1 -- 172.21.15.175:0/1463117680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 msgr2=0x563af784cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcce00640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af784cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcce00640 1 -- 172.21.15.175:0/1463117680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af78633a0 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcd800640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af784cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dcce00640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784c030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6db8004700 tx=0x7f6db800f5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a2f36e01ac02b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6db6a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6db6a00640 1 -- 172.21.15.175:0/1463117680 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db801e730 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6db6a00640 1 -- 172.21.15.175:0/1463117680 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f6db8005020 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db0005520 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6db6a00640 1 -- 172.21.15.175:0/1463117680 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6db8005220 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dd22d1740 10 client.25058 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.453+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563af78645e0 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6db6a00640 1 -- 172.21.15.175:0/1463117680 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0005520 con 0x563af7848580 2024-07-29T00:36:14.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6db6a00640 1 -- 172.21.15.175:0/1463117680 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6db803e430 con 0x563af7848580 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dd22d1740 10 client.25058 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dd22d1740 10 client.25058 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dd22d1740 20 client.25058 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af78dc230 0x563af78daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dd22d1740 4 client.25058 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "status"}) -- 0x563af77b6210 con 0x563af78dc230 2024-07-29T00:36:14.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.454+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af78dc230 0x563af78daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6dbfe00640 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af78dc230 0x563af78daf40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16d3d9f401c53599 server_cookie=743cbcd52e31540b in_seq=0 out_seq=0 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6db6a00640 10 client.25058 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6db6a00640 1 -- 172.21.15.175:0/1463117680 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563af77b6210 con 0x563af78dc230 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6db6a00640 10 client.25058 handle_command_reply: tid=0 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6dd22d1740 1 client.25058 shutdown 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 msgr2=0x563af784c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.455+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784c030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6db8004700 tx=0x7f6db800f5f0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 shutdown_connections 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x563af78dc230 0x563af78daf40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af7853760 0x563af784cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563af7848ea0 0x563af784c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 --2- 172.21.15.175:0/1463117680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563af7848580 0x563af784c030 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6db6a00640 0 client.25058 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 >> 172.21.15.175:0/1463117680 conn(0x563af7844b10 msgr2=0x563af7843450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 shutdown_connections 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 1 -- 172.21.15.175:0/1463117680 wait complete. 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.456+0000 7f6dd22d1740 20 client.25058 trim_cache size 0 max 16384 2024-07-29T00:36:14.459 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 3, 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24499, 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 122.797583174, 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 91, 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 163.504214386 2024-07-29T00:36:14.460 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6dac0a2c00 msgr2=0x7f6dac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6dac0a2c00 0x7f6dac0a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x563af7845970 tx=0x7f6db8005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 msgr2=0x7f6dc819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 0x7f6dc819c870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000eb10 tx=0x7f6dc000c2d0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 shutdown_connections 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc810bcb0 0x7f6dc819c870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f6dac0a2c00 0x7f6dac0a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dc81031a0 0x7f6dc81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 --2- 172.21.15.175:0/3619073003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc8102880 0x7f6dc8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:14.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 >> 172.21.15.175:0/3619073003 conn(0x7f6dc80fc300 msgr2=0x7f6dc80fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:14.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 shutdown_connections 2024-07-29T00:36:14.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:14.457+0000 7f6dce200640 1 -- 172.21.15.175:0/3619073003 wait complete. 2024-07-29T00:36:14.976 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:14 smithi175 ceph-mon[32138]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:36:14.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:14 smithi181 ceph-mon[36706]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:36:15.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:14 smithi204 ceph-mon[37368]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-29T00:36:15.437 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.h status 2024-07-29T00:36:15.901 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.603+0000 7f83e4e00640 1 -- 172.21.15.175:0/3735701741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01076b0 msgr2=0x7f83e0107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.603+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3735701741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01076b0 0x7f83e0107a90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009cd0 tx=0x7f83d002a8e0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 -- 172.21.15.175:0/3735701741 shutdown_connections 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3735701741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83e0102050 0x7f83e010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3735701741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83e01016b0 0x7f83e0101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3735701741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01076b0 0x7f83e0107a90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 -- 172.21.15.175:0/3735701741 >> 172.21.15.175:0/3735701741 conn(0x7f83e00fd430 msgr2=0x7f83e00ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:16.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 -- 172.21.15.175:0/3735701741 shutdown_connections 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.604+0000 7f83e4e00640 1 -- 172.21.15.175:0/3735701741 wait complete. 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.605+0000 7f83e4e00640 1 Processor -- start 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.605+0000 7f83e4e00640 1 -- start start 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83e4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 0x7f83e0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83e4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83e0102050 0x7f83e0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83e4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83e01076b0 0x7f83e019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83e4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0111b00 con 0x7f83e0102050 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83e4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0111980 con 0x7f83e01076b0 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83e4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0111c80 con 0x7f83e01016b0 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83dea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83e0102050 0x7f83e0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 0x7f83e0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 0x7f83e0197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34950/0 (socket says 172.21.15.175:34950) 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 -- 172.21.15.175:0/3458372701 learned_addr learned my addr 172.21.15.175:0/3458372701 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83dfe00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83e01076b0 0x7f83e019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 -- 172.21.15.175:0/3458372701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83e01076b0 msgr2=0x7f83e019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83e01076b0 0x7f83e019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 -- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83e0102050 msgr2=0x7f83e0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.606+0000 7f83df400640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83e0102050 0x7f83e0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83df400640 1 -- 172.21.15.175:0/3458372701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e019cf70 con 0x7f83e01016b0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83df400640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 0x7f83e0197fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f83d0033040 tx=0x7f83d00093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83d7e00640 1 -- 172.21.15.175:0/3458372701 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0004040 con 0x7f83e01016b0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e019d200 con 0x7f83e01016b0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83d7e00640 1 -- 172.21.15.175:0/3458372701 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d00046c0 con 0x7f83e01016b0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a4b40 con 0x7f83e01016b0 2024-07-29T00:36:16.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.607+0000 7f83d7e00640 1 -- 172.21.15.175:0/3458372701 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0030c00 con 0x7f83e01016b0 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83d7e00640 1 -- 172.21.15.175:0/3458372701 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f83d00305a0 con 0x7f83e01016b0 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 Processor -- start 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 -- start start 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c6e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c7bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556702bd4790 con 0x556702c79760 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556702b6c880 con 0x556702c6e580 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.611+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556702bd0750 con 0x556702c6eea0 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c6e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83df400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c6e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56252/0 (socket says 172.21.15.175:56252) 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83dfe00640 1 -- 172.21.15.175:0/807961963 learned_addr learned my addr 172.21.15.175:0/807961963 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83dea00640 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c7bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83df400640 1 -- 172.21.15.175:0/807961963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 msgr2=0x556702c6e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83df400640 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c6e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83df400640 1 -- 172.21.15.175:0/807961963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 msgr2=0x556702c7bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83df400640 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c7bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83df400640 1 -- 172.21.15.175:0/807961963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567029462b0 con 0x556702c6eea0 2024-07-29T00:36:16.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83d7e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f83c40a2c00 0x7f83c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.612+0000 7f83dfe00640 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c6e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.613+0000 7f83d7e00640 1 -- 172.21.15.175:0/3458372701 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f83d00f0460 con 0x7f83e01016b0 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.613+0000 7f83dfe00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f83c40a2c00 0x7f83c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.613+0000 7f83df400640 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c79220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f83d00ede90 tx=0x7f83d00b8ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3de045a9a6b5df1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.613+0000 7f83d4c00640 1 -- 172.21.15.175:0/807961963 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00bc9f0 con 0x556702c6eea0 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.613+0000 7f83d4c00640 1 -- 172.21.15.175:0/807961963 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0036070 con 0x556702c6eea0 2024-07-29T00:36:16.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.613+0000 7f83d4c00640 1 -- 172.21.15.175:0/807961963 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00bc580 con 0x556702c6eea0 2024-07-29T00:36:16.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83dfe00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f83c40a2c00 0x7f83c40a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83d8012450 tx=0x7f83d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:16.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 -- 172.21.15.175:0/807961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 msgr2=0x556702c79220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c79220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f83d00ede90 tx=0x7f83d00b8ec0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 -- 172.21.15.175:0/807961963 shutdown_connections 2024-07-29T00:36:16.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c7bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c79220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/807961963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c6e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.614+0000 7f83e8ff4740 1 -- 172.21.15.175:0/807961963 >> 172.21.15.175:0/807961963 conn(0x556702c6ab10 msgr2=0x556702c6aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.615+0000 7f83e8ff4740 1 -- 172.21.15.175:0/807961963 shutdown_connections 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.615+0000 7f83e8ff4740 1 -- 172.21.15.175:0/807961963 wait complete. 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.615+0000 7f83e8ff4740 1 -- start start 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 Processor -- start 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c77160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556702c7c0f0 con 0x556702c6e580 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556702b8f960 con 0x556702c6eea0 2024-07-29T00:36:16.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556702c7c270 con 0x556702c79760 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.616+0000 7f83e8ff4740 10 client.0 _ll_register_callbacks cb 0x5567028e37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35852/0 (socket says 172.21.15.175:35852) 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 -- 172.21.15.175:0/1327877610 learned_addr learned my addr 172.21.15.175:0/1327877610 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83dea00640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83dfe00640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 -- 172.21.15.175:0/1327877610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 msgr2=0x556702c77160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c77160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 -- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 msgr2=0x556702c76c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 -- 172.21.15.175:0/1327877610 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556702b9ea90 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83df400640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c766e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000f80 tx=0x7f83d0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d2cb2b82e8b355 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.617+0000 7f83cd600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83cd600640 1 -- 172.21.15.175:0/1327877610 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00da060 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83cd600640 1 -- 172.21.15.175:0/1327877610 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f83d0030da0 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d0036070 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83cd600640 1 -- 172.21.15.175:0/1327877610 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83d00bc760 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83e8ff4740 10 client.14976 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556702c7c0f0 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83cd600640 1 -- 172.21.15.175:0/1327877610 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0036070 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83cd600640 1 -- 172.21.15.175:0/1327877610 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83d004e070 con 0x556702c6e580 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.618+0000 7f83e8ff4740 10 client.14976 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.619+0000 7f83e8ff4740 10 client.14976 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-29T00:36:16.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.619+0000 7f83e8ff4740 20 client.14976 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:16.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.619+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702c89f80 0x556702c8b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.619+0000 7f83e8ff4740 4 client.14976 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:16.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.619+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556702be1d00 con 0x556702c89f80 2024-07-29T00:36:16.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.619+0000 7f83dea00640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702c89f80 0x556702c8b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.620+0000 7f83dea00640 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702c89f80 0x556702c8b3f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28d9c35abdea74dc server_cookie=2102ecc089ff964e in_seq=0 out_seq=0 2024-07-29T00:36:16.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.620+0000 7f83cd600640 10 client.14976 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:36:16.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.620+0000 7f83cd600640 1 -- 172.21.15.175:0/1327877610 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556702be1d00 con 0x556702c89f80 2024-07-29T00:36:16.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.620+0000 7f83cd600640 10 client.14976 handle_command_reply: tid=0 2024-07-29T00:36:16.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.620+0000 7f83e8ff4740 1 client.14976 shutdown 2024-07-29T00:36:16.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 msgr2=0x556702c766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c766e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000f80 tx=0x7f83d0005ef0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 shutdown_connections 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c77160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702c89f80 0x556702c8b3f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/1327877610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c766e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 >> 172.21.15.175:0/1327877610 conn(0x556702c6ab10 msgr2=0x556702c707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83cd600640 0 client.14976 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 shutdown_connections 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 1 -- 172.21.15.175:0/1327877610 wait complete. 2024-07-29T00:36:16.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.621+0000 7f83e8ff4740 20 client.14976 trim_cache size 0 max 16384 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 Processor -- start 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 -- start start 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c70fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c73940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556702c89730 con 0x556702c6e580 2024-07-29T00:36:16.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556702c89e00 con 0x556702c6eea0 2024-07-29T00:36:16.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556702c8a4d0 con 0x556702c79760 2024-07-29T00:36:16.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c70fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c70fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35854/0 (socket says 172.21.15.175:35854) 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83df400640 1 -- 172.21.15.175:0/3830181951 learned_addr learned my addr 172.21.15.175:0/3830181951 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.641+0000 7f83dfe00640 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c73940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83dea00640 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83df400640 1 -- 172.21.15.175:0/3830181951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 msgr2=0x556702c73940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83df400640 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c73940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83df400640 1 -- 172.21.15.175:0/3830181951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 msgr2=0x556702c76fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83df400640 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83df400640 1 -- 172.21.15.175:0/3830181951 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556702c85370 con 0x556702c6e580 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83dea00640 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83df400640 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c70fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b95a0 tx=0x7f83d0005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ac531e505b0a77 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83cfe00640 1 -- 172.21.15.175:0/3830181951 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00f7050 con 0x556702c6e580 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83cfe00640 1 -- 172.21.15.175:0/3830181951 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0036040 con 0x556702c6e580 2024-07-29T00:36:16.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.642+0000 7f83cfe00640 1 -- 172.21.15.175:0/3830181951 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00da070 con 0x556702c6e580 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 -- 172.21.15.175:0/3830181951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 msgr2=0x556702c70fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c70fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b95a0 tx=0x7f83d0005f60 comp rx=0 tx=0).stop 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 -- 172.21.15.175:0/3830181951 shutdown_connections 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c79760 0x556702c73940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6eea0 0x556702c76fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/3830181951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c6e580 0x556702c70fe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 -- 172.21.15.175:0/3830181951 >> 172.21.15.175:0/3830181951 conn(0x556702c6ab10 msgr2=0x556702c719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 -- 172.21.15.175:0/3830181951 shutdown_connections 2024-07-29T00:36:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.643+0000 7f83e8ff4740 1 -- 172.21.15.175:0/3830181951 wait complete. 2024-07-29T00:36:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.644+0000 7f83e8ff4740 1 -- start start 2024-07-29T00:36:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.644+0000 7f83e8ff4740 1 Processor -- start 2024-07-29T00:36:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.644+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c93d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.644+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.644+0000 7f83e8ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556702c8b270 con 0x556702c79760 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556702c8aba0 con 0x556702c6e580 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83e8ff4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556702c8b940 con 0x556702c6eea0 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83e8ff4740 10 client.0 _ll_register_callbacks cb 0x7f83d00bdf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c93d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c93d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56268/0 (socket says 172.21.15.175:56268) 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83dfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 -- 172.21.15.175:0/4208601712 learned_addr learned my addr 172.21.15.175:0/4208601712 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83dea00640 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 -- 172.21.15.175:0/4208601712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 msgr2=0x556702c910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 -- 172.21.15.175:0/4208601712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 msgr2=0x556702c91600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c91600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.645+0000 7f83df400640 1 -- 172.21.15.175:0/4208601712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556702c91e30 con 0x556702c6e580 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83df400640 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c93d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f83d0003f40 tx=0x7f83d00308d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf5a5c9bedb175b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83cd600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83cd600640 1 -- 172.21.15.175:0/4208601712 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00f7040 con 0x556702c6e580 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83cd600640 1 -- 172.21.15.175:0/4208601712 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f83d00bce80 con 0x556702c6e580 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d0036040 con 0x556702c6e580 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83cd600640 1 -- 172.21.15.175:0/4208601712 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83d00da040 con 0x556702c6e580 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83e8ff4740 10 client.25076 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556702c91e30 con 0x556702c6e580 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83cd600640 1 -- 172.21.15.175:0/4208601712 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0036040 con 0x556702c6e580 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83cd600640 1 -- 172.21.15.175:0/4208601712 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83d004e040 con 0x556702c6e580 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.646+0000 7f83e8ff4740 10 client.25076 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83e8ff4740 10 client.25076 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83e8ff4740 20 client.25076 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702cfcb40 0x556702cfb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83e8ff4740 4 client.25076 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "status"}) -- 0x556702bdc210 con 0x556702cfcb40 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83dea00640 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702cfcb40 0x556702cfb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83dea00640 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702cfcb40 0x556702cfb850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9935d388d50adf8 server_cookie=19d0c25849cbd337 in_seq=0 out_seq=0 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.647+0000 7f83cd600640 10 client.25076 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83cd600640 1 -- 172.21.15.175:0/4208601712 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x556702bdc210 con 0x556702cfcb40 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83cd600640 10 client.25076 handle_command_reply: tid=0 2024-07-29T00:36:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 client.25076 shutdown 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 msgr2=0x556702c93d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c93d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f83d0003f40 tx=0x7f83d00308d0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 shutdown_connections 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556702c79760 0x556702c91600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556702c6eea0 0x556702c910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556702cfcb40 0x556702cfb850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 --2- 172.21.15.175:0/4208601712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556702c6e580 0x556702c93d20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83cd600640 0 client.25076 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 >> 172.21.15.175:0/4208601712 conn(0x556702c6ab10 msgr2=0x556702c74fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 shutdown_connections 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 1 -- 172.21.15.175:0/4208601712 wait complete. 2024-07-29T00:36:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.648+0000 7f83e8ff4740 20 client.25076 trim_cache size 0 max 16384 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 2, 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24503, 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 126.000512597, 2024-07-29T00:36:16.653 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:36:16.654 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 91, 2024-07-29T00:36:16.654 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:36:16.654 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 168.274123659 2024-07-29T00:36:16.654 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f83c40a2c00 msgr2=0x7f83c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f83c40a2c00 0x7f83c40a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83d8012450 tx=0x7f83d8005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 msgr2=0x7f83e0197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 0x7f83e0197fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f83d0033040 tx=0x7f83d00093d0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 shutdown_connections 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83e01076b0 0x7f83e019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83e0102050 0x7f83e0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f83c40a2c00 0x7f83c40a50c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 --2- 172.21.15.175:0/3458372701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e01016b0 0x7f83e0197fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 >> 172.21.15.175:0/3458372701 conn(0x7f83e00fd430 msgr2=0x7f83e00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.649+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 shutdown_connections 2024-07-29T00:36:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:16.650+0000 7f83e4e00640 1 -- 172.21.15.175:0/3458372701 wait complete. 2024-07-29T00:36:16.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:16 smithi181 ceph-mon[36706]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:36:17.006 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:16 smithi175 ceph-mon[32138]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:36:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:16 smithi204 ceph-mon[37368]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-29T00:36:17.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.g status 2024-07-29T00:36:17.996 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 -- 172.21.15.175:0/2370405553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 msgr2=0x7f512810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 --2- 172.21.15.175:0/2370405553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 0x7f512810e580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5120009cd0 tx=0x7f512002a8f0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 -- 172.21.15.175:0/2370405553 shutdown_connections 2024-07-29T00:36:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 --2- 172.21.15.175:0/2370405553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 0x7f512810e580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 --2- 172.21.15.175:0/2370405553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281016b0 0x7f5128101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 --2- 172.21.15.175:0/2370405553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281076b0 0x7f5128107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 -- 172.21.15.175:0/2370405553 >> 172.21.15.175:0/2370405553 conn(0x7f51280fd430 msgr2=0x7f51280ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 -- 172.21.15.175:0/2370405553 shutdown_connections 2024-07-29T00:36:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.681+0000 7f5127e00640 1 -- 172.21.15.175:0/2370405553 wait complete. 2024-07-29T00:36:18.683 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:18 smithi175 ceph-mon[32138]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.682+0000 7f5127e00640 1 Processor -- start 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 -- start start 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281016b0 0x7f512819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 0x7f512819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 0x7f512819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5128111b00 con 0x7f5128102050 2024-07-29T00:36:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5128111980 con 0x7f51281016b0 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5128111c80 con 0x7f51281076b0 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5127400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 0x7f512819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5126a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281016b0 0x7f512819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.683+0000 7f5126000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 0x7f512819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 0x7f512819ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35012/0 (socket says 172.21.15.175:35012) 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 -- 172.21.15.175:0/2782714404 learned_addr learned my addr 172.21.15.175:0/2782714404 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 -- 172.21.15.175:0/2782714404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281016b0 msgr2=0x7f512819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281016b0 0x7f512819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 -- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 msgr2=0x7f512819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 0x7f512819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 -- 172.21.15.175:0/2782714404 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512819f170 con 0x7f51281076b0 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5126a00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281016b0 0x7f512819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.684+0000 7f5127400640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 0x7f512819ea90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5120005e30 tx=0x7f5120003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.685+0000 7f511f400640 1 -- 172.21.15.175:0/2782714404 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120030e40 con 0x7f51281076b0 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.685+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512819f400 con 0x7f51281076b0 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.685+0000 7f511f400640 1 -- 172.21.15.175:0/2782714404 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120040070 con 0x7f51281076b0 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.685+0000 7f511f400640 1 -- 172.21.15.175:0/2782714404 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120045ad0 con 0x7f51281076b0 2024-07-29T00:36:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.685+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281a6d40 con 0x7f51281076b0 2024-07-29T00:36:18.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.689+0000 7f51307bc740 1 Processor -- start 2024-07-29T00:36:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.689+0000 7f511f400640 1 -- 172.21.15.175:0/2782714404 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5120045c70 con 0x7f51281076b0 2024-07-29T00:36:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 -- start start 2024-07-29T00:36:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aecef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecfa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecfcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597aec55790 con 0x5597aecfa760 2024-07-29T00:36:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5597aebed880 con 0x5597aecefea0 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f51307bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5597aec51750 con 0x5597aecef580 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f5126000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecfcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f5126000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecfcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35898/0 (socket says 172.21.15.175:35898) 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f5126000640 1 -- 172.21.15.175:0/1345169164 learned_addr learned my addr 172.21.15.175:0/1345169164 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f5126a00640 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecfa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.690+0000 7f5126000640 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aecef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126000640 1 -- 172.21.15.175:0/1345169164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 msgr2=0x5597aecef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126000640 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aecef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126000640 1 -- 172.21.15.175:0/1345169164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 msgr2=0x5597aecfa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126000640 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecfa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126000640 1 -- 172.21.15.175:0/1345169164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ae9c72b0 con 0x5597aecfa760 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f511f400640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f510c0a2b80 0x7f510c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126000640 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecfcb50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f51100058c0 tx=0x7f5110005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b7df9f72959c96 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f511f400640 1 -- 172.21.15.175:0/2782714404 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f51200f01c0 con 0x7f51281076b0 2024-07-29T00:36:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.691+0000 7f5126a00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f510c0a2b80 0x7f510c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.692+0000 7f5117e00640 1 -- 172.21.15.175:0/1345169164 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110004030 con 0x5597aecfa760 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.692+0000 7f5117e00640 1 -- 172.21.15.175:0/1345169164 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5110005500 con 0x5597aecfa760 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.692+0000 7f5117e00640 1 -- 172.21.15.175:0/1345169164 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110004d70 con 0x5597aecfa760 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 -- 172.21.15.175:0/1345169164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 msgr2=0x5597aecfcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f5126a00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f510c0a2b80 0x7f510c0a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5597aecec970 tx=0x7f511800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecfcb50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f51100058c0 tx=0x7f5110005850 comp rx=0 tx=0).stop 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 -- 172.21.15.175:0/1345169164 shutdown_connections 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecfcb50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecfa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 --2- 172.21.15.175:0/1345169164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aecef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 -- 172.21.15.175:0/1345169164 >> 172.21.15.175:0/1345169164 conn(0x5597aecebb10 msgr2=0x5597aecebef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 -- 172.21.15.175:0/1345169164 shutdown_connections 2024-07-29T00:36:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 -- 172.21.15.175:0/1345169164 wait complete. 2024-07-29T00:36:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.693+0000 7f51307bc740 1 -- start start 2024-07-29T00:36:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 Processor -- start 2024-07-29T00:36:18.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aecf7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 0x5597aecf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecfa760 0x5597aecf80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597aecfd330 con 0x5597aecefea0 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5597aec10960 con 0x5597aecef580 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5597aecfd4b0 con 0x5597aecfa760 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f51307bc740 10 client.0 _ll_register_callbacks cb 0x5597ae9647b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f5126000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 0x5597aecf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f5126a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aecf7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f5126000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 0x5597aecf7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35910/0 (socket says 172.21.15.175:35910) 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f5126000640 1 -- 172.21.15.175:0/624451147 learned_addr learned my addr 172.21.15.175:0/624451147 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.694+0000 7f5127400640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecfa760 0x5597aecf80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126000640 1 -- 172.21.15.175:0/624451147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecfa760 msgr2=0x5597aecf80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126000640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecfa760 0x5597aecf80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126000640 1 -- 172.21.15.175:0/624451147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 msgr2=0x5597aecf7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126000640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aecf7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126000640 1 -- 172.21.15.175:0/624451147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597aec1fa90 con 0x5597aecefea0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126a00640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aecf7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5126000640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 0x5597aecf7b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5110005420 tx=0x7f5110005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f067310c4c05dfc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.695+0000 7f5114c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f5114c00640 1 -- 172.21.15.175:0/624451147 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100046c0 con 0x5597aecefea0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110005500 con 0x5597aecefea0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f5114c00640 1 -- 172.21.15.175:0/624451147 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f5110019040 con 0x5597aecefea0 2024-07-29T00:36:18.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f5114c00640 1 -- 172.21.15.175:0/624451147 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5110004d00 con 0x5597aecefea0 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f51307bc740 10 client.14994 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597aed08f60 con 0x5597aecefea0 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f5114c00640 1 -- 172.21.15.175:0/624451147 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5110005500 con 0x5597aecefea0 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.696+0000 7f5114c00640 1 -- 172.21.15.175:0/624451147 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f511001f6a0 con 0x5597aecefea0 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f51307bc740 10 client.14994 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f51307bc740 10 client.14994 resolve_mds: resolved name 'g' to daemon mds.g 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f51307bc740 20 client.14994 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f51307bc740 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed0b270 0x5597aed0c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f51307bc740 4 client.14994 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597aec62d00 con 0x5597aed0b270 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f5126a00640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed0b270 0x5597aed0c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f5126a00640 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed0b270 0x5597aed0c6e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b339c93460d27f server_cookie=d3a2c9a37a405d56 in_seq=0 out_seq=0 2024-07-29T00:36:18.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.697+0000 7f5114c00640 10 client.14994 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.698+0000 7f5114c00640 1 -- 172.21.15.175:0/624451147 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597aec62d00 con 0x5597aed0b270 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.698+0000 7f5114c00640 10 client.14994 handle_command_reply: tid=0 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.698+0000 7f51307bc740 1 client.14994 shutdown 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 msgr2=0x5597aecf7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 0x5597aecf7b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5110005420 tx=0x7f5110005450 comp rx=0 tx=0).stop 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 shutdown_connections 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed0b270 0x5597aed0c6e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecfa760 0x5597aecf80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecefea0 0x5597aecf7b70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f5114c00640 0 client.14994 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:36:18.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 --2- 172.21.15.175:0/624451147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aecf7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 >> 172.21.15.175:0/624451147 conn(0x5597aecebb10 msgr2=0x5597aecf17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:18.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 shutdown_connections 2024-07-29T00:36:18.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 1 -- 172.21.15.175:0/624451147 wait complete. 2024-07-29T00:36:18.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.699+0000 7f51307bc740 20 client.14994 trim_cache size 0 max 16384 2024-07-29T00:36:18.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.719+0000 7f51307bc740 1 Processor -- start 2024-07-29T00:36:18.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.719+0000 7f51307bc740 1 -- start start 2024-07-29T00:36:18.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f51307bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aed0c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f51307bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecf7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f51307bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecf9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f51307bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597aed0aa20 con 0x5597aecfa760 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f51307bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5597aed0b0f0 con 0x5597aecefea0 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f51307bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5597aed0b7c0 con 0x5597aecef580 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f5126a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aed0c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f5127400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecf9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f5126a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aed0c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35042/0 (socket says 172.21.15.175:35042) 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f5126a00640 1 -- 172.21.15.175:0/2794142239 learned_addr learned my addr 172.21.15.175:0/2794142239 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:18.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.720+0000 7f5126000640 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecf7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126a00640 1 -- 172.21.15.175:0/2794142239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 msgr2=0x5597aecf7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126a00640 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecf7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126a00640 1 -- 172.21.15.175:0/2794142239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 msgr2=0x5597aecf9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126a00640 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecf9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126a00640 1 -- 172.21.15.175:0/2794142239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597aecefcd0 con 0x5597aecef580 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126000640 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecf7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5127400640 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecf9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5126a00640 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aed0c6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5597aecf87a0 tx=0x7f5118019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5853085c0548dfca server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5117400640 1 -- 172.21.15.175:0/2794142239 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511802a4e0 con 0x5597aecef580 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5117400640 1 -- 172.21.15.175:0/2794142239 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511802ab60 con 0x5597aecef580 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.721+0000 7f5117400640 1 -- 172.21.15.175:0/2794142239 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511801d410 con 0x5597aecef580 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 -- 172.21.15.175:0/2794142239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 msgr2=0x5597aed0c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aed0c6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5597aecf87a0 tx=0x7f5118019000 comp rx=0 tx=0).stop 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 -- 172.21.15.175:0/2794142239 shutdown_connections 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aecf9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecefea0 0x5597aecf7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 --2- 172.21.15.175:0/2794142239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecef580 0x5597aed0c6e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 -- 172.21.15.175:0/2794142239 >> 172.21.15.175:0/2794142239 conn(0x5597aecebb10 msgr2=0x5597aecf8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 -- 172.21.15.175:0/2794142239 shutdown_connections 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.722+0000 7f51307bc740 1 -- 172.21.15.175:0/2794142239 wait complete. 2024-07-29T00:36:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 -- start start 2024-07-29T00:36:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 Processor -- start 2024-07-29T00:36:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aed14a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecefea0 0x5597aed14fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aed0ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597aed0c560 con 0x5597aecfa760 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5597aed0be90 con 0x5597aecef580 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5597aed0cc30 con 0x5597aecefea0 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.723+0000 7f51307bc740 10 client.0 _ll_register_callbacks cb 0x7f511800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aed14a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aed14a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56334/0 (socket says 172.21.15.175:56334) 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 -- 172.21.15.175:0/2581905247 learned_addr learned my addr 172.21.15.175:0/2581905247 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126000640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecefea0 0x5597aed14fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5127400640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aed0ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 -- 172.21.15.175:0/2581905247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecefea0 msgr2=0x5597aed14fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecefea0 0x5597aed14fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 -- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 msgr2=0x5597aed0ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aed0ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.724+0000 7f5126a00640 1 -- 172.21.15.175:0/2581905247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597aed0e480 con 0x5597aecef580 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f5126a00640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aed14a90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5118010960 tx=0x7f5118010550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde15b62b4eae394 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f5114c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f5114c00640 1 -- 172.21.15.175:0/2581905247 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118042060 con 0x5597aecef580 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511802ab60 con 0x5597aecef580 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f5114c00640 1 -- 172.21.15.175:0/2581905247 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f511801e760 con 0x5597aecef580 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f5114c00640 1 -- 172.21.15.175:0/2581905247 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f511801eb00 con 0x5597aecef580 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f51307bc740 10 client.25094 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597aed12f70 con 0x5597aecef580 2024-07-29T00:36:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.725+0000 7f5114c00640 1 -- 172.21.15.175:0/2581905247 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511802ab60 con 0x5597aecef580 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f5114c00640 1 -- 172.21.15.175:0/2581905247 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f511803c030 con 0x5597aecef580 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f51307bc740 10 client.25094 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f51307bc740 10 client.25094 resolve_mds: resolved name 'g' to daemon mds.g 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f51307bc740 20 client.25094 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f51307bc740 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed79d00 0x5597aed78a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f51307bc740 4 client.25094 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "status"}) -- 0x5597aec5d210 con 0x5597aed79d00 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.726+0000 7f5126000640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed79d00 0x5597aed78a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f5126000640 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed79d00 0x5597aed78a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b66799b7bdb6fdc server_cookie=7bd76d96895b7847 in_seq=0 out_seq=0 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f5114c00640 10 client.25094 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f5114c00640 1 -- 172.21.15.175:0/2581905247 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5597aec5d210 con 0x5597aed79d00 2024-07-29T00:36:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f5114c00640 10 client.25094 handle_command_reply: tid=0 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 client.25094 shutdown 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 msgr2=0x5597aed14a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aed14a90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5118010960 tx=0x7f5118010550 comp rx=0 tx=0).stop 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 shutdown_connections 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5597aed79d00 0x5597aed78a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597aecfa760 0x5597aed0ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597aecefea0 0x5597aed14fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 --2- 172.21.15.175:0/2581905247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5597aecef580 0x5597aed14a90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f5114c00640 0 client.25094 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:36:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.727+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 >> 172.21.15.175:0/2581905247 conn(0x5597aecebb10 msgr2=0x5597aecf17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.728+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 shutdown_connections 2024-07-29T00:36:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.728+0000 7f51307bc740 1 -- 172.21.15.175:0/2581905247 wait complete. 2024-07-29T00:36:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.728+0000 7f51307bc740 20 client.25094 trim_cache size 0 max 16384 2024-07-29T00:36:18.732 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:18.732 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 1, 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24515, 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 129.08857310299999, 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 91, 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 163.082142428 2024-07-29T00:36:18.733 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f510c0a2b80 msgr2=0x7f510c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f510c0a2b80 0x7f510c0a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5597aecec970 tx=0x7f511800b390 comp rx=0 tx=0).stop 2024-07-29T00:36:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 msgr2=0x7f512819ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 0x7f512819ea90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5120005e30 tx=0x7f5120003f10 comp rx=0 tx=0).stop 2024-07-29T00:36:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 shutdown_connections 2024-07-29T00:36:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f510c0a2b80 0x7f510c0a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51281076b0 0x7f512819ea90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5128102050 0x7f512819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 --2- 172.21.15.175:0/2782714404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51281016b0 0x7f512819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:18.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 >> 172.21.15.175:0/2782714404 conn(0x7f51280fd430 msgr2=0x7f512810cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:18.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 shutdown_connections 2024-07-29T00:36:18.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:18.729+0000 7f5127e00640 1 -- 172.21.15.175:0/2782714404 wait complete. 2024-07-29T00:36:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:18 smithi181 ceph-mon[36706]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:36:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:18 smithi204 ceph-mon[37368]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-29T00:36:19.653 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.b status 2024-07-29T00:36:19.998 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:19 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:20.024 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:19 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:20.033 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:19 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:20.746 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:20.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:20 smithi175 ceph-mon[32138]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:36:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:20 smithi181 ceph-mon[36706]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:36:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:20 smithi204 ceph-mon[37368]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 -- 172.21.15.175:0/1619348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65901098c0 msgr2=0x7f6590109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 --2- 172.21.15.175:0/1619348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65901098c0 0x7f6590109ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6580009930 tx=0x7f658002a420 comp rx=0 tx=0).stop 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 -- 172.21.15.175:0/1619348757 shutdown_connections 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 --2- 172.21.15.175:0/1619348757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6590104260 0x7f6590110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 --2- 172.21.15.175:0/1619348757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65901038c0 0x7f6590103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 --2- 172.21.15.175:0/1619348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65901098c0 0x7f6590109ca0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.407+0000 7f6595800640 1 -- 172.21.15.175:0/1619348757 >> 172.21.15.175:0/1619348757 conn(0x7f65900ff600 msgr2=0x7f6590101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:21.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.408+0000 7f6595800640 1 -- 172.21.15.175:0/1619348757 shutdown_connections 2024-07-29T00:36:21.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.408+0000 7f6595800640 1 -- 172.21.15.175:0/1619348757 wait complete. 2024-07-29T00:36:21.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 Processor -- start 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 -- start start 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 0x7f659019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6590104260 0x7f659019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65901098c0 0x7f659019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6590113d10 con 0x7f6590104260 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6590113b90 con 0x7f65901038c0 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f6595800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6590113e90 con 0x7f65901098c0 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.409+0000 7f658fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 0x7f659019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 0x7f659019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53750/0 (socket says 172.21.15.175:53750) 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 -- 172.21.15.175:0/2029440203 learned_addr learned my addr 172.21.15.175:0/2029440203 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:21.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f6594e00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65901098c0 0x7f659019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658f400640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6590104260 0x7f659019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 -- 172.21.15.175:0/2029440203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65901098c0 msgr2=0x7f659019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65901098c0 0x7f659019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 -- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6590104260 msgr2=0x7f659019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6590104260 0x7f659019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 -- 172.21.15.175:0/2029440203 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659019f120 con 0x7f65901038c0 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.410+0000 7f658fe00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 0x7f659019a170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f658002a940 tx=0x7f6580003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.411+0000 7f658cc00640 1 -- 172.21.15.175:0/2029440203 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658003f070 con 0x7f65901038c0 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.411+0000 7f658cc00640 1 -- 172.21.15.175:0/2029440203 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6580031c90 con 0x7f65901038c0 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.411+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659019f3b0 con 0x7f65901038c0 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.411+0000 7f658cc00640 1 -- 172.21.15.175:0/2029440203 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580035030 con 0x7f65901038c0 2024-07-29T00:36:21.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.411+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901a6cf0 con 0x7f65901038c0 2024-07-29T00:36:21.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.414+0000 7f658cc00640 1 -- 172.21.15.175:0/2029440203 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6580042070 con 0x7f65901038c0 2024-07-29T00:36:21.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.414+0000 7f6599872740 1 Processor -- start 2024-07-29T00:36:21.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.414+0000 7f6599872740 1 -- start start 2024-07-29T00:36:21.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6599872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c67960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6599872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6599872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c72760 0x558009c74b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6599872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558009bcd790 con 0x558009c72760 2024-07-29T00:36:21.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6599872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558009b65880 con 0x558009c67580 2024-07-29T00:36:21.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6599872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558009bc9750 con 0x558009c67ea0 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6594e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c67960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f658fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6594e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c67960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53764/0 (socket says 172.21.15.175:53764) 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6594e00640 1 -- 172.21.15.175:0/4231279195 learned_addr learned my addr 172.21.15.175:0/4231279195 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f658f400640 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c72760 0x558009c74b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6594e00640 1 -- 172.21.15.175:0/4231279195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 msgr2=0x558009c72220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.415+0000 7f6594e00640 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c72220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6594e00640 1 -- 172.21.15.175:0/4231279195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c72760 msgr2=0x558009c74b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6594e00640 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c72760 0x558009c74b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6594e00640 1 -- 172.21.15.175:0/4231279195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800993f2b0 con 0x558009c67580 2024-07-29T00:36:21.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f658f400640 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c72760 0x558009c74b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:36:21.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6594e00640 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c67960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6588009f70 tx=0x7f6588010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48ec1671e4124f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:21.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f658cc00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65740a2c00 0x7f65740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6585600640 1 -- 172.21.15.175:0/4231279195 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588016070 con 0x558009c67580 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6585600640 1 -- 172.21.15.175:0/4231279195 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6588004d20 con 0x558009c67580 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f6585600640 1 -- 172.21.15.175:0/4231279195 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588012950 con 0x558009c67580 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f658fe00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65740a2c00 0x7f65740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.416+0000 7f658cc00640 1 -- 172.21.15.175:0/2029440203 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f65800ef410 con 0x7f65901038c0 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 -- 172.21.15.175:0/4231279195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 msgr2=0x558009c67960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f658fe00640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65740a2c00 0x7f65740a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65800b80e0 tx=0x7f65800f2e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c67960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6588009f70 tx=0x7f6588010800 comp rx=0 tx=0).stop 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 -- 172.21.15.175:0/4231279195 shutdown_connections 2024-07-29T00:36:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c72760 0x558009c74b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c72220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 --2- 172.21.15.175:0/4231279195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c67960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.417+0000 7f6599872740 1 -- 172.21.15.175:0/4231279195 >> 172.21.15.175:0/4231279195 conn(0x558009c63b10 msgr2=0x558009c63ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:21.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.418+0000 7f6599872740 1 -- 172.21.15.175:0/4231279195 shutdown_connections 2024-07-29T00:36:21.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.418+0000 7f6599872740 1 -- 172.21.15.175:0/4231279195 wait complete. 2024-07-29T00:36:21.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.418+0000 7f6599872740 1 -- start start 2024-07-29T00:36:21.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.418+0000 7f6599872740 1 Processor -- start 2024-07-29T00:36:21.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c6f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c6fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c72760 0x558009c70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558009c75330 con 0x558009c67580 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558009b88960 con 0x558009c72760 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558009c754b0 con 0x558009c67ea0 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6599872740 10 client.0 _ll_register_callbacks cb 0x5580098dc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6594e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c6f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f658f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c6fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6594e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c6f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55428/0 (socket says 172.21.15.175:55428) 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6594e00640 1 -- 172.21.15.175:0/2247406953 learned_addr learned my addr 172.21.15.175:0/2247406953 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.419+0000 7f6594e00640 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c72760 0x558009c70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f6594e00640 1 -- 172.21.15.175:0/2247406953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 msgr2=0x558009c6fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f6594e00640 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c6fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f6594e00640 1 -- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c72760 msgr2=0x558009c70100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f6594e00640 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c72760 0x558009c70100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f6594e00640 1 -- 172.21.15.175:0/2247406953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558009b97a90 con 0x558009c67580 2024-07-29T00:36:21.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f6594e00640 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c6f680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6588004bc0 tx=0x7f65880124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ec9bfe383bdaaf server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:21.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.420+0000 7f657a000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f657a000640 1 -- 172.21.15.175:0/2247406953 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588016070 con 0x558009c67580 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f657a000640 1 -- 172.21.15.175:0/2247406953 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f6588003b70 con 0x558009c67580 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6588004d20 con 0x558009c67580 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f657a000640 1 -- 172.21.15.175:0/2247406953 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6588023040 con 0x558009c67580 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f6599872740 10 client.15009 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558009c80ee0 con 0x558009c67580 2024-07-29T00:36:21.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.421+0000 7f657a000640 1 -- 172.21.15.175:0/2247406953 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6588004d20 con 0x558009c67580 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f657a000640 1 -- 172.21.15.175:0/2247406953 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f658800f040 con 0x558009c67580 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f6599872740 10 client.15009 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f6599872740 10 client.15009 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f6599872740 20 client.15009 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f6599872740 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009c83270 0x558009c846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f6599872740 4 client.15009 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558009bdad00 con 0x558009c83270 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.422+0000 7f658f400640 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009c83270 0x558009c846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.423+0000 7f657a000640 10 client.15009 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:21.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.423+0000 7f658f400640 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009c83270 0x558009c846e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12cb233b61bff6a1 server_cookie=bb8c5ebe0b116a7d in_seq=0 out_seq=0 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f657a000640 1 -- 172.21.15.175:0/2247406953 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558009bdad00 con 0x558009c83270 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f657a000640 10 client.15009 handle_command_reply: tid=0 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 client.15009 shutdown 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 msgr2=0x558009c6f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c6f680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6588004bc0 tx=0x7f65880124f0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 shutdown_connections 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009c83270 0x558009c846e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c72760 0x558009c70100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c67ea0 0x558009c6fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 --2- 172.21.15.175:0/2247406953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c6f680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f657a000640 0 client.15009 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 >> 172.21.15.175:0/2247406953 conn(0x558009c63b10 msgr2=0x558009c697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 shutdown_connections 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 1 -- 172.21.15.175:0/2247406953 wait complete. 2024-07-29T00:36:21.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.424+0000 7f6599872740 20 client.15009 trim_cache size 0 max 16384 2024-07-29T00:36:21.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 Processor -- start 2024-07-29T00:36:21.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 -- start start 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67ea0 0x558009c6df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c6e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558009c82a20 con 0x558009c67580 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558009c830f0 con 0x558009c67ea0 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.444+0000 7f6599872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558009c837c0 con 0x558009c72760 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f6594e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f658f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67ea0 0x558009c6df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f6594e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c846e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55430/0 (socket says 172.21.15.175:55430) 2024-07-29T00:36:21.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f6594e00640 1 -- 172.21.15.175:0/1198525520 learned_addr learned my addr 172.21.15.175:0/1198525520 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:21.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f6594e00640 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c6e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f658f400640 1 -- 172.21.15.175:0/1198525520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 msgr2=0x558009c6e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f658f400640 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c6e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f658f400640 1 -- 172.21.15.175:0/1198525520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 msgr2=0x558009c846e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f658f400640 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c846e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f658f400640 1 -- 172.21.15.175:0/1198525520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558009c620d0 con 0x558009c67ea0 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.445+0000 7f6594e00640 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c6e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:21.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.446+0000 7f658f400640 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67ea0 0x558009c6df50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x558009c6ef10 tx=0x7f657c014840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83382e14add42191 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.446+0000 7f6584c00640 1 -- 172.21.15.175:0/1198525520 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c014e80 con 0x558009c67ea0 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.446+0000 7f6584c00640 1 -- 172.21.15.175:0/1198525520 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c005500 con 0x558009c67ea0 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.446+0000 7f6584c00640 1 -- 172.21.15.175:0/1198525520 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c012450 con 0x558009c67ea0 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 -- 172.21.15.175:0/1198525520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67ea0 msgr2=0x558009c6df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67ea0 0x558009c6df50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x558009c6ef10 tx=0x7f657c014840 comp rx=0 tx=0).stop 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 -- 172.21.15.175:0/1198525520 shutdown_connections 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c6e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67ea0 0x558009c6df50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 --2- 172.21.15.175:0/1198525520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67580 0x558009c846e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 -- 172.21.15.175:0/1198525520 >> 172.21.15.175:0/1198525520 conn(0x558009c63b10 msgr2=0x558009c6a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 -- 172.21.15.175:0/1198525520 shutdown_connections 2024-07-29T00:36:21.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.447+0000 7f6599872740 1 -- 172.21.15.175:0/1198525520 wait complete. 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 -- start start 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 Processor -- start 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c6c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67ea0 0x558009c6c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c8b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558009c84560 con 0x558009c67ea0 2024-07-29T00:36:21.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558009c83e90 con 0x558009c67580 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558009c84c30 con 0x558009c72760 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.448+0000 7f6599872740 10 client.0 _ll_register_callbacks cb 0x7f65800f2d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c6c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c6c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53780/0 (socket says 172.21.15.175:53780) 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 -- 172.21.15.175:0/257902547 learned_addr learned my addr 172.21.15.175:0/257902547 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f658f400640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67ea0 0x558009c6c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c8b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 -- 172.21.15.175:0/257902547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 msgr2=0x558009c8b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c8b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 -- 172.21.15.175:0/257902547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67ea0 msgr2=0x558009c6c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67ea0 0x558009c6c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 -- 172.21.15.175:0/257902547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558009c8be80 con 0x558009c67580 2024-07-29T00:36:21.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f6594e00640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c6c050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6588010d00 tx=0x7f6588010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b950d1aa6ca9a610 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.449+0000 7f657a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f657a000640 1 -- 172.21.15.175:0/257902547 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588016070 con 0x558009c67580 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657c005500 con 0x558009c67580 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f657a000640 1 -- 172.21.15.175:0/257902547 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f6588003d80 con 0x558009c67580 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f657a000640 1 -- 172.21.15.175:0/257902547 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65880050e0 con 0x558009c67580 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f6599872740 10 client.25112 fetch_fsmap learned FSMap version 28 2024-07-29T00:36:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558009c8d160 con 0x558009c67580 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.450+0000 7f657a000640 1 -- 172.21.15.175:0/257902547 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c005500 con 0x558009c67580 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f657a000640 1 -- 172.21.15.175:0/257902547 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f658803e030 con 0x558009c67580 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f6599872740 10 client.25112 fetch_fsmap finished waiting for FSMap version 28 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f6599872740 10 client.25112 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f6599872740 20 client.25112 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f6599872740 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009cf1e10 0x558009cf0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f6599872740 4 client.25112 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "status"}) -- 0x558009bd5210 con 0x558009cf1e10 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.451+0000 7f658f400640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009cf1e10 0x558009cf0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f658f400640 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009cf1e10 0x558009cf0b20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6fc3bf097a0317 server_cookie=140b1175f522c452 in_seq=0 out_seq=0 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f657a000640 10 client.25112 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:21.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f657a000640 1 -- 172.21.15.175:0/257902547 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558009bd5210 con 0x558009cf1e10 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f657a000640 10 client.25112 handle_command_reply: tid=0 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 client.25112 shutdown 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 msgr2=0x558009c6c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c6c050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6588010d00 tx=0x7f6588010fd0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 shutdown_connections 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558009cf1e10 0x558009cf0b20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f657a000640 0 client.25112 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558009c72760 0x558009c8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558009c67ea0 0x558009c6c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 --2- 172.21.15.175:0/257902547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558009c67580 0x558009c6c050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 >> 172.21.15.175:0/257902547 conn(0x558009c63b10 msgr2=0x558009c6db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 shutdown_connections 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.452+0000 7f6599872740 1 -- 172.21.15.175:0/257902547 wait complete. 2024-07-29T00:36:21.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6599872740 20 client.25112 trim_cache size 0 max 16384 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 0, 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24539, 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 142.71463597600001, 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 28, 2024-07-29T00:36:21.456 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 91, 2024-07-29T00:36:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 87, 2024-07-29T00:36:21.457 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 159.030627522 2024-07-29T00:36:21.457 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:21.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65740a2c00 msgr2=0x7f65740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65740a2c00 0x7f65740a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65800b80e0 tx=0x7f65800f2e50 comp rx=0 tx=0).stop 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 msgr2=0x7f659019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 0x7f659019a170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f658002a940 tx=0x7f6580003b90 comp rx=0 tx=0).stop 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 shutdown_connections 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f65740a2c00 0x7f65740a50c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65901098c0 0x7f659019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6590104260 0x7f659019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 --2- 172.21.15.175:0/2029440203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65901038c0 0x7f659019a170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.453+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 >> 172.21.15.175:0/2029440203 conn(0x7f65900ff600 msgr2=0x7f6590100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.454+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 shutdown_connections 2024-07-29T00:36:21.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:21.454+0000 7f6595800640 1 -- 172.21.15.175:0/2029440203 wait complete. 2024-07-29T00:36:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:21 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:21 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:21 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:22.325 INFO:teuthology.run_tasks:Running task sequential... 2024-07-29T00:36:22.334 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-29T00:36:22.334 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-29T00:36:22.334 INFO:teuthology.task.print:Enabling mgr modules 2024-07-29T00:36:22.334 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-07-29T00:36:22.335 INFO:teuthology.task.exec:Executing custom commands... 2024-07-29T00:36:22.335 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:36:22.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-07-29T00:36:22.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 -- 172.21.15.175:0/1785362130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840103400 msgr2=0x7f2840109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:22.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 --2- 172.21.15.175:0/1785362130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840103400 0x7f2840109c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2838009ca0 tx=0x7f283802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:36:22.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 -- 172.21.15.175:0/1785362130 shutdown_connections 2024-07-29T00:36:22.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 --2- 172.21.15.175:0/1785362130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840103400 0x7f2840109c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:22.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 --2- 172.21.15.175:0/1785362130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840102a40 0x7f2840102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:22.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 --2- 172.21.15.175:0/1785362130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840101850 0x7f2840101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:22.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 -- 172.21.15.175:0/1785362130 >> 172.21.15.175:0/1785362130 conn(0x7f28400fcfd0 msgr2=0x7f28400ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:22.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 -- 172.21.15.175:0/1785362130 shutdown_connections 2024-07-29T00:36:22.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.520+0000 7f2846600640 1 -- 172.21.15.175:0/1785362130 wait complete. 2024-07-29T00:36:22.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.521+0000 7f2846600640 1 Processor -- start 2024-07-29T00:36:22.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.521+0000 7f2846600640 1 -- start start 2024-07-29T00:36:22.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.521+0000 7f2846600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 0x7f2840199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:22.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.521+0000 7f2846600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840102a40 0x7f284019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:22.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.521+0000 7f2846600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840103400 0x7f28401a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:22.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f2846600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f284010d4d0 con 0x7f2840103400 2024-07-29T00:36:22.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f2846600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f284010d350 con 0x7f2840101850 2024-07-29T00:36:22.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f2846600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f284010d650 con 0x7f2840102a40 2024-07-29T00:36:22.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840103400 0x7f28401a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:22.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 0x7f2840199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:22.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 0x7f2840199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53806/0 (socket says 172.21.15.175:53806) 2024-07-29T00:36:22.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840103400 0x7f28401a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55446/0 (socket says 172.21.15.175:55446) 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 -- 172.21.15.175:0/1975613055 learned_addr learned my addr 172.21.15.175:0/1975613055 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283ea00640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840102a40 0x7f284019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 -- 172.21.15.175:0/1975613055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840102a40 msgr2=0x7f284019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840102a40 0x7f284019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 -- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840103400 msgr2=0x7f28401a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840103400 0x7f28401a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:22.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.522+0000 7f283f400640 1 -- 172.21.15.175:0/1975613055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28401a1910 con 0x7f2840101850 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.523+0000 7f283f400640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 0x7f2840199d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f283000e9c0 tx=0x7f283000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.523+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283000cde0 con 0x7f2840101850 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.523+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28401a1c00 con 0x7f2840101850 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.523+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2830010910 con 0x7f2840101850 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.523+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830018a10 con 0x7f2840101850 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.523+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a21a0 con 0x7f2840101850 2024-07-29T00:36:22.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.524+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28401a2370 con 0x7f2840101850 2024-07-29T00:36:22.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.526+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f283001f020 con 0x7f2840101850 2024-07-29T00:36:22.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.527+0000 7f2837e00640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f28240a2cb0 0x7f28240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:22.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.528+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f28300cf3f0 con 0x7f2840101850 2024-07-29T00:36:22.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.528+0000 7f283ea00640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f28240a2cb0 0x7f28240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:22.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.529+0000 7f283ea00640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f28240a2cb0 0x7f28240a5170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f282c009bd0 tx=0x7f282c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:22.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.530+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f283009c2b0 con 0x7f2840101850 2024-07-29T00:36:22.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:22.809+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f28401a2700 con 0x7f2840101850 2024-07-29T00:36:22.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:22 smithi181 ceph-mon[36706]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:36:23.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:22 smithi175 ceph-mon[32138]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:36:23.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:22 smithi204 ceph-mon[37368]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-29T00:36:23.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.725+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f2830098190 con 0x7f2840101850 2024-07-29T00:36:23.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.727+0000 7f2837e00640 1 -- 172.21.15.175:0/1975613055 <== mon.2 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f28300d0440 con 0x7f2840101850 2024-07-29T00:36:23.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f28240a2cb0 msgr2=0x7f28240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:23.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f28240a2cb0 0x7f28240a5170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f282c009bd0 tx=0x7f282c005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:23.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 msgr2=0x7f2840199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:23.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 0x7f2840199d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f283000e9c0 tx=0x7f283000ee80 comp rx=0 tx=0).stop 2024-07-29T00:36:23.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 shutdown_connections 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2840103400 0x7f28401a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f28240a2cb0 0x7f28240a5170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2840102a40 0x7f284019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 --2- 172.21.15.175:0/1975613055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2840101850 0x7f2840199d00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 >> 172.21.15.175:0/1975613055 conn(0x7f28400fcfd0 msgr2=0x7f2840074110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 shutdown_connections 2024-07-29T00:36:23.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:23.732+0000 7f2846600640 1 -- 172.21.15.175:0/1975613055 wait complete. 2024-07-29T00:36:23.760 INFO:teuthology.run_tasks:Running task kclient... 2024-07-29T00:36:23.786 INFO:tasks.kclient:Mounting kernel clients... 2024-07-29T00:36:23.786 INFO:tasks.kclient:config is {'client.0': {}} 2024-07-29T00:36:23.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:23.786 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-29T00:36:23.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:23.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-07-29T00:36:23.939 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-07-29T00:36:23.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:36:23.942 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-07-29T00:36:23.942 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-29T00:36:23.942 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-29T00:36:23.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs ls 2024-07-29T00:36:23.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:23 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1975613055' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-29T00:36:23.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:23 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-29T00:36:23.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:23.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:23 smithi175 ceph-mon[32138]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:23.996 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ignoring --setuser ceph since I am not root 2024-07-29T00:36:23.996 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ignoring --setgroup ceph since I am not root 2024-07-29T00:36:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1975613055' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-29T00:36:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-29T00:36:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:24.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-mon[36706]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:24.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: ignoring --setuser ceph since I am not root 2024-07-29T00:36:24.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: ignoring --setgroup ceph since I am not root 2024-07-29T00:36:24.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-mgr[38075]: -- 172.21.15.181:0/2896925534 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557dcd44f0e0 con 0x557dcd42d000 2024-07-29T00:36:24.000 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:23.870+0000 7f2b58b4d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:36:24.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1975613055' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-29T00:36:24.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-29T00:36:24.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:24.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-mon[37368]: from='mgr.14150 172.21.15.175:0/1024837834' entity='mgr.x' 2024-07-29T00:36:24.018 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: ignoring --setuser ceph since I am not root 2024-07-29T00:36:24.018 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: ignoring --setgroup ceph since I am not root 2024-07-29T00:36:24.018 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-mgr[39196]: -- 172.21.15.204:0/2373688720 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56412bdc70e0 con 0x56412bda5000 2024-07-29T00:36:24.018 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:23.872+0000 7f2383d8d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:36:24.018 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:24.017+0000 7f2383d8d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:36:24.270 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:24.002+0000 7f724f9de040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-07-29T00:36:24.271 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:24.186+0000 7f724f9de040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:36:24.271 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:24.270+0000 7f724f9de040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:36:24.320 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:24.084+0000 7f2383d8d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:36:24.320 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:24.320+0000 7f2383d8d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:36:24.320 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:24.017+0000 7f2b58b4d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:36:24.320 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:24.084+0000 7f2b58b4d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-07-29T00:36:24.320 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:24.320+0000 7f2b58b4d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:36:24.383 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:24.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:24.378+0000 7f2383d8d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:36:24.594 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:24.382+0000 7f2b58b4d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:36:24.798 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:24.583+0000 7f724f9de040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-07-29T00:36:24.798 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:24.657+0000 7f724f9de040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-07-29T00:36:24.798 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-29T00:36:24.798 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:24 smithi175 ceph-mon[32138]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-07-29T00:36:24.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:24 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-29T00:36:24.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:24 smithi181 ceph-mon[36706]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-07-29T00:36:25.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.057+0000 7f3e1be00640 1 -- 172.21.15.175:0/2636977518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c106650 msgr2=0x7f3e1c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:25.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.057+0000 7f3e1be00640 1 --2- 172.21.15.175:0/2636977518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c106650 0x7f3e1c106a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009930 tx=0x7f3e0c02a420 comp rx=0 tx=0).stop 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 -- 172.21.15.175:0/2636977518 shutdown_connections 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 --2- 172.21.15.175:0/2636977518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c100f80 0x7f3e1c10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 --2- 172.21.15.175:0/2636977518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 --2- 172.21.15.175:0/2636977518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c106650 0x7f3e1c106a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 -- 172.21.15.175:0/2636977518 >> 172.21.15.175:0/2636977518 conn(0x7f3e1c0fc300 msgr2=0x7f3e1c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 -- 172.21.15.175:0/2636977518 shutdown_connections 2024-07-29T00:36:25.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.058+0000 7f3e1be00640 1 -- 172.21.15.175:0/2636977518 wait complete. 2024-07-29T00:36:25.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 Processor -- start 2024-07-29T00:36:25.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 -- start start 2024-07-29T00:36:25.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 0x7f3e1c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e1c106650 0x7f3e1c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c111a70 con 0x7f3e1c100f80 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1118f0 con 0x7f3e1c106650 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.059+0000 7f3e1be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c111bf0 con 0x7f3e1c1005c0 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 0x7f3e1c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58404/0 (socket says 172.21.15.175:58404) 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 -- 172.21.15.175:0/1284707156 learned_addr learned my addr 172.21.15.175:0/1284707156 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 0x7f3e1c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55494/0 (socket says 172.21.15.175:55494) 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1b400640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e1c106650 0x7f3e1c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:25.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 -- 172.21.15.175:0/1284707156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e1c106650 msgr2=0x7f3e1c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:25.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e1c106650 0x7f3e1c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 -- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 msgr2=0x7f3e1c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:25.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 0x7f3e1c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1aa00640 1 -- 172.21.15.175:0/1284707156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c19cea0 con 0x7f3e1c1005c0 2024-07-29T00:36:25.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.060+0000 7f3e1a000640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 0x7f3e1c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:25.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.061+0000 7f3e1aa00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c197ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009230 tx=0x7f3e0c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:25.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.061+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c03f030 con 0x7f3e1c1005c0 2024-07-29T00:36:25.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.061+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c19d130 con 0x7f3e1c1005c0 2024-07-29T00:36:25.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.061+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c004440 con 0x7f3e1c1005c0 2024-07-29T00:36:25.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.061+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c035070 con 0x7f3e1c1005c0 2024-07-29T00:36:25.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.061+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1a4a70 con 0x7f3e1c1005c0 2024-07-29T00:36:25.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.062+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1c100a40 con 0x7f3e1c1005c0 2024-07-29T00:36:25.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.064+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f3e0c042030 con 0x7f3e1c1005c0 2024-07-29T00:36:25.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.065+0000 7f3e20a00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 0x7f3e000a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:25.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.066+0000 7f3e1a000640 1 -- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 msgr2=0x7f3e000a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3246788131 2024-07-29T00:36:25.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.066+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7f3e0c0ef5c0 con 0x7f3e1c1005c0 2024-07-29T00:36:25.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.066+0000 7f3e1a000640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 0x7f3e000a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-29T00:36:25.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.067+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e0c0bbe30 con 0x7f3e1c1005c0 2024-07-29T00:36:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:24 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-29T00:36:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:24 smithi204 ceph-mon[37368]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-07-29T00:36:25.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.267+0000 7f3e1a000640 1 -- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 msgr2=0x7f3e000a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3246788131 2024-07-29T00:36:25.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.267+0000 7f3e1a000640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 0x7f3e000a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-29T00:36:25.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.295+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e1c108b30 con 0x7f3e1c1005c0 2024-07-29T00:36:25.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.296+0000 7f3e20a00640 1 -- 172.21.15.175:0/1284707156 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f3e0c0c0130 con 0x7f3e1c1005c0 2024-07-29T00:36:25.299 INFO:teuthology.orchestra.run.smithi175.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-29T00:36:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.299+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 msgr2=0x7f3e000a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-29T00:36:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.299+0000 7f3e1be00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 0x7f3e000a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.299+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 msgr2=0x7f3e1c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.299+0000 7f3e1be00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c197ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009230 tx=0x7f3e0c004040 comp rx=0 tx=0).stop 2024-07-29T00:36:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 shutdown_connections 2024-07-29T00:36:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e1c106650 0x7f3e1c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e1c100f80 0x7f3e1c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.175:6800/3246788131,v1:172.21.15.175:6801/3246788131] conn(0x7f3e000a2bd0 0x7f3e000a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 --2- 172.21.15.175:0/1284707156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e1c1005c0 0x7f3e1c197ef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:25.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 >> 172.21.15.175:0/1284707156 conn(0x7f3e1c0fc300 msgr2=0x7f3e1c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:25.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 shutdown_connections 2024-07-29T00:36:25.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:25.300+0000 7f3e1be00640 1 -- 172.21.15.175:0/1284707156 wait complete. 2024-07-29T00:36:25.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.307+0000 7f2383d8d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:36:25.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.423+0000 7f2383d8d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:36:25.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.479+0000 7f2383d8d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:36:25.718 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:25.354+0000 7f2b58b4d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:36:25.718 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:25.473+0000 7f2b58b4d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:36:25.718 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:25.532+0000 7f2b58b4d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:36:25.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1284707156' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-29T00:36:25.968 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.692+0000 7f2383d8d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:36:25.968 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.755+0000 7f2383d8d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:36:25.968 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.822+0000 7f2383d8d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:36:25.968 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:25.968+0000 7f2383d8d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:36:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1284707156' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-29T00:36:25.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:25.750+0000 7f2b58b4d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:36:25.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:25.810+0000 7f2b58b4d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:36:25.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:25.876+0000 7f2b58b4d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:36:26.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:25 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1284707156' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-29T00:36:26.034 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:25.911+0000 7f724f9de040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-07-29T00:36:26.198 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-29T00:36:26.198 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-29T00:36:26.198 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-07-29T00:36:26.198 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-29T00:36:26.198 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-29T00:36:26.198 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-29T00:36:26.199 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-29T00:36:26.199 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-29T00:36:26.199 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-29T00:36:26.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:26.199 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-07-29T00:36:26.251 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-29T00:36:26.251 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-29T00:36:26.252 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-07-29T00:36:26.253 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-29T00:36:26.253 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-07-29T00:36:26.253 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-29T00:36:26.253 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 41975sec preferred_lft 41975sec 2024-07-29T00:36:26.253 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-07-29T00:36:26.253 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-29T00:36:26.253 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add name ceph-brx type bridge 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr flush dev ceph-brx 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set ceph-brx up 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-29T00:36:26.254 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-29T00:36:26.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-29T00:36:26.378 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:26.050+0000 7f724f9de040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-07-29T00:36:26.378 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:26.118+0000 7f724f9de040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-07-29T00:36:26.432 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:26.021+0000 7f2b58b4d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:36:26.634 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:26.384+0000 7f724f9de040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-07-29T00:36:26.634 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:26.460+0000 7f724f9de040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-07-29T00:36:26.634 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:26.541+0000 7f724f9de040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-07-29T00:36:26.702 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:26.386+0000 7f2383d8d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:36:26.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:26.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:26.736 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-29T00:36:26.746 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:26.432+0000 7f2b58b4d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:36:26.800 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-07-29T00:36:26.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:26.804 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-07-29T00:36:26.856 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-07-29T00:36:26.856 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-29T00:36:26.856 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-29T00:36:26.856 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-29T00:36:26.856 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-29T00:36:26.857 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-29T00:36:26.857 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-29T00:36:26.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-29T00:36:26.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-29T00:36:26.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-29T00:36:26.858 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-29T00:36:26.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-29T00:36:26.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:26.718+0000 7f724f9de040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-07-29T00:36:26.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:26.747+0000 7f2b58b4d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:36:26.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:26.814+0000 7f2b58b4d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:36:26.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:26.964+0000 7f2b58b4d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:36:27.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:26.702+0000 7f2383d8d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:36:27.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:26.771+0000 7f2383d8d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:36:27.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:26.922+0000 7f2383d8d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:36:27.534 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:27.272+0000 7f724f9de040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-07-29T00:36:27.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:27.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.233+0000 7f2383d8d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:36:27.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.294+0000 7f2383d8d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:36:27.585 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.478+0000 7f2383d8d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:36:27.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:27.587 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-29T00:36:27.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:27.630 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-07-29T00:36:27.661 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.272+0000 7f2b58b4d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:36:27.661 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.332+0000 7f2b58b4d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:36:27.661 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.517+0000 7f2b58b4d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:36:27.661 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.660+0000 7f2b58b4d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:36:27.724 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-29T00:36:27.724 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-29T00:36:27.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-29T00:36:27.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-29T00:36:27.724 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-29T00:36:27.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-29T00:36:27.845 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:27.646+0000 7f724f9de040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-07-29T00:36:27.845 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:27.728+0000 7f724f9de040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-07-29T00:36:27.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:27.898 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-29T00:36:27.898 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-29T00:36:27.898 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-29T00:36:27.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-29T00:36:27.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-29T00:36:27.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-29T00:36:27.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-29T00:36:27.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-29T00:36:27.899 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-29T00:36:27.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-29T00:36:27.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.724+0000 7f2b58b4d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:36:27.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.806+0000 7f2b58b4d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:36:27.999 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:27.865+0000 7f2b58b4d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:36:28.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.625+0000 7f2383d8d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:36:28.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.690+0000 7f2383d8d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:36:28.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.775+0000 7f2383d8d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:36:28.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:27.835+0000 7f2383d8d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:36:28.099 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:27.912+0000 7f724f9de040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-07-29T00:36:28.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:28.175 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-29T00:36:28.176 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-29T00:36:28.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-07-29T00:36:28.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.0 master ceph-brx 2024-07-29T00:36:28.176 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-29T00:36:28.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-29T00:36:28.355 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:28.299+0000 7f724f9de040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-07-29T00:36:28.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:28.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-29T00:36:28.367 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-29T00:36:28.408 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-29T00:36:28.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-29T00:36:28.409 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-29T00:36:28.499 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-07-29T00:36:28.499 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-07-29T00:36:28.500 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/790caf24-b417-4f46-948e-eb25ca9090b1 2024-07-29T00:36:28.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:28.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/790caf24-b417-4f46-948e-eb25ca9090b1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-07-29T00:36:28.616 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:28.376+0000 7f724f9de040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-07-29T00:36:28.616 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: Standby manager daemon z restarted 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: Standby manager daemon z started 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: Standby manager daemon y restarted 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: Standby manager daemon y started 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-29T00:36:28.617 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-mon[32138]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:36:28.626 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-07-29T00:36:28.626 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-07-29T00:36:28.626 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-29T00:36:28.749 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:28.394+0000 7f2b58b4d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:36:28.749 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y[38071]: 2024-07-29T00:36:28.551+0000 7f2b58b4d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:36:28.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: Standby manager daemon z restarted 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: Standby manager daemon z started 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: Standby manager daemon y restarted 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: Standby manager daemon y started 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-29T00:36:28.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:28 smithi181 ceph-mon[36706]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:36:28.835 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:28.371+0000 7f2383d8d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z[39192]: 2024-07-29T00:36:28.531+0000 7f2383d8d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: Standby manager daemon z restarted 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: Standby manager daemon z started 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.204:0/3719948869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: Standby manager daemon y restarted 2024-07-29T00:36:28.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: Standby manager daemon y started 2024-07-29T00:36:28.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-29T00:36:28.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-29T00:36:28.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-29T00:36:28.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:28 smithi204 ceph-mon[37368]: from='mgr.? 172.21.15.181:0/2488952569' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-29T00:36:28.891 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:28.615+0000 7f724f9de040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-07-29T00:36:28.891 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:28.812+0000 7f724f9de040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-07-29T00:36:28.891 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:28.890+0000 7f724f9de040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-07-29T00:36:29.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:28.989+0000 7f724f9de040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-07-29T00:36:29.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:29.063+0000 7f724f9de040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-07-29T00:36:29.434 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-07-29T00:36:29.434 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-07-29T00:36:29.434 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-29T00:36:29.434 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.175:3300,172.21.15.181:3300,172.21.15.204:3300" 2024-07-29T00:36:29.435 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.175:3300,172.21.15.181:3300,172.21.15.204:3300:/volumes/qa/sv_1/790caf24-b417-4f46-948e-eb25ca9090b1 2024-07-29T00:36:29.435 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=f484ff76-4d40-11ef-bcc8-c7b262605968" will pass to kernel 2024-07-29T00:36:29.443 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-29T00:36:29.443 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-07-29T00:36:29.443 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-29T00:36:29.443 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-07-29T00:36:29.444 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-29T00:36:29.449 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-29T00:36:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:36:29.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-29T00:36:29.527 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-29T00:36:29.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-29T00:36:29.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:29.642 INFO:teuthology.orchestra.run.smithi175.stdout:client15042 2024-07-29T00:36:29.643 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:36:29.643 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-29T00:36:29.692 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-29T00:36:29.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-29T00:36:29.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:29.850 INFO:teuthology.orchestra.run.smithi175.stdout:client15042 2024-07-29T00:36:29.850 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-29T00:36:29.906 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:29 smithi175 ceph-mon[32138]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-07-29T00:36:29.907 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:29.716+0000 7f724f9de040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-07-29T00:36:29.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-29T00:36:29.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:29.968 INFO:teuthology.orchestra.run.smithi175.stdout:client15042 2024-07-29T00:36:29.969 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968.client15042/status') 2024-07-29T00:36:30.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:29 smithi181 ceph-mon[36706]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-07-29T00:36:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-29T00:36:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:29 smithi204 ceph-mon[37368]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-07-29T00:36:30.114 INFO:teuthology.orchestra.run.smithi175.stdout:instance: client.15042 (3)192.168.0.1:0/562942242 2024-07-29T00:36:30.114 INFO:teuthology.orchestra.run.smithi175.stdout:blocklisted: false 2024-07-29T00:36:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2024-07-29T00:36:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2024-07-29T00:36:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:69 bytes copied, 9.5931e-05 s, 719 kB/s 2024-07-29T00:36:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-29T00:36:30.131 INFO:teuthology.run_tasks:Running task exec... 2024-07-29T00:36:30.139 INFO:teuthology.task.exec:Executing custom commands... 2024-07-29T00:36:30.140 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:36:30.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-07-29T00:36:30.222 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:29.906+0000 7f724f9de040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-07-29T00:36:30.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.376+0000 7fdf2d400640 1 -- 172.21.15.175:0/3227236145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf281027f0 msgr2=0x7fdf28104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:30.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.376+0000 7fdf2d400640 1 --2- 172.21.15.175:0/3227236145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf281027f0 0x7fdf28104c60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009900 tx=0x7fdf1402a3f0 comp rx=0 tx=0).stop 2024-07-29T00:36:30.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 -- 172.21.15.175:0/3227236145 shutdown_connections 2024-07-29T00:36:30.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 --2- 172.21.15.175:0/3227236145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf281051a0 0x7fdf28105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:30.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 --2- 172.21.15.175:0/3227236145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf281027f0 0x7fdf28104c60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:30.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 --2- 172.21.15.175:0/3227236145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf280ffea0 0x7fdf281022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:30.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 -- 172.21.15.175:0/3227236145 >> 172.21.15.175:0/3227236145 conn(0x7fdf280fbc90 msgr2=0x7fdf280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:30.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 -- 172.21.15.175:0/3227236145 shutdown_connections 2024-07-29T00:36:30.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.377+0000 7fdf2d400640 1 -- 172.21.15.175:0/3227236145 wait complete. 2024-07-29T00:36:30.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.378+0000 7fdf2d400640 1 Processor -- start 2024-07-29T00:36:30.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.378+0000 7fdf2d400640 1 -- start start 2024-07-29T00:36:30.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.378+0000 7fdf2d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 0x7fdf2819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:30.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.378+0000 7fdf2d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf281027f0 0x7fdf2819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:30.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf2d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf281051a0 0x7fdf281a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf2d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281138f0 con 0x7fdf281027f0 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf2d400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28113770 con 0x7fdf281051a0 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf2d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28113a70 con 0x7fdf280ffea0 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf281027f0 0x7fdf2819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 0x7fdf2819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf2ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf281051a0 0x7fdf281a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 0x7fdf2819df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33538/0 (socket says 172.21.15.175:33538) 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 -- 172.21.15.175:0/1825462161 learned_addr learned my addr 172.21.15.175:0/1825462161 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:30.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 -- 172.21.15.175:0/1825462161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf281051a0 msgr2=0x7fdf281a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf281051a0 0x7fdf281a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 -- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf281027f0 msgr2=0x7fdf2819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf281027f0 0x7fdf2819e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.379+0000 7fdf27e00640 1 -- 172.21.15.175:0/1825462161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf281a5ba0 con 0x7fdf280ffea0 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.380+0000 7fdf27e00640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 0x7fdf2819df80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9f0 tx=0x7fdf1800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.380+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cd80 con 0x7fdf280ffea0 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.380+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18005500 con 0x7fdf280ffea0 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.380+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18010690 con 0x7fdf280ffea0 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.380+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf281a5e90 con 0x7fdf280ffea0 2024-07-29T00:36:30.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.380+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281a62a0 con 0x7fdf280ffea0 2024-07-29T00:36:30.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.381+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf280fd780 con 0x7fdf280ffea0 2024-07-29T00:36:30.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.383+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7fdf180046c0 con 0x7fdf280ffea0 2024-07-29T00:36:30.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.384+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdf180ceff0 con 0x7fdf280ffea0 2024-07-29T00:36:30.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.388+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdf1809c210 con 0x7fdf280ffea0 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.254+0000 7f71aaa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.254+0000 7f71aaa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.254+0000 7f71aaa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.254+0000 7f71aaa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.254+0000 7f71aaa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.256+0000 7f71a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.256+0000 7f71a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.256+0000 7f71a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.476 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.256+0000 7f71a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.477 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:30.256+0000 7f71a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:36:30.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.663+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fdf28062bb0 con 0x7fdf280ffea0 2024-07-29T00:36:30.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: Active manager daemon x restarted 2024-07-29T00:36:30.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: Activating manager daemon x 2024-07-29T00:36:30.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: osdmap e92: 12 total, 12 up, 12 in 2024-07-29T00:36:30.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: mgrmap e21: x(active, starting, since 0.0233267s), standbys: z, y 2024-07-29T00:36:30.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:36:30.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:36:30.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: Manager daemon x is now available 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:36:30.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:36:30.789 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:30 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:36:30.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.934+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fdf18097860 con 0x7fdf280ffea0 2024-07-29T00:36:30.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.934+0000 7fdf24c00640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf0c0a9a00 0x7fdf0c0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:30.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.936+0000 7fdf27400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf0c0a9a00 0x7fdf0c0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:30.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:30.942+0000 7fdf27400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf0c0a9a00 0x7fdf0c0abdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf14031440 tx=0x7fdf140315b0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:30.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: Active manager daemon x restarted 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: Activating manager daemon x 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: osdmap e92: 12 total, 12 up, 12 in 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: mgrmap e21: x(active, starting, since 0.0233267s), standbys: z, y 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:36:31.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:36:31.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: Manager daemon x is now available 2024-07-29T00:36:31.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:31.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:36:31.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:36:31.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:36:31.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:30 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: Active manager daemon x restarted 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: Activating manager daemon x 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: osdmap e92: 12 total, 12 up, 12 in 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: mgrmap e21: x(active, starting, since 0.0233267s), standbys: z, y 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-29T00:36:31.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-29T00:36:31.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-29T00:36:31.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: Manager daemon x is now available 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:36:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:30 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-29T00:36:31.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.609+0000 7fdf24c00640 1 -- 172.21.15.175:0/1825462161 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7fdf18092e00 con 0x7fdf280ffea0 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.612+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf0c0a9a00 msgr2=0x7fdf0c0abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.612+0000 7fdf2d400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf0c0a9a00 0x7fdf0c0abdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf14031440 tx=0x7fdf140315b0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.612+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 msgr2=0x7fdf2819df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.612+0000 7fdf2d400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 0x7fdf2819df80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9f0 tx=0x7fdf1800edf0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 shutdown_connections 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf281051a0 0x7fdf281a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf0c0a9a00 0x7fdf0c0abdf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf281027f0 0x7fdf2819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 --2- 172.21.15.175:0/1825462161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf280ffea0 0x7fdf2819df80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 >> 172.21.15.175:0/1825462161 conn(0x7fdf280fbc90 msgr2=0x7fdf28100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:31.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 shutdown_connections 2024-07-29T00:36:31.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.613+0000 7fdf2d400640 1 -- 172.21.15.175:0/1825462161 wait complete. 2024-07-29T00:36:31.632 INFO:teuthology.run_tasks:Running task check-counter... 2024-07-29T00:36:31.663 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-07-29T00:36:31.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-29T00:36:31.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 -- 172.21.15.175:0/3009941239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48102a10 msgr2=0x7fce48102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 --2- 172.21.15.175:0/3009941239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48102a10 0x7fce48102e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce34009900 tx=0x7fce3402a3f0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 -- 172.21.15.175:0/3009941239 shutdown_connections 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 --2- 172.21.15.175:0/3009941239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce481033d0 0x7fce48109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 --2- 172.21.15.175:0/3009941239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48102a10 0x7fce48102e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 --2- 172.21.15.175:0/3009941239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce48101820 0x7fce48101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 -- 172.21.15.175:0/3009941239 >> 172.21.15.175:0/3009941239 conn(0x7fce480fcfa0 msgr2=0x7fce480ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 -- 172.21.15.175:0/3009941239 shutdown_connections 2024-07-29T00:36:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.883+0000 7fce4fe00640 1 -- 172.21.15.175:0/3009941239 wait complete. 2024-07-29T00:36:31.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.884+0000 7fce4fe00640 1 Processor -- start 2024-07-29T00:36:31.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.884+0000 7fce4fe00640 1 -- start start 2024-07-29T00:36:31.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 0x7fce48199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:31.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce48102a10 0x7fce4819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:31.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce481033d0 0x7fce481a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce4810d4d0 con 0x7fce481033d0 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce4810d350 con 0x7fce48101820 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce4810d650 con 0x7fce48102a10 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 0x7fce48199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 0x7fce48199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:44480/0 (socket says 172.21.15.175:44480) 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 -- 172.21.15.175:0/844981687 learned_addr learned my addr 172.21.15.175:0/844981687 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4de00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce481033d0 0x7fce481a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4ca00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce48102a10 0x7fce4819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 -- 172.21.15.175:0/844981687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce48102a10 msgr2=0x7fce4819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce48102a10 0x7fce4819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 -- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce481033d0 msgr2=0x7fce481a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce481033d0 0x7fce481a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.885+0000 7fce4d400640 1 -- 172.21.15.175:0/844981687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481a1970 con 0x7fce48101820 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.886+0000 7fce4d400640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 0x7fce48199d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e9c0 tx=0x7fce3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.886+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00cd80 con 0x7fce48101820 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.886+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce3c0035f0 con 0x7fce48101820 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.886+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481a1c60 con 0x7fce48101820 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.886+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c010690 con 0x7fce48101820 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.886+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481a21d0 con 0x7fce48101820 2024-07-29T00:36:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.887+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce481a23a0 con 0x7fce48101820 2024-07-29T00:36:31.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.889+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fce3c0037a0 con 0x7fce48101820 2024-07-29T00:36:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.890+0000 7fce46000640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce300a2a10 0x7fce300a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.891+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fce3c0cf0e0 con 0x7fce48101820 2024-07-29T00:36:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.891+0000 7fce4ca00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce300a2a10 0x7fce300a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.892+0000 7fce4ca00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce300a2a10 0x7fce300a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fce34031440 tx=0x7fce340315b0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:31.892+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fce3c09c1b0 con 0x7fce48101820 2024-07-29T00:36:31.964 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1825462161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-29T00:36:31.964 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-29T00:36:31.964 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-mon[32138]: mgrmap e22: x(active, since 1.02067s), standbys: z, y 2024-07-29T00:36:31.964 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:31.677+0000 7f7188e00640 -1 log_channel(cephadm) log [ERR] : [29/Jul/2024:00:36:31] ENGINE Error in HTTPServer.serve 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: Traceback (most recent call last): 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._connections.run(self.expiration_interval) 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._run(expiration_interval) 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:36:31.965 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: s = self.context.wrap_socket( 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: return self.sslsocket_class._create( 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self.do_handshake() 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: self._sslobj.do_handshake() 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:36:31.966 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:36:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:36:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:31 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1825462161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-29T00:36:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:31 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-29T00:36:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:31 smithi181 ceph-mon[36706]: mgrmap e22: x(active, since 1.02067s), standbys: z, y 2024-07-29T00:36:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:31 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1825462161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-29T00:36:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:31 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-29T00:36:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:31 smithi204 ceph-mon[37368]: mgrmap e22: x(active, since 1.02067s), standbys: z, y 2024-07-29T00:36:32.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.127+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce481a2570 con 0x7fce48101820 2024-07-29T00:36:32.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.129+0000 7fce46000640 1 -- 172.21.15.175:0/844981687 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+23219 (secure 0 0 0) 0x7fce3c09bab0 con 0x7fce48101820 2024-07-29T00:36:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:32.130 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":92,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:36:29.909354+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1189171033":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/3246788131":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/257505941":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/2019123994":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/4008532755":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/3246788131":"2024-07-30T00:36:29.909303+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:36:32.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce300a2a10 msgr2=0x7fce300a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce300a2a10 0x7fce300a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fce34031440 tx=0x7fce340315b0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 msgr2=0x7fce48199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 0x7fce48199d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e9c0 tx=0x7fce3c00ee80 comp rx=0 tx=0).stop 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 shutdown_connections 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce300a2a10 0x7fce300a4ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce481033d0 0x7fce481a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce48102a10 0x7fce4819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 --2- 172.21.15.175:0/844981687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce48101820 0x7fce48199d50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 >> 172.21.15.175:0/844981687 conn(0x7fce480fcfa0 msgr2=0x7fce48074110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 shutdown_connections 2024-07-29T00:36:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.132+0000 7fce4fe00640 1 -- 172.21.15.175:0/844981687 wait complete. 2024-07-29T00:36:32.150 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-29T00:29:04.051354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-29T00:33:52.192870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-29T00:33:55.363149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-29T00:36:32.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-29T00:36:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 -- 172.21.15.175:0/1673414986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0104ac0 msgr2=0x7fd3d0106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/1673414986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0104ac0 0x7fd3d0106eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009900 tx=0x7fd3bc02a410 comp rx=0 tx=0).stop 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 -- 172.21.15.175:0/1673414986 shutdown_connections 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/1673414986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d01073f0 0x7fd3d0109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/1673414986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0104ac0 0x7fd3d0106eb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/1673414986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0102190 0x7fd3d0104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 -- 172.21.15.175:0/1673414986 >> 172.21.15.175:0/1673414986 conn(0x7fd3d00fbd50 msgr2=0x7fd3d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 -- 172.21.15.175:0/1673414986 shutdown_connections 2024-07-29T00:36:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.354+0000 7fd3d4a00640 1 -- 172.21.15.175:0/1673414986 wait complete. 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.355+0000 7fd3d4a00640 1 Processor -- start 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.355+0000 7fd3d4a00640 1 -- start start 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 0x7fd3d0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d0104ac0 0x7fd3d019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d01073f0 0x7fd3d01a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010d3f0 con 0x7fd3d01073f0 2024-07-29T00:36:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010d270 con 0x7fd3d0104ac0 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010d570 con 0x7fd3d0102190 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 0x7fd3d0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d0104ac0 0x7fd3d019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 0x7fd3d0199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33580/0 (socket says 172.21.15.175:33580) 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 -- 172.21.15.175:0/705334968 learned_addr learned my addr 172.21.15.175:0/705334968 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cfe00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d01073f0 0x7fd3d01a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 -- 172.21.15.175:0/705334968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d0104ac0 msgr2=0x7fd3d019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d0104ac0 0x7fd3d019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 -- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d01073f0 msgr2=0x7fd3d01a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d01073f0 0x7fd3d01a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cf400640 1 -- 172.21.15.175:0/705334968 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01a1960 con 0x7fd3d0102190 2024-07-29T00:36:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.356+0000 7fd3cfe00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d01073f0 0x7fd3d01a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.357+0000 7fd3cf400640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 0x7fd3d0199d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000d950 tx=0x7fd3c000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.357+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0017070 con 0x7fd3d0102190 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.357+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01a1c50 con 0x7fd3d0102190 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.357+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c0009c30 con 0x7fd3d0102190 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.357+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000f6b0 con 0x7fd3d0102190 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.357+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a21a0 con 0x7fd3d0102190 2024-07-29T00:36:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.358+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d0108f30 con 0x7fd3d0102190 2024-07-29T00:36:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.360+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fd3c0010460 con 0x7fd3d0102190 2024-07-29T00:36:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.361+0000 7fd3c7e00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd3b40a2990 0x7fd3b40a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.362+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd3c00ceab0 con 0x7fd3d0102190 2024-07-29T00:36:32.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.362+0000 7fd3cea00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd3b40a2990 0x7fd3b40a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.363+0000 7fd3cea00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd3b40a2990 0x7fd3b40a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc005ab0 tx=0x7fd3bc005a20 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:32.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.366+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd3c009bb80 con 0x7fd3d0102190 2024-07-29T00:36:32.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.573+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3d002d7e0 con 0x7fd3d0102190 2024-07-29T00:36:32.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.573+0000 7fd3c7e00640 1 -- 172.21.15.175:0/705334968 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fd3c0009de0 con 0x7fd3d0102190 2024-07-29T00:36:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 1 2024-07-29T00:36:32.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd3b40a2990 msgr2=0x7fd3b40a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd3b40a2990 0x7fd3b40a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc005ab0 tx=0x7fd3bc005a20 comp rx=0 tx=0).stop 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 msgr2=0x7fd3d0199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 0x7fd3d0199d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000d950 tx=0x7fd3c000de10 comp rx=0 tx=0).stop 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 shutdown_connections 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd3b40a2990 0x7fd3b40a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d01073f0 0x7fd3d01a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d0104ac0 0x7fd3d019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 --2- 172.21.15.175:0/705334968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0102190 0x7fd3d0199d50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 >> 172.21.15.175:0/705334968 conn(0x7fd3d00fbd50 msgr2=0x7fd3d0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:32.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 shutdown_connections 2024-07-29T00:36:32.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.576+0000 7fd3d4a00640 1 -- 172.21.15.175:0/705334968 wait complete. 2024-07-29T00:36:32.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-29T00:36:32.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 -- 172.21.15.175:0/2260362619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c4100a30 msgr2=0x7f82c4100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 --2- 172.21.15.175:0/2260362619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c4100a30 0x7f82c4100e90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009860 tx=0x7f82b002a3d0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 -- 172.21.15.175:0/2260362619 shutdown_connections 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 --2- 172.21.15.175:0/2260362619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41013d0 0x7f82c4101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 --2- 172.21.15.175:0/2260362619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c4100a30 0x7f82c4100e90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 --2- 172.21.15.175:0/2260362619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c41000f0 0x7f82c41004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 -- 172.21.15.175:0/2260362619 >> 172.21.15.175:0/2260362619 conn(0x7f82c40fbe80 msgr2=0x7f82c40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 -- 172.21.15.175:0/2260362619 shutdown_connections 2024-07-29T00:36:32.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.799+0000 7f82cc200640 1 -- 172.21.15.175:0/2260362619 wait complete. 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.800+0000 7f82cc200640 1 Processor -- start 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.800+0000 7f82cc200640 1 -- start start 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82cc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41000f0 0x7f82c4199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82cc200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c4100a30 0x7f82c419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82cc200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c41013d0 0x7f82c41a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82cc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410f6d0 con 0x7f82c41000f0 2024-07-29T00:36:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82cc200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410f550 con 0x7f82c41013d0 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82cc200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410f850 con 0x7f82c4100a30 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82c9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41000f0 0x7f82c4199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82c9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41000f0 0x7f82c4199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57228/0 (socket says 172.21.15.175:57228) 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82c9800640 1 -- 172.21.15.175:0/2888354641 learned_addr learned my addr 172.21.15.175:0/2888354641 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82ca200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c41013d0 0x7f82c41a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82c8e00640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c4100a30 0x7f82c419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82ca200640 1 -- 172.21.15.175:0/2888354641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c4100a30 msgr2=0x7f82c419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82ca200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c4100a30 0x7f82c419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82ca200640 1 -- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41000f0 msgr2=0x7f82c4199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82ca200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41000f0 0x7f82c4199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.801+0000 7f82ca200640 1 -- 172.21.15.175:0/2888354641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a19c0 con 0x7f82c41013d0 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.802+0000 7f82ca200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c41013d0 0x7f82c41a1350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00ef00 tx=0x7f82bc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.802+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004b20 con 0x7f82c41013d0 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.802+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc003e50 con 0x7f82c41013d0 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.802+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00b570 con 0x7f82c41013d0 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.802+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a1cb0 con 0x7f82c41013d0 2024-07-29T00:36:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.802+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a2180 con 0x7f82c41013d0 2024-07-29T00:36:32.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.803+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c410b080 con 0x7f82c41013d0 2024-07-29T00:36:32.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.805+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 23) ==== 137954+0+0 (secure 0 0 0) 0x7f82bc01d020 con 0x7f82c41013d0 2024-07-29T00:36:32.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.806+0000 7f82c2000640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f82ac0a2d00 0x7f82ac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:32.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.806+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f82bc0cf310 con 0x7f82c41013d0 2024-07-29T00:36:32.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.807+0000 7f82c9800640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f82ac0a2d00 0x7f82ac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:32.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.807+0000 7f82c9800640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f82ac0a2d00 0x7f82ac0a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009cd0 tx=0x7f82b8005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:32.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:32.810+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f82bc09beb0 con 0x7f82c41013d0 2024-07-29T00:36:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: [29/Jul/2024:00:36:31] ENGINE Bus STARTING 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: [29/Jul/2024:00:36:31] ENGINE Serving on http://172.21.15.175:8765 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: [29/Jul/2024:00:36:31] ENGINE Serving on https://172.21.15.175:7150 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: [29/Jul/2024:00:36:31] ENGINE Bus STARTED 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: [29/Jul/2024:00:36:31] ENGINE Error in HTTPServer.serve 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout: Traceback (most recent call last): 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:36:33.000 INFO:journalctl@ceph.mon.b.smithi181.stdout: self._connections.run(self.expiration_interval) 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: self._run(expiration_interval) 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:36:33.001 INFO:journalctl@ceph.mon.b.smithi181.stdout: s = self.context.wrap_socket( 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: return self.sslsocket_class._create( 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: self.do_handshake() 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: self._sslobj.do_handshake() 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:36:33.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/844981687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:33.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/705334968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:36:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: [29/Jul/2024:00:36:31] ENGINE Bus STARTING 2024-07-29T00:36:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: [29/Jul/2024:00:36:31] ENGINE Serving on http://172.21.15.175:8765 2024-07-29T00:36:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: [29/Jul/2024:00:36:31] ENGINE Serving on https://172.21.15.175:7150 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: [29/Jul/2024:00:36:31] ENGINE Bus STARTED 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: [29/Jul/2024:00:36:31] ENGINE Error in HTTPServer.serve 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: Traceback (most recent call last): 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._connections.run(self.expiration_interval) 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._run(expiration_interval) 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:36:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: s = self.context.wrap_socket( 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: return self.sslsocket_class._create( 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: self.do_handshake() 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: self._sslobj.do_handshake() 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/844981687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:33.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/705334968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:36:33.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.060+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82c402d750 con 0x7f82c41013d0 2024-07-29T00:36:33.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.060+0000 7f82c2000640 1 -- 172.21.15.175:0/2888354641 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f82bc09b7b0 con 0x7f82c41013d0 2024-07-29T00:36:33.060 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 32 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f82ac0a2d00 msgr2=0x7f82ac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f82ac0a2d00 0x7f82ac0a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009cd0 tx=0x7f82b8005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c41013d0 msgr2=0x7f82c41a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c41013d0 0x7f82c41a1350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00ef00 tx=0x7f82bc00c740 comp rx=0 tx=0).stop 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 shutdown_connections 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c41013d0 0x7f82c41a1350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c4100a30 0x7f82c419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f82ac0a2d00 0x7f82ac0a51c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.063+0000 7f82cc200640 1 --2- 172.21.15.175:0/2888354641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c41000f0 0x7f82c4199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.064+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 >> 172.21.15.175:0/2888354641 conn(0x7f82c40fbe80 msgr2=0x7f82c40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:33.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.064+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 shutdown_connections 2024-07-29T00:36:33.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.064+0000 7f82cc200640 1 -- 172.21.15.175:0/2888354641 wait complete. 2024-07-29T00:36:33.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-29T00:36:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: [29/Jul/2024:00:36:31] ENGINE Bus STARTING 2024-07-29T00:36:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: [29/Jul/2024:00:36:31] ENGINE Serving on http://172.21.15.175:8765 2024-07-29T00:36:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: [29/Jul/2024:00:36:31] ENGINE Serving on https://172.21.15.175:7150 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: [29/Jul/2024:00:36:31] ENGINE Bus STARTED 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: [29/Jul/2024:00:36:31] ENGINE Error in HTTPServer.serve 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout: Traceback (most recent call last): 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-29T00:36:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._connections.run(self.expiration_interval) 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._run(expiration_interval) 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-29T00:36:33.087 INFO:journalctl@ceph.mon.c.smithi204.stdout: s = self.context.wrap_socket( 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: return self.sslsocket_class._create( 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: self.do_handshake() 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._sslobj.do_handshake() 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/844981687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/705334968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 -- 172.21.15.175:0/3313791261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d50100070 msgr2=0x7f0d50100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3313791261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d50100070 0x7f0d50100470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009900 tx=0x7f0d4002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 -- 172.21.15.175:0/3313791261 shutdown_connections 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3313791261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50101350 0x7f0d50101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3313791261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d501009b0 0x7f0d50100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3313791261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d50100070 0x7f0d50100470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 -- 172.21.15.175:0/3313791261 >> 172.21.15.175:0/3313791261 conn(0x7f0d500fbe80 msgr2=0x7f0d500fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:33.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 -- 172.21.15.175:0/3313791261 shutdown_connections 2024-07-29T00:36:33.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.281+0000 7f0d55e00640 1 -- 172.21.15.175:0/3313791261 wait complete. 2024-07-29T00:36:33.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.282+0000 7f0d55e00640 1 Processor -- start 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 -- start start 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50100070 0x7f0d50199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d501009b0 0x7f0d5019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 0x7f0d501a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010f670 con 0x7f0d501009b0 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010f4f0 con 0x7f0d50101350 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.283+0000 7f0d55e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010f7f0 con 0x7f0d50100070 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50100070 0x7f0d50199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 0x7f0d501a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 0x7f0d501a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:44560/0 (socket says 172.21.15.175:44560) 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50100070 0x7f0d50199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33610/0 (socket says 172.21.15.175:33610) 2024-07-29T00:36:33.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 -- 172.21.15.175:0/3076418841 learned_addr learned my addr 172.21.15.175:0/3076418841 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:33.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4e000640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d501009b0 0x7f0d5019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:33.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 -- 172.21.15.175:0/3076418841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50100070 msgr2=0x7f0d50199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50100070 0x7f0d50199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 -- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d501009b0 msgr2=0x7f0d5019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d501009b0 0x7f0d5019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.284+0000 7f0d4f400640 1 -- 172.21.15.175:0/3076418841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a1960 con 0x7f0d50101350 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.285+0000 7f0d4f400640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 0x7f0d501a12c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800eb10 tx=0x7f0d4800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.285+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48004b20 con 0x7f0d50101350 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.285+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48004cc0 con 0x7f0d50101350 2024-07-29T00:36:33.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.285+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a1c50 con 0x7f0d50101350 2024-07-29T00:36:33.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.285+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48010760 con 0x7f0d50101350 2024-07-29T00:36:33.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.285+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a21a0 con 0x7f0d50101350 2024-07-29T00:36:33.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.286+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5010afb0 con 0x7f0d50101350 2024-07-29T00:36:33.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.288+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 23) ==== 137954+0+0 (secure 0 0 0) 0x7f0d48005020 con 0x7f0d50101350 2024-07-29T00:36:33.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.289+0000 7f0d54a00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0d340a2d80 0x7f0d340a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:33.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.290+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0d480cfb80 con 0x7f0d50101350 2024-07-29T00:36:33.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.290+0000 7f0d4ea00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0d340a2d80 0x7f0d340a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:33.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.290+0000 7f0d4ea00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0d340a2d80 0x7f0d340a5240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d5019b220 tx=0x7f0d40000bc0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:33.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.293+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0d4809c720 con 0x7f0d50101350 2024-07-29T00:36:33.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.500+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d5002d7f0 con 0x7f0d50101350 2024-07-29T00:36:33.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.500+0000 7f0d54a00640 1 -- 172.21.15.175:0/3076418841 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f0d4809c020 con 0x7f0d50101350 2024-07-29T00:36:33.501 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 64 2024-07-29T00:36:33.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0d340a2d80 msgr2=0x7f0d340a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0d340a2d80 0x7f0d340a5240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d5019b220 tx=0x7f0d40000bc0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 msgr2=0x7f0d501a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:33.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 0x7f0d501a12c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800eb10 tx=0x7f0d4800c2d0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 shutdown_connections 2024-07-29T00:36:33.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d50101350 0x7f0d501a12c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0d340a2d80 0x7f0d340a5240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d501009b0 0x7f0d5019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 --2- 172.21.15.175:0/3076418841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d50100070 0x7f0d50199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:33.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 >> 172.21.15.175:0/3076418841 conn(0x7f0d500fbe80 msgr2=0x7f0d500fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:33.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 shutdown_connections 2024-07-29T00:36:33.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:33.503+0000 7f0d55e00640 1 -- 172.21.15.175:0/3076418841 wait complete. 2024-07-29T00:36:33.522 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-07-29T00:36:33.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:36:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:36:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-07-29T00:36:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2888354641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:33.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:33 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3076418841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-29T00:36:33.948 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2888354641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-29T00:36:34.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:33 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3076418841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-07-29T00:36:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2888354641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:34.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:33 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3076418841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-29T00:36:34.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 -- 172.21.15.175:0/3737274545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44103a80 msgr2=0x7f6d44103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 --2- 172.21.15.175:0/3737274545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44103a80 0x7f6d44103ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009930 tx=0x7f6d3402a420 comp rx=0 tx=0).stop 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 -- 172.21.15.175:0/3737274545 shutdown_connections 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 --2- 172.21.15.175:0/3737274545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44104420 0x7f6d44110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 --2- 172.21.15.175:0/3737274545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44103a80 0x7f6d44103ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 --2- 172.21.15.175:0/3737274545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44109a80 0x7f6d44109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 -- 172.21.15.175:0/3737274545 >> 172.21.15.175:0/3737274545 conn(0x7f6d440ff780 msgr2=0x7f6d44101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.579+0000 7f6d49600640 1 -- 172.21.15.175:0/3737274545 shutdown_connections 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.580+0000 7f6d49600640 1 -- 172.21.15.175:0/3737274545 wait complete. 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 Processor -- start 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 -- start start 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 0x7f6d440763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:34.620 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44104420 0x7f6d44076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44109a80 0x7f6d44076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113e40 con 0x7f6d44109a80 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113cc0 con 0x7f6d44104420 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.581+0000 7f6d49600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113fc0 con 0x7f6d44103a80 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 0x7f6d440763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44104420 0x7f6d44076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 0x7f6d440763f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33628/0 (socket says 172.21.15.175:33628) 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 -- 172.21.15.175:0/1275322415 learned_addr learned my addr 172.21.15.175:0/1275322415 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d43400640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44109a80 0x7f6d44076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 -- 172.21.15.175:0/1275322415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44104420 msgr2=0x7f6d44076930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44104420 0x7f6d44076930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 -- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44109a80 msgr2=0x7f6d44076e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:34.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44109a80 0x7f6d44076e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 -- 172.21.15.175:0/1275322415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d441aa9e0 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42000640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44104420 0x7f6d44076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d43400640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44109a80 0x7f6d44076e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.582+0000 7f6d42a00640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 0x7f6d440763f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000ea90 tx=0x7f6d3000ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.583+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000ce40 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.583+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d300035f0 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.583+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d441aacd0 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.583+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30010690 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.583+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441ab190 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.584+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d441051c0 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.586+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) ==== 137954+0+0 (secure 0 0 0) 0x7f6d300037a0 con 0x7f6d44103a80 2024-07-29T00:36:34.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.587+0000 7f6d3b400640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d280a2cf0 0x7f6d280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:34.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.588+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6d300cf8b0 con 0x7f6d44103a80 2024-07-29T00:36:34.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.588+0000 7f6d42000640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d280a2cf0 0x7f6d280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:34.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.589+0000 7f6d42000640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d280a2cf0 0x7f6d280a51b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009900 tx=0x7f6d340058d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:34.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.592+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6d3009c450 con 0x7f6d44103a80 2024-07-29T00:36:34.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.652+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f6d300983c0 con 0x7f6d44103a80 2024-07-29T00:36:34.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.840+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d44077920 con 0x7f6d44103a80 2024-07-29T00:36:34.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.841+0000 7f6d3b400640 1 -- 172.21.15.175:0/1275322415 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f6d30051a00 con 0x7f6d44103a80 2024-07-29T00:36:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:34.842 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:36:34.842 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:36:34.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d280a2cf0 msgr2=0x7f6d280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:34.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d280a2cf0 0x7f6d280a51b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009900 tx=0x7f6d340058d0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 msgr2=0x7f6d440763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:34.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 0x7f6d440763f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000ea90 tx=0x7f6d3000ef50 comp rx=0 tx=0).stop 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 shutdown_connections 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d280a2cf0 0x7f6d280a51b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d44109a80 0x7f6d44076e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44104420 0x7f6d44076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 --2- 172.21.15.175:0/1275322415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d44103a80 0x7f6d440763f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.844+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 >> 172.21.15.175:0/1275322415 conn(0x7f6d440ff780 msgr2=0x7f6d440fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.845+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 shutdown_connections 2024-07-29T00:36:34.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:34.845+0000 7f6d49600640 1 -- 172.21.15.175:0/1275322415 wait complete. 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.920 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi175 to 176.4M 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1275322415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: Unable to set osd_memory_target on smithi175 to 185029017: error parsing value: Value '185029017' is below minimum 939524096 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:36:35.921 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:35 smithi175 ceph-mon[32138]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:36:35.923 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-07-29T00:36:35.923 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-07-29T00:36:35.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-29T00:36:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:36:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-07-29T00:36:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi175 to 176.4M 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1275322415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: Unable to set osd_memory_target on smithi175 to 185029017: error parsing value: Value '185029017' is below minimum 939524096 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:36.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:36.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:36:36.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:36:36.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:35 smithi181 ceph-mon[36706]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:36:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:36:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-07-29T00:36:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi175 to 176.4M 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1275322415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:36.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: Unable to set osd_memory_target on smithi175 to 185029017: error parsing value: Value '185029017' is below minimum 939524096 2024-07-29T00:36:36.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:36.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:36.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-29T00:36:36.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: Updating smithi181:/etc/ceph/ceph.conf 2024-07-29T00:36:36.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:35 smithi204 ceph-mon[37368]: Updating smithi204:/etc/ceph/ceph.conf 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.106+0000 7f5ea6400640 1 -- 172.21.15.175:0/3540267343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 msgr2=0x7f5ea0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.106+0000 7f5ea6400640 1 --2- 172.21.15.175:0/3540267343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0100b20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009900 tx=0x7f5e9002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 -- 172.21.15.175:0/3540267343 shutdown_connections 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 --2- 172.21.15.175:0/3540267343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea01022d0 0x7f5ea0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 --2- 172.21.15.175:0/3540267343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea0101910 0x7f5ea0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 --2- 172.21.15.175:0/3540267343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0100b20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 -- 172.21.15.175:0/3540267343 >> 172.21.15.175:0/3540267343 conn(0x7f5ea00fbe80 msgr2=0x7f5ea00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:36.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 -- 172.21.15.175:0/3540267343 shutdown_connections 2024-07-29T00:36:36.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.107+0000 7f5ea6400640 1 -- 172.21.15.175:0/3540267343 wait complete. 2024-07-29T00:36:36.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 Processor -- start 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 -- start start 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0101910 0x7f5ea0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea01022d0 0x7f5ea019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea010c3d0 con 0x7f5ea0101910 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea010c250 con 0x7f5ea0100720 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.108+0000 7f5ea6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea010c550 con 0x7f5ea01022d0 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:44590/0 (socket says 172.21.15.175:44590) 2024-07-29T00:36:36.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 -- 172.21.15.175:0/2308184055 learned_addr learned my addr 172.21.15.175:0/2308184055 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9ea00640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0101910 0x7f5ea0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9fe00640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea01022d0 0x7f5ea019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 -- 172.21.15.175:0/2308184055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea01022d0 msgr2=0x7f5ea019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea01022d0 0x7f5ea019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 -- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0101910 msgr2=0x7f5ea0198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0101910 0x7f5ea0198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 -- 172.21.15.175:0/2308184055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea019f7f0 con 0x7f5ea0100720 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9ea00640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0101910 0x7f5ea0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.109+0000 7f5e9f400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0197bd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e90031610 tx=0x7f5e90032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.110+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9003f030 con 0x7f5ea0100720 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.110+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e90003ce0 con 0x7f5ea0100720 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.110+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea019fa80 con 0x7f5ea0100720 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.110+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90035070 con 0x7f5ea0100720 2024-07-29T00:36:36.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.110+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea019ffc0 con 0x7f5ea0100720 2024-07-29T00:36:36.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.111+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea01a0190 con 0x7f5ea0100720 2024-07-29T00:36:36.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.113+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f5e90042030 con 0x7f5ea0100720 2024-07-29T00:36:36.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.114+0000 7f5e97e00640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e840a2e10 0x7f5e840a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.115+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f5e900efb10 con 0x7f5ea0100720 2024-07-29T00:36:36.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.115+0000 7f5e9ea00640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e840a2e10 0x7f5e840a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.116+0000 7f5e9ea00640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e840a2e10 0x7f5e840a52d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0058c0 tx=0x7f5e8c005850 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:36.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.118+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5e900bc540 con 0x7f5ea0100720 2024-07-29T00:36:36.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.322+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ea006a760 con 0x7f5e840a2e10 2024-07-29T00:36:36.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.332+0000 7f5e97e00640 1 -- 172.21.15.175:0/2308184055 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258137 (secure 0 0 0) 0x7f5ea006a760 con 0x7f5e840a2e10 2024-07-29T00:36:36.337 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:36.338 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e840a2e10 msgr2=0x7f5e840a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e840a2e10 0x7f5e840a52d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0058c0 tx=0x7f5e8c005850 comp rx=0 tx=0).stop 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 msgr2=0x7f5ea0197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0197bd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e90031610 tx=0x7f5e90032ce0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 shutdown_connections 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea01022d0 0x7f5ea019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e840a2e10 0x7f5e840a52d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0101910 0x7f5ea0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 --2- 172.21.15.175:0/2308184055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea0100720 0x7f5ea0197bd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 >> 172.21.15.175:0/2308184055 conn(0x7f5ea00fbe80 msgr2=0x7f5ea006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 shutdown_connections 2024-07-29T00:36:36.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.337+0000 7f5ea6400640 1 -- 172.21.15.175:0/2308184055 wait complete. 2024-07-29T00:36:36.357 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-07-29T00:36:35.939808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191488,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1612,"num_read_kb":986,"num_write":562,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357528,"kb_used_data":12488,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584296,"statfs":{"total":1151940427776,"available":1151574319104,"internally_reserved":0,"allocated":12787712,"data_stored":9152531,"data_compressed":341605,"data_compressed_allocated":458752,"data_compressed_original":3457024,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.986219"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270840+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:36:30.270840+0000","last_peered":"2024-07-29T00:36:30.270840+0000","last_clean":"2024-07-29T00:36:30.270840+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:36:30.270840+0000","last_undegraded":"2024-07-29T00:36:30.270840+0000","last_fullsized":"2024-07-29T00:36:30.270840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122176+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:36:30.122176+0000","last_peered":"2024-07-29T00:36:30.122176+0000","last_clean":"2024-07-29T00:36:30.122176+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:36:30.122176+0000","last_undegraded":"2024-07-29T00:36:30.122176+0000","last_fullsized":"2024-07-29T00:36:30.122176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121890+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:36:30.121890+0000","last_peered":"2024-07-29T00:36:30.121890+0000","last_clean":"2024-07-29T00:36:30.121890+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:36:30.121890+0000","last_undegraded":"2024-07-29T00:36:30.121890+0000","last_fullsized":"2024-07-29T00:36:30.121890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919430+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:36:29.919430+0000","last_peered":"2024-07-29T00:36:29.919430+0000","last_clean":"2024-07-29T00:36:29.919430+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:36:29.919430+0000","last_undegraded":"2024-07-29T00:36:29.919430+0000","last_fullsized":"2024-07-29T00:36:29.919430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030174+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:36:30.030174+0000","last_peered":"2024-07-29T00:36:30.030174+0000","last_clean":"2024-07-29T00:36:30.030174+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:36:30.030174+0000","last_undegraded":"2024-07-29T00:36:30.030174+0000","last_fullsized":"2024-07-29T00:36:30.030174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039832+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:36:30.039832+0000","last_peered":"2024-07-29T00:36:30.039832+0000","last_clean":"2024-07-29T00:36:30.039832+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:36:30.039832+0000","last_undegraded":"2024-07-29T00:36:30.039832+0000","last_fullsized":"2024-07-29T00:36:30.039832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978714+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:36:29.978714+0000","last_peered":"2024-07-29T00:36:29.978714+0000","last_clean":"2024-07-29T00:36:29.978714+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:36:29.978714+0000","last_undegraded":"2024-07-29T00:36:29.978714+0000","last_fullsized":"2024-07-29T00:36:29.978714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006206+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:36:30.006206+0000","last_peered":"2024-07-29T00:36:30.006206+0000","last_clean":"2024-07-29T00:36:30.006206+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:36:30.006206+0000","last_undegraded":"2024-07-29T00:36:30.006206+0000","last_fullsized":"2024-07-29T00:36:30.006206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918987+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:36:29.918987+0000","last_peered":"2024-07-29T00:36:29.918987+0000","last_clean":"2024-07-29T00:36:29.918987+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:36:29.918987+0000","last_undegraded":"2024-07-29T00:36:29.918987+0000","last_fullsized":"2024-07-29T00:36:29.918987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.040027+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:36:30.040027+0000","last_peered":"2024-07-29T00:36:30.040027+0000","last_clean":"2024-07-29T00:36:30.040027+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:36:30.040027+0000","last_undegraded":"2024-07-29T00:36:30.040027+0000","last_fullsized":"2024-07-29T00:36:30.040027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978875+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:36:29.978875+0000","last_peered":"2024-07-29T00:36:29.978875+0000","last_clean":"2024-07-29T00:36:29.978875+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:36:29.978875+0000","last_undegraded":"2024-07-29T00:36:29.978875+0000","last_fullsized":"2024-07-29T00:36:29.978875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919501+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:36:29.919501+0000","last_peered":"2024-07-29T00:36:29.919501+0000","last_clean":"2024-07-29T00:36:29.919501+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:36:29.919501+0000","last_undegraded":"2024-07-29T00:36:29.919501+0000","last_fullsized":"2024-07-29T00:36:29.919501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003149+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:36:30.003149+0000","last_peered":"2024-07-29T00:36:30.003149+0000","last_clean":"2024-07-29T00:36:30.003149+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:36:30.003149+0000","last_undegraded":"2024-07-29T00:36:30.003149+0000","last_fullsized":"2024-07-29T00:36:30.003149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122517+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:36:30.122517+0000","last_peered":"2024-07-29T00:36:30.122517+0000","last_clean":"2024-07-29T00:36:30.122517+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:36:30.122517+0000","last_undegraded":"2024-07-29T00:36:30.122517+0000","last_fullsized":"2024-07-29T00:36:30.122517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121450+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:36:30.121450+0000","last_peered":"2024-07-29T00:36:30.121450+0000","last_clean":"2024-07-29T00:36:30.121450+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:36:30.121450+0000","last_undegraded":"2024-07-29T00:36:30.121450+0000","last_fullsized":"2024-07-29T00:36:30.121450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918029+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:36:29.918029+0000","last_peered":"2024-07-29T00:36:29.918029+0000","last_clean":"2024-07-29T00:36:29.918029+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:36:29.918029+0000","last_undegraded":"2024-07-29T00:36:29.918029+0000","last_fullsized":"2024-07-29T00:36:29.918029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270846+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:36:30.270846+0000","last_peered":"2024-07-29T00:36:30.270846+0000","last_clean":"2024-07-29T00:36:30.270846+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:36:30.270846+0000","last_undegraded":"2024-07-29T00:36:30.270846+0000","last_fullsized":"2024-07-29T00:36:30.270846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921670+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:36:29.921670+0000","last_peered":"2024-07-29T00:36:29.921670+0000","last_clean":"2024-07-29T00:36:29.921670+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:36:29.921670+0000","last_undegraded":"2024-07-29T00:36:29.921670+0000","last_fullsized":"2024-07-29T00:36:29.921670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119558+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:36:30.119558+0000","last_peered":"2024-07-29T00:36:30.119558+0000","last_clean":"2024-07-29T00:36:30.119558+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:36:30.119558+0000","last_undegraded":"2024-07-29T00:36:30.119558+0000","last_fullsized":"2024-07-29T00:36:30.119558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270504+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:36:30.270504+0000","last_peered":"2024-07-29T00:36:30.270504+0000","last_clean":"2024-07-29T00:36:30.270504+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:36:30.270504+0000","last_undegraded":"2024-07-29T00:36:30.270504+0000","last_fullsized":"2024-07-29T00:36:30.270504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003134+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:36:30.003134+0000","last_peered":"2024-07-29T00:36:30.003134+0000","last_clean":"2024-07-29T00:36:30.003134+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:36:30.003134+0000","last_undegraded":"2024-07-29T00:36:30.003134+0000","last_fullsized":"2024-07-29T00:36:30.003134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918840+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:36:29.918840+0000","last_peered":"2024-07-29T00:36:29.918840+0000","last_clean":"2024-07-29T00:36:29.918840+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:36:29.918840+0000","last_undegraded":"2024-07-29T00:36:29.918840+0000","last_fullsized":"2024-07-29T00:36:29.918840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006207+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:36:30.006207+0000","last_peered":"2024-07-29T00:36:30.006207+0000","last_clean":"2024-07-29T00:36:30.006207+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:36:30.006207+0000","last_undegraded":"2024-07-29T00:36:30.006207+0000","last_fullsized":"2024-07-29T00:36:30.006207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978362+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:36:29.978362+0000","last_peered":"2024-07-29T00:36:29.978362+0000","last_clean":"2024-07-29T00:36:29.978362+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:36:29.978362+0000","last_undegraded":"2024-07-29T00:36:29.978362+0000","last_fullsized":"2024-07-29T00:36:29.978362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006816+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:36:30.006816+0000","last_peered":"2024-07-29T00:36:30.006816+0000","last_clean":"2024-07-29T00:36:30.006816+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:36:30.006816+0000","last_undegraded":"2024-07-29T00:36:30.006816+0000","last_fullsized":"2024-07-29T00:36:30.006816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119656+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:36:30.119656+0000","last_peered":"2024-07-29T00:36:30.119656+0000","last_clean":"2024-07-29T00:36:30.119656+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:36:30.119656+0000","last_undegraded":"2024-07-29T00:36:30.119656+0000","last_fullsized":"2024-07-29T00:36:30.119656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270796+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:36:30.270796+0000","last_peered":"2024-07-29T00:36:30.270796+0000","last_clean":"2024-07-29T00:36:30.270796+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:36:30.270796+0000","last_undegraded":"2024-07-29T00:36:30.270796+0000","last_fullsized":"2024-07-29T00:36:30.270796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003369+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:36:30.003369+0000","last_peered":"2024-07-29T00:36:30.003369+0000","last_clean":"2024-07-29T00:36:30.003369+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:36:30.003369+0000","last_undegraded":"2024-07-29T00:36:30.003369+0000","last_fullsized":"2024-07-29T00:36:30.003369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121371+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:36:30.121371+0000","last_peered":"2024-07-29T00:36:30.121371+0000","last_clean":"2024-07-29T00:36:30.121371+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:36:30.121371+0000","last_undegraded":"2024-07-29T00:36:30.121371+0000","last_fullsized":"2024-07-29T00:36:30.121371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119269+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:36:30.119269+0000","last_peered":"2024-07-29T00:36:30.119269+0000","last_clean":"2024-07-29T00:36:30.119269+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:36:30.119269+0000","last_undegraded":"2024-07-29T00:36:30.119269+0000","last_fullsized":"2024-07-29T00:36:30.119269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119276+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:36:30.119276+0000","last_peered":"2024-07-29T00:36:30.119276+0000","last_clean":"2024-07-29T00:36:30.119276+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:36:30.119276+0000","last_undegraded":"2024-07-29T00:36:30.119276+0000","last_fullsized":"2024-07-29T00:36:30.119276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270536+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:36:30.270536+0000","last_peered":"2024-07-29T00:36:30.270536+0000","last_clean":"2024-07-29T00:36:30.270536+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:36:30.270536+0000","last_undegraded":"2024-07-29T00:36:30.270536+0000","last_fullsized":"2024-07-29T00:36:30.270536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029964+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:36:30.029964+0000","last_peered":"2024-07-29T00:36:30.029964+0000","last_clean":"2024-07-29T00:36:30.029964+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:36:30.029964+0000","last_undegraded":"2024-07-29T00:36:30.029964+0000","last_fullsized":"2024-07-29T00:36:30.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":186,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:34.892782+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:36:34.892782+0000","last_peered":"2024-07-29T00:36:34.892782+0000","last_clean":"2024-07-29T00:36:34.892782+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:36:34.892782+0000","last_undegraded":"2024-07-29T00:36:34.892782+0000","last_fullsized":"2024-07-29T00:36:34.892782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978592+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:36:29.978592+0000","last_peered":"2024-07-29T00:36:29.978592+0000","last_clean":"2024-07-29T00:36:29.978592+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:36:29.978592+0000","last_undegraded":"2024-07-29T00:36:29.978592+0000","last_fullsized":"2024-07-29T00:36:29.978592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039762+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:36:30.039762+0000","last_peered":"2024-07-29T00:36:30.039762+0000","last_clean":"2024-07-29T00:36:30.039762+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:36:30.039762+0000","last_undegraded":"2024-07-29T00:36:30.039762+0000","last_fullsized":"2024-07-29T00:36:30.039762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002737+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:36:30.002737+0000","last_peered":"2024-07-29T00:36:30.002737+0000","last_clean":"2024-07-29T00:36:30.002737+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:36:30.002737+0000","last_undegraded":"2024-07-29T00:36:30.002737+0000","last_fullsized":"2024-07-29T00:36:30.002737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006358+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:36:30.006358+0000","last_peered":"2024-07-29T00:36:30.006358+0000","last_clean":"2024-07-29T00:36:30.006358+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:36:30.006358+0000","last_undegraded":"2024-07-29T00:36:30.006358+0000","last_fullsized":"2024-07-29T00:36:30.006358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978697+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:36:29.978697+0000","last_peered":"2024-07-29T00:36:29.978697+0000","last_clean":"2024-07-29T00:36:29.978697+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:36:29.978697+0000","last_undegraded":"2024-07-29T00:36:29.978697+0000","last_fullsized":"2024-07-29T00:36:29.978697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.920897+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:36:29.920897+0000","last_peered":"2024-07-29T00:36:29.920897+0000","last_clean":"2024-07-29T00:36:29.920897+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:36:29.920897+0000","last_undegraded":"2024-07-29T00:36:29.920897+0000","last_fullsized":"2024-07-29T00:36:29.920897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.844192+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:36:30.844192+0000","last_peered":"2024-07-29T00:36:30.844192+0000","last_clean":"2024-07-29T00:36:30.844192+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:36:30.844192+0000","last_undegraded":"2024-07-29T00:36:30.844192+0000","last_fullsized":"2024-07-29T00:36:30.844192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":275,"num_read_kb":275,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918684+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:36:29.918684+0000","last_peered":"2024-07-29T00:36:29.918684+0000","last_clean":"2024-07-29T00:36:29.918684+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:36:29.918684+0000","last_undegraded":"2024-07-29T00:36:29.918684+0000","last_fullsized":"2024-07-29T00:36:29.918684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979141+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:36:29.979141+0000","last_peered":"2024-07-29T00:36:29.979141+0000","last_clean":"2024-07-29T00:36:29.979141+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:36:29.979141+0000","last_undegraded":"2024-07-29T00:36:29.979141+0000","last_fullsized":"2024-07-29T00:36:29.979141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122906+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:36:30.122906+0000","last_peered":"2024-07-29T00:36:30.122906+0000","last_clean":"2024-07-29T00:36:30.122906+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:36:30.122906+0000","last_undegraded":"2024-07-29T00:36:30.122906+0000","last_fullsized":"2024-07-29T00:36:30.122906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978649+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:36:29.978649+0000","last_peered":"2024-07-29T00:36:29.978649+0000","last_clean":"2024-07-29T00:36:29.978649+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:36:29.978649+0000","last_undegraded":"2024-07-29T00:36:29.978649+0000","last_fullsized":"2024-07-29T00:36:29.978649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918482+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:36:29.918482+0000","last_peered":"2024-07-29T00:36:29.918482+0000","last_clean":"2024-07-29T00:36:29.918482+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:36:29.918482+0000","last_undegraded":"2024-07-29T00:36:29.918482+0000","last_fullsized":"2024-07-29T00:36:29.918482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":377,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.847995+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:36:30.847995+0000","last_peered":"2024-07-29T00:36:30.847995+0000","last_clean":"2024-07-29T00:36:30.847995+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:36:30.847995+0000","last_undegraded":"2024-07-29T00:36:30.847995+0000","last_fullsized":"2024-07-29T00:36:30.847995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66470,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":208,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030244+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:36:30.030244+0000","last_peered":"2024-07-29T00:36:30.030244+0000","last_clean":"2024-07-29T00:36:30.030244+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:36:30.030244+0000","last_undegraded":"2024-07-29T00:36:30.030244+0000","last_fullsized":"2024-07-29T00:36:30.030244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119352+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:36:30.119352+0000","last_peered":"2024-07-29T00:36:30.119352+0000","last_clean":"2024-07-29T00:36:30.119352+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:36:30.119352+0000","last_undegraded":"2024-07-29T00:36:30.119352+0000","last_fullsized":"2024-07-29T00:36:30.119352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002636+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:36:30.002636+0000","last_peered":"2024-07-29T00:36:30.002636+0000","last_clean":"2024-07-29T00:36:30.002636+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:36:30.002636+0000","last_undegraded":"2024-07-29T00:36:30.002636+0000","last_fullsized":"2024-07-29T00:36:30.002636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978452+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:36:29.978452+0000","last_peered":"2024-07-29T00:36:29.978452+0000","last_clean":"2024-07-29T00:36:29.978452+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:36:29.978452+0000","last_undegraded":"2024-07-29T00:36:29.978452+0000","last_fullsized":"2024-07-29T00:36:29.978452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978851+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:36:29.978851+0000","last_peered":"2024-07-29T00:36:29.978851+0000","last_clean":"2024-07-29T00:36:29.978851+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:36:29.978851+0000","last_undegraded":"2024-07-29T00:36:29.978851+0000","last_fullsized":"2024-07-29T00:36:29.978851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":183,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.264032+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:36:30.264032+0000","last_peered":"2024-07-29T00:36:30.264032+0000","last_clean":"2024-07-29T00:36:30.264032+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:36:30.264032+0000","last_undegraded":"2024-07-29T00:36:30.264032+0000","last_fullsized":"2024-07-29T00:36:30.264032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":5,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978642+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:36:29.978642+0000","last_peered":"2024-07-29T00:36:29.978642+0000","last_clean":"2024-07-29T00:36:29.978642+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:36:29.978642+0000","last_undegraded":"2024-07-29T00:36:29.978642+0000","last_fullsized":"2024-07-29T00:36:29.978642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029821+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:36:30.029821+0000","last_peered":"2024-07-29T00:36:30.029821+0000","last_clean":"2024-07-29T00:36:30.029821+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:36:30.029821+0000","last_undegraded":"2024-07-29T00:36:30.029821+0000","last_fullsized":"2024-07-29T00:36:30.029821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978327+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:36:29.978327+0000","last_peered":"2024-07-29T00:36:29.978327+0000","last_clean":"2024-07-29T00:36:29.978327+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:36:29.978327+0000","last_undegraded":"2024-07-29T00:36:29.978327+0000","last_fullsized":"2024-07-29T00:36:29.978327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030159+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:36:30.030159+0000","last_peered":"2024-07-29T00:36:30.030159+0000","last_clean":"2024-07-29T00:36:30.030159+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:36:30.030159+0000","last_undegraded":"2024-07-29T00:36:30.030159+0000","last_fullsized":"2024-07-29T00:36:30.030159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270695+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:36:30.270695+0000","last_peered":"2024-07-29T00:36:30.270695+0000","last_clean":"2024-07-29T00:36:30.270695+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:36:30.270695+0000","last_undegraded":"2024-07-29T00:36:30.270695+0000","last_fullsized":"2024-07-29T00:36:30.270695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003638+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:36:30.003638+0000","last_peered":"2024-07-29T00:36:30.003638+0000","last_clean":"2024-07-29T00:36:30.003638+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:36:30.003638+0000","last_undegraded":"2024-07-29T00:36:30.003638+0000","last_fullsized":"2024-07-29T00:36:30.003638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979263+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:36:29.979263+0000","last_peered":"2024-07-29T00:36:29.979263+0000","last_clean":"2024-07-29T00:36:29.979263+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:36:29.979263+0000","last_undegraded":"2024-07-29T00:36:29.979263+0000","last_fullsized":"2024-07-29T00:36:29.979263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.212250+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:36:30.212250+0000","last_peered":"2024-07-29T00:36:30.212250+0000","last_clean":"2024-07-29T00:36:30.212250+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:36:30.212250+0000","last_undegraded":"2024-07-29T00:36:30.212250+0000","last_fullsized":"2024-07-29T00:36:30.212250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978805+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:36:29.978805+0000","last_peered":"2024-07-29T00:36:29.978805+0000","last_clean":"2024-07-29T00:36:29.978805+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:36:29.978805+0000","last_undegraded":"2024-07-29T00:36:29.978805+0000","last_fullsized":"2024-07-29T00:36:29.978805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.844581+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:36:30.844581+0000","last_peered":"2024-07-29T00:36:30.844581+0000","last_clean":"2024-07-29T00:36:30.844581+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:36:30.844581+0000","last_undegraded":"2024-07-29T00:36:30.844581+0000","last_fullsized":"2024-07-29T00:36:30.844581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979102+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:36:29.979102+0000","last_peered":"2024-07-29T00:36:29.979102+0000","last_clean":"2024-07-29T00:36:29.979102+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:36:29.979102+0000","last_undegraded":"2024-07-29T00:36:29.979102+0000","last_fullsized":"2024-07-29T00:36:29.979102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978819+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:36:29.978819+0000","last_peered":"2024-07-29T00:36:29.978819+0000","last_clean":"2024-07-29T00:36:29.978819+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:36:29.978819+0000","last_undegraded":"2024-07-29T00:36:29.978819+0000","last_fullsized":"2024-07-29T00:36:29.978819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918549+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:36:29.918549+0000","last_peered":"2024-07-29T00:36:29.918549+0000","last_clean":"2024-07-29T00:36:29.918549+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:36:29.918549+0000","last_undegraded":"2024-07-29T00:36:29.918549+0000","last_fullsized":"2024-07-29T00:36:29.918549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006458+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:36:30.006458+0000","last_peered":"2024-07-29T00:36:30.006458+0000","last_clean":"2024-07-29T00:36:30.006458+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:36:30.006458+0000","last_undegraded":"2024-07-29T00:36:30.006458+0000","last_fullsized":"2024-07-29T00:36:30.006458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270459+0000","last_change":"2024-07-29T00:35:51.676982+0000","last_active":"2024-07-29T00:36:30.270459+0000","last_peered":"2024-07-29T00:36:30.270459+0000","last_clean":"2024-07-29T00:36:30.270459+0000","last_became_active":"2024-07-29T00:35:51.674179+0000","last_became_peered":"2024-07-29T00:35:51.674179+0000","last_unstale":"2024-07-29T00:36:30.270459+0000","last_undegraded":"2024-07-29T00:36:30.270459+0000","last_fullsized":"2024-07-29T00:36:30.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:24:23.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002590+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:36:30.002590+0000","last_peered":"2024-07-29T00:36:30.002590+0000","last_clean":"2024-07-29T00:36:30.002590+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:36:30.002590+0000","last_undegraded":"2024-07-29T00:36:30.002590+0000","last_fullsized":"2024-07-29T00:36:30.002590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003199+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:36:30.003199+0000","last_peered":"2024-07-29T00:36:30.003199+0000","last_clean":"2024-07-29T00:36:30.003199+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:36:30.003199+0000","last_undegraded":"2024-07-29T00:36:30.003199+0000","last_fullsized":"2024-07-29T00:36:30.003199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978450+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:36:29.978450+0000","last_peered":"2024-07-29T00:36:29.978450+0000","last_clean":"2024-07-29T00:36:29.978450+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:36:29.978450+0000","last_undegraded":"2024-07-29T00:36:29.978450+0000","last_fullsized":"2024-07-29T00:36:29.978450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979139+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:36:29.979139+0000","last_peered":"2024-07-29T00:36:29.979139+0000","last_clean":"2024-07-29T00:36:29.979139+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:36:29.979139+0000","last_undegraded":"2024-07-29T00:36:29.979139+0000","last_fullsized":"2024-07-29T00:36:29.979139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":895,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039767+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:36:30.039767+0000","last_peered":"2024-07-29T00:36:30.039767+0000","last_clean":"2024-07-29T00:36:30.039767+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:36:30.039767+0000","last_undegraded":"2024-07-29T00:36:30.039767+0000","last_fullsized":"2024-07-29T00:36:30.039767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921376+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:36:29.921376+0000","last_peered":"2024-07-29T00:36:29.921376+0000","last_clean":"2024-07-29T00:36:29.921376+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:36:29.921376+0000","last_undegraded":"2024-07-29T00:36:29.921376+0000","last_fullsized":"2024-07-29T00:36:29.921376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039877+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:36:30.039877+0000","last_peered":"2024-07-29T00:36:30.039877+0000","last_clean":"2024-07-29T00:36:30.039877+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:36:30.039877+0000","last_undegraded":"2024-07-29T00:36:30.039877+0000","last_fullsized":"2024-07-29T00:36:30.039877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030289+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:36:30.030289+0000","last_peered":"2024-07-29T00:36:30.030289+0000","last_clean":"2024-07-29T00:36:30.030289+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:36:30.030289+0000","last_undegraded":"2024-07-29T00:36:30.030289+0000","last_fullsized":"2024-07-29T00:36:30.030289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003485+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:36:30.003485+0000","last_peered":"2024-07-29T00:36:30.003485+0000","last_clean":"2024-07-29T00:36:30.003485+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:36:30.003485+0000","last_undegraded":"2024-07-29T00:36:30.003485+0000","last_fullsized":"2024-07-29T00:36:30.003485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'52","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.216413+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:36:30.216413+0000","last_peered":"2024-07-29T00:36:30.216413+0000","last_clean":"2024-07-29T00:36:30.216413+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:36:30.216413+0000","last_undegraded":"2024-07-29T00:36:30.216413+0000","last_fullsized":"2024-07-29T00:36:30.216413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006932+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:36:30.006932+0000","last_peered":"2024-07-29T00:36:30.006932+0000","last_clean":"2024-07-29T00:36:30.006932+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:36:30.006932+0000","last_undegraded":"2024-07-29T00:36:30.006932+0000","last_fullsized":"2024-07-29T00:36:30.006932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919572+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:36:29.919572+0000","last_peered":"2024-07-29T00:36:29.919572+0000","last_clean":"2024-07-29T00:36:29.919572+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:36:29.919572+0000","last_undegraded":"2024-07-29T00:36:29.919572+0000","last_fullsized":"2024-07-29T00:36:29.919572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122292+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:36:30.122292+0000","last_peered":"2024-07-29T00:36:30.122292+0000","last_clean":"2024-07-29T00:36:30.122292+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:36:30.122292+0000","last_undegraded":"2024-07-29T00:36:30.122292+0000","last_fullsized":"2024-07-29T00:36:30.122292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918120+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:36:29.918120+0000","last_peered":"2024-07-29T00:36:29.918120+0000","last_clean":"2024-07-29T00:36:29.918120+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:36:29.918120+0000","last_undegraded":"2024-07-29T00:36:29.918120+0000","last_fullsized":"2024-07-29T00:36:29.918120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029897+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:36:30.029897+0000","last_peered":"2024-07-29T00:36:30.029897+0000","last_clean":"2024-07-29T00:36:30.029897+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:36:30.029897+0000","last_undegraded":"2024-07-29T00:36:30.029897+0000","last_fullsized":"2024-07-29T00:36:30.029897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006317+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:36:30.006317+0000","last_peered":"2024-07-29T00:36:30.006317+0000","last_clean":"2024-07-29T00:36:30.006317+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:36:30.006317+0000","last_undegraded":"2024-07-29T00:36:30.006317+0000","last_fullsized":"2024-07-29T00:36:30.006317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270586+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:36:30.270586+0000","last_peered":"2024-07-29T00:36:30.270586+0000","last_clean":"2024-07-29T00:36:30.270586+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:36:30.270586+0000","last_undegraded":"2024-07-29T00:36:30.270586+0000","last_fullsized":"2024-07-29T00:36:30.270586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119490+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:36:30.119490+0000","last_peered":"2024-07-29T00:36:30.119490+0000","last_clean":"2024-07-29T00:36:30.119490+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:36:30.119490+0000","last_undegraded":"2024-07-29T00:36:30.119490+0000","last_fullsized":"2024-07-29T00:36:30.119490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006281+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:36:30.006281+0000","last_peered":"2024-07-29T00:36:30.006281+0000","last_clean":"2024-07-29T00:36:30.006281+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:36:30.006281+0000","last_undegraded":"2024-07-29T00:36:30.006281+0000","last_fullsized":"2024-07-29T00:36:30.006281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029962+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:36:30.029962+0000","last_peered":"2024-07-29T00:36:30.029962+0000","last_clean":"2024-07-29T00:36:30.029962+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:36:30.029962+0000","last_undegraded":"2024-07-29T00:36:30.029962+0000","last_fullsized":"2024-07-29T00:36:30.029962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978784+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:36:29.978784+0000","last_peered":"2024-07-29T00:36:29.978784+0000","last_clean":"2024-07-29T00:36:29.978784+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:36:29.978784+0000","last_undegraded":"2024-07-29T00:36:29.978784+0000","last_fullsized":"2024-07-29T00:36:29.978784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978667+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:36:29.978667+0000","last_peered":"2024-07-29T00:36:29.978667+0000","last_clean":"2024-07-29T00:36:29.978667+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:36:29.978667+0000","last_undegraded":"2024-07-29T00:36:29.978667+0000","last_fullsized":"2024-07-29T00:36:29.978667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030262+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:36:30.030262+0000","last_peered":"2024-07-29T00:36:30.030262+0000","last_clean":"2024-07-29T00:36:30.030262+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:36:30.030262+0000","last_undegraded":"2024-07-29T00:36:30.030262+0000","last_fullsized":"2024-07-29T00:36:30.030262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918288+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:36:29.918288+0000","last_peered":"2024-07-29T00:36:29.918288+0000","last_clean":"2024-07-29T00:36:29.918288+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:36:29.918288+0000","last_undegraded":"2024-07-29T00:36:29.918288+0000","last_fullsized":"2024-07-29T00:36:29.918288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270633+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:36:30.270633+0000","last_peered":"2024-07-29T00:36:30.270633+0000","last_clean":"2024-07-29T00:36:30.270633+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:36:30.270633+0000","last_undegraded":"2024-07-29T00:36:30.270633+0000","last_fullsized":"2024-07-29T00:36:30.270633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122784+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:36:30.122784+0000","last_peered":"2024-07-29T00:36:30.122784+0000","last_clean":"2024-07-29T00:36:30.122784+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:36:30.122784+0000","last_undegraded":"2024-07-29T00:36:30.122784+0000","last_fullsized":"2024-07-29T00:36:30.122784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006887+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:36:30.006887+0000","last_peered":"2024-07-29T00:36:30.006887+0000","last_clean":"2024-07-29T00:36:30.006887+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:36:30.006887+0000","last_undegraded":"2024-07-29T00:36:30.006887+0000","last_fullsized":"2024-07-29T00:36:30.006887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921570+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:36:29.921570+0000","last_peered":"2024-07-29T00:36:29.921570+0000","last_clean":"2024-07-29T00:36:29.921570+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:36:29.921570+0000","last_undegraded":"2024-07-29T00:36:29.921570+0000","last_fullsized":"2024-07-29T00:36:29.921570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":185,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921917+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:36:29.921917+0000","last_peered":"2024-07-29T00:36:29.921917+0000","last_clean":"2024-07-29T00:36:29.921917+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:36:29.921917+0000","last_undegraded":"2024-07-29T00:36:29.921917+0000","last_fullsized":"2024-07-29T00:36:29.921917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600886,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1476,"num_read_kb":867,"num_write":379,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":1802658,"data_compressed":85489,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416429,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":492125,"data_compressed":11554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678065,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1291549,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":501069,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":894663,"data_compressed":33607,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1094412,"data_compressed":50309,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":492079,"data_compressed":11554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1225632,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":562665,"data_compressed":24044,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1025427,"data_compressed":39763,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":501529,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":568476,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":502905,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":70431,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135200,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":201117,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:36:36.362 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-29T00:36:36.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 -- 172.21.15.175:0/1939578918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0768a0 msgr2=0x7fe09c076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 --2- 172.21.15.175:0/1939578918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0768a0 0x7fe09c076ca0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe090009860 tx=0x7fe09002a340 comp rx=0 tx=0).stop 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 -- 172.21.15.175:0/1939578918 shutdown_connections 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 --2- 172.21.15.175:0/1939578918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c078480 0x7fe09c078d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 --2- 172.21.15.175:0/1939578918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c077ae0 0x7fe09c077f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 --2- 172.21.15.175:0/1939578918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0768a0 0x7fe09c076ca0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 -- 172.21.15.175:0/1939578918 >> 172.21.15.175:0/1939578918 conn(0x7fe09c0ff1a0 msgr2=0x7fe09c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 -- 172.21.15.175:0/1939578918 shutdown_connections 2024-07-29T00:36:36.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.576+0000 7fe0a3800640 1 -- 172.21.15.175:0/1939578918 wait complete. 2024-07-29T00:36:36.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.577+0000 7fe0a3800640 1 Processor -- start 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.577+0000 7fe0a3800640 1 -- start start 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a3800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 0x7fe09c19e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c077ae0 0x7fe09c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c078480 0x7fe09c1a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113ab0 con 0x7fe09c078480 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a3800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113930 con 0x7fe09c0768a0 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113c30 con 0x7fe09c077ae0 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 0x7fe09c19e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 0x7fe09c19e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:44620/0 (socket says 172.21.15.175:44620) 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 -- 172.21.15.175:0/3655083132 learned_addr learned my addr 172.21.15.175:0/3655083132 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:36.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a1800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c078480 0x7fe09c1a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe09be00640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c077ae0 0x7fe09c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 -- 172.21.15.175:0/3655083132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c077ae0 msgr2=0x7fe09c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c077ae0 0x7fe09c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 -- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c078480 msgr2=0x7fe09c1a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c078480 0x7fe09c1a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a0e00640 1 -- 172.21.15.175:0/3655083132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c1a5d90 con 0x7fe09c0768a0 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.578+0000 7fe0a1800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c078480 0x7fe09c1a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.579+0000 7fe0a0e00640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 0x7fe09c19e170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe090005940 tx=0x7fe090033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.579+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09003f020 con 0x7fe09c0768a0 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.579+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c1a6020 con 0x7fe09c0768a0 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.579+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe090030850 con 0x7fe09c0768a0 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.579+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090037040 con 0x7fe09c0768a0 2024-07-29T00:36:36.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.579+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a6560 con 0x7fe09c0768a0 2024-07-29T00:36:36.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.580+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c110520 con 0x7fe09c0768a0 2024-07-29T00:36:36.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.582+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7fe090042020 con 0x7fe09c0768a0 2024-07-29T00:36:36.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.583+0000 7fe099600640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0840a2e10 0x7fe0840a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:36.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.584+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe0900ef980 con 0x7fe09c0768a0 2024-07-29T00:36:36.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.584+0000 7fe09be00640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0840a2e10 0x7fe0840a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:36.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.584+0000 7fe09be00640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0840a2e10 0x7fe0840a52d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe09c19f680 tx=0x7fe088009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:36.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.587+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe090049020 con 0x7fe09c0768a0 2024-07-29T00:36:36.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.794+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe09c062320 con 0x7fe0840a2e10 2024-07-29T00:36:36.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.802+0000 7fe099600640 1 -- 172.21.15.175:0/3655083132 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258137 (secure 0 0 0) 0x7fe09c062320 con 0x7fe0840a2e10 2024-07-29T00:36:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:36.807 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:36:36.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0840a2e10 msgr2=0x7fe0840a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0840a2e10 0x7fe0840a52d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe09c19f680 tx=0x7fe088009260 comp rx=0 tx=0).stop 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 msgr2=0x7fe09c19e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 0x7fe09c19e170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe090005940 tx=0x7fe090033d90 comp rx=0 tx=0).stop 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 shutdown_connections 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0840a2e10 0x7fe0840a52d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe09c078480 0x7fe09c1a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c077ae0 0x7fe09c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 --2- 172.21.15.175:0/3655083132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe09c0768a0 0x7fe09c19e170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 >> 172.21.15.175:0/3655083132 conn(0x7fe09c0ff1a0 msgr2=0x7fe09c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 shutdown_connections 2024-07-29T00:36:36.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:36.807+0000 7fe0a3800640 1 -- 172.21.15.175:0/3655083132 wait complete. 2024-07-29T00:36:36.829 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-07-29T00:36:35.939808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191488,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1612,"num_read_kb":986,"num_write":562,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357528,"kb_used_data":12488,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584296,"statfs":{"total":1151940427776,"available":1151574319104,"internally_reserved":0,"allocated":12787712,"data_stored":9152531,"data_compressed":341605,"data_compressed_allocated":458752,"data_compressed_original":3457024,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.986219"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270840+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:36:30.270840+0000","last_peered":"2024-07-29T00:36:30.270840+0000","last_clean":"2024-07-29T00:36:30.270840+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:36:30.270840+0000","last_undegraded":"2024-07-29T00:36:30.270840+0000","last_fullsized":"2024-07-29T00:36:30.270840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122176+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:36:30.122176+0000","last_peered":"2024-07-29T00:36:30.122176+0000","last_clean":"2024-07-29T00:36:30.122176+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:36:30.122176+0000","last_undegraded":"2024-07-29T00:36:30.122176+0000","last_fullsized":"2024-07-29T00:36:30.122176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121890+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:36:30.121890+0000","last_peered":"2024-07-29T00:36:30.121890+0000","last_clean":"2024-07-29T00:36:30.121890+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:36:30.121890+0000","last_undegraded":"2024-07-29T00:36:30.121890+0000","last_fullsized":"2024-07-29T00:36:30.121890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919430+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:36:29.919430+0000","last_peered":"2024-07-29T00:36:29.919430+0000","last_clean":"2024-07-29T00:36:29.919430+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:36:29.919430+0000","last_undegraded":"2024-07-29T00:36:29.919430+0000","last_fullsized":"2024-07-29T00:36:29.919430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030174+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:36:30.030174+0000","last_peered":"2024-07-29T00:36:30.030174+0000","last_clean":"2024-07-29T00:36:30.030174+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:36:30.030174+0000","last_undegraded":"2024-07-29T00:36:30.030174+0000","last_fullsized":"2024-07-29T00:36:30.030174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039832+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:36:30.039832+0000","last_peered":"2024-07-29T00:36:30.039832+0000","last_clean":"2024-07-29T00:36:30.039832+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:36:30.039832+0000","last_undegraded":"2024-07-29T00:36:30.039832+0000","last_fullsized":"2024-07-29T00:36:30.039832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978714+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:36:29.978714+0000","last_peered":"2024-07-29T00:36:29.978714+0000","last_clean":"2024-07-29T00:36:29.978714+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:36:29.978714+0000","last_undegraded":"2024-07-29T00:36:29.978714+0000","last_fullsized":"2024-07-29T00:36:29.978714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006206+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:36:30.006206+0000","last_peered":"2024-07-29T00:36:30.006206+0000","last_clean":"2024-07-29T00:36:30.006206+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:36:30.006206+0000","last_undegraded":"2024-07-29T00:36:30.006206+0000","last_fullsized":"2024-07-29T00:36:30.006206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918987+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:36:29.918987+0000","last_peered":"2024-07-29T00:36:29.918987+0000","last_clean":"2024-07-29T00:36:29.918987+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:36:29.918987+0000","last_undegraded":"2024-07-29T00:36:29.918987+0000","last_fullsized":"2024-07-29T00:36:29.918987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.040027+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:36:30.040027+0000","last_peered":"2024-07-29T00:36:30.040027+0000","last_clean":"2024-07-29T00:36:30.040027+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:36:30.040027+0000","last_undegraded":"2024-07-29T00:36:30.040027+0000","last_fullsized":"2024-07-29T00:36:30.040027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978875+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:36:29.978875+0000","last_peered":"2024-07-29T00:36:29.978875+0000","last_clean":"2024-07-29T00:36:29.978875+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:36:29.978875+0000","last_undegraded":"2024-07-29T00:36:29.978875+0000","last_fullsized":"2024-07-29T00:36:29.978875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919501+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:36:29.919501+0000","last_peered":"2024-07-29T00:36:29.919501+0000","last_clean":"2024-07-29T00:36:29.919501+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:36:29.919501+0000","last_undegraded":"2024-07-29T00:36:29.919501+0000","last_fullsized":"2024-07-29T00:36:29.919501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003149+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:36:30.003149+0000","last_peered":"2024-07-29T00:36:30.003149+0000","last_clean":"2024-07-29T00:36:30.003149+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:36:30.003149+0000","last_undegraded":"2024-07-29T00:36:30.003149+0000","last_fullsized":"2024-07-29T00:36:30.003149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122517+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:36:30.122517+0000","last_peered":"2024-07-29T00:36:30.122517+0000","last_clean":"2024-07-29T00:36:30.122517+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:36:30.122517+0000","last_undegraded":"2024-07-29T00:36:30.122517+0000","last_fullsized":"2024-07-29T00:36:30.122517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121450+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:36:30.121450+0000","last_peered":"2024-07-29T00:36:30.121450+0000","last_clean":"2024-07-29T00:36:30.121450+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:36:30.121450+0000","last_undegraded":"2024-07-29T00:36:30.121450+0000","last_fullsized":"2024-07-29T00:36:30.121450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918029+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:36:29.918029+0000","last_peered":"2024-07-29T00:36:29.918029+0000","last_clean":"2024-07-29T00:36:29.918029+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:36:29.918029+0000","last_undegraded":"2024-07-29T00:36:29.918029+0000","last_fullsized":"2024-07-29T00:36:29.918029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270846+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:36:30.270846+0000","last_peered":"2024-07-29T00:36:30.270846+0000","last_clean":"2024-07-29T00:36:30.270846+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:36:30.270846+0000","last_undegraded":"2024-07-29T00:36:30.270846+0000","last_fullsized":"2024-07-29T00:36:30.270846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921670+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:36:29.921670+0000","last_peered":"2024-07-29T00:36:29.921670+0000","last_clean":"2024-07-29T00:36:29.921670+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:36:29.921670+0000","last_undegraded":"2024-07-29T00:36:29.921670+0000","last_fullsized":"2024-07-29T00:36:29.921670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119558+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:36:30.119558+0000","last_peered":"2024-07-29T00:36:30.119558+0000","last_clean":"2024-07-29T00:36:30.119558+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:36:30.119558+0000","last_undegraded":"2024-07-29T00:36:30.119558+0000","last_fullsized":"2024-07-29T00:36:30.119558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270504+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:36:30.270504+0000","last_peered":"2024-07-29T00:36:30.270504+0000","last_clean":"2024-07-29T00:36:30.270504+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:36:30.270504+0000","last_undegraded":"2024-07-29T00:36:30.270504+0000","last_fullsized":"2024-07-29T00:36:30.270504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003134+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:36:30.003134+0000","last_peered":"2024-07-29T00:36:30.003134+0000","last_clean":"2024-07-29T00:36:30.003134+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:36:30.003134+0000","last_undegraded":"2024-07-29T00:36:30.003134+0000","last_fullsized":"2024-07-29T00:36:30.003134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918840+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:36:29.918840+0000","last_peered":"2024-07-29T00:36:29.918840+0000","last_clean":"2024-07-29T00:36:29.918840+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:36:29.918840+0000","last_undegraded":"2024-07-29T00:36:29.918840+0000","last_fullsized":"2024-07-29T00:36:29.918840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006207+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:36:30.006207+0000","last_peered":"2024-07-29T00:36:30.006207+0000","last_clean":"2024-07-29T00:36:30.006207+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:36:30.006207+0000","last_undegraded":"2024-07-29T00:36:30.006207+0000","last_fullsized":"2024-07-29T00:36:30.006207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978362+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:36:29.978362+0000","last_peered":"2024-07-29T00:36:29.978362+0000","last_clean":"2024-07-29T00:36:29.978362+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:36:29.978362+0000","last_undegraded":"2024-07-29T00:36:29.978362+0000","last_fullsized":"2024-07-29T00:36:29.978362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006816+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:36:30.006816+0000","last_peered":"2024-07-29T00:36:30.006816+0000","last_clean":"2024-07-29T00:36:30.006816+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:36:30.006816+0000","last_undegraded":"2024-07-29T00:36:30.006816+0000","last_fullsized":"2024-07-29T00:36:30.006816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119656+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:36:30.119656+0000","last_peered":"2024-07-29T00:36:30.119656+0000","last_clean":"2024-07-29T00:36:30.119656+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:36:30.119656+0000","last_undegraded":"2024-07-29T00:36:30.119656+0000","last_fullsized":"2024-07-29T00:36:30.119656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270796+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:36:30.270796+0000","last_peered":"2024-07-29T00:36:30.270796+0000","last_clean":"2024-07-29T00:36:30.270796+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:36:30.270796+0000","last_undegraded":"2024-07-29T00:36:30.270796+0000","last_fullsized":"2024-07-29T00:36:30.270796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003369+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:36:30.003369+0000","last_peered":"2024-07-29T00:36:30.003369+0000","last_clean":"2024-07-29T00:36:30.003369+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:36:30.003369+0000","last_undegraded":"2024-07-29T00:36:30.003369+0000","last_fullsized":"2024-07-29T00:36:30.003369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121371+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:36:30.121371+0000","last_peered":"2024-07-29T00:36:30.121371+0000","last_clean":"2024-07-29T00:36:30.121371+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:36:30.121371+0000","last_undegraded":"2024-07-29T00:36:30.121371+0000","last_fullsized":"2024-07-29T00:36:30.121371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119269+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:36:30.119269+0000","last_peered":"2024-07-29T00:36:30.119269+0000","last_clean":"2024-07-29T00:36:30.119269+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:36:30.119269+0000","last_undegraded":"2024-07-29T00:36:30.119269+0000","last_fullsized":"2024-07-29T00:36:30.119269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119276+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:36:30.119276+0000","last_peered":"2024-07-29T00:36:30.119276+0000","last_clean":"2024-07-29T00:36:30.119276+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:36:30.119276+0000","last_undegraded":"2024-07-29T00:36:30.119276+0000","last_fullsized":"2024-07-29T00:36:30.119276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270536+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:36:30.270536+0000","last_peered":"2024-07-29T00:36:30.270536+0000","last_clean":"2024-07-29T00:36:30.270536+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:36:30.270536+0000","last_undegraded":"2024-07-29T00:36:30.270536+0000","last_fullsized":"2024-07-29T00:36:30.270536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029964+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:36:30.029964+0000","last_peered":"2024-07-29T00:36:30.029964+0000","last_clean":"2024-07-29T00:36:30.029964+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:36:30.029964+0000","last_undegraded":"2024-07-29T00:36:30.029964+0000","last_fullsized":"2024-07-29T00:36:30.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":186,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:34.892782+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:36:34.892782+0000","last_peered":"2024-07-29T00:36:34.892782+0000","last_clean":"2024-07-29T00:36:34.892782+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:36:34.892782+0000","last_undegraded":"2024-07-29T00:36:34.892782+0000","last_fullsized":"2024-07-29T00:36:34.892782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978592+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:36:29.978592+0000","last_peered":"2024-07-29T00:36:29.978592+0000","last_clean":"2024-07-29T00:36:29.978592+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:36:29.978592+0000","last_undegraded":"2024-07-29T00:36:29.978592+0000","last_fullsized":"2024-07-29T00:36:29.978592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039762+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:36:30.039762+0000","last_peered":"2024-07-29T00:36:30.039762+0000","last_clean":"2024-07-29T00:36:30.039762+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:36:30.039762+0000","last_undegraded":"2024-07-29T00:36:30.039762+0000","last_fullsized":"2024-07-29T00:36:30.039762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002737+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:36:30.002737+0000","last_peered":"2024-07-29T00:36:30.002737+0000","last_clean":"2024-07-29T00:36:30.002737+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:36:30.002737+0000","last_undegraded":"2024-07-29T00:36:30.002737+0000","last_fullsized":"2024-07-29T00:36:30.002737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006358+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:36:30.006358+0000","last_peered":"2024-07-29T00:36:30.006358+0000","last_clean":"2024-07-29T00:36:30.006358+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:36:30.006358+0000","last_undegraded":"2024-07-29T00:36:30.006358+0000","last_fullsized":"2024-07-29T00:36:30.006358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978697+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:36:29.978697+0000","last_peered":"2024-07-29T00:36:29.978697+0000","last_clean":"2024-07-29T00:36:29.978697+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:36:29.978697+0000","last_undegraded":"2024-07-29T00:36:29.978697+0000","last_fullsized":"2024-07-29T00:36:29.978697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.920897+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:36:29.920897+0000","last_peered":"2024-07-29T00:36:29.920897+0000","last_clean":"2024-07-29T00:36:29.920897+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:36:29.920897+0000","last_undegraded":"2024-07-29T00:36:29.920897+0000","last_fullsized":"2024-07-29T00:36:29.920897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.844192+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:36:30.844192+0000","last_peered":"2024-07-29T00:36:30.844192+0000","last_clean":"2024-07-29T00:36:30.844192+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:36:30.844192+0000","last_undegraded":"2024-07-29T00:36:30.844192+0000","last_fullsized":"2024-07-29T00:36:30.844192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":275,"num_read_kb":275,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918684+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:36:29.918684+0000","last_peered":"2024-07-29T00:36:29.918684+0000","last_clean":"2024-07-29T00:36:29.918684+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:36:29.918684+0000","last_undegraded":"2024-07-29T00:36:29.918684+0000","last_fullsized":"2024-07-29T00:36:29.918684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979141+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:36:29.979141+0000","last_peered":"2024-07-29T00:36:29.979141+0000","last_clean":"2024-07-29T00:36:29.979141+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:36:29.979141+0000","last_undegraded":"2024-07-29T00:36:29.979141+0000","last_fullsized":"2024-07-29T00:36:29.979141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122906+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:36:30.122906+0000","last_peered":"2024-07-29T00:36:30.122906+0000","last_clean":"2024-07-29T00:36:30.122906+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:36:30.122906+0000","last_undegraded":"2024-07-29T00:36:30.122906+0000","last_fullsized":"2024-07-29T00:36:30.122906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978649+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:36:29.978649+0000","last_peered":"2024-07-29T00:36:29.978649+0000","last_clean":"2024-07-29T00:36:29.978649+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:36:29.978649+0000","last_undegraded":"2024-07-29T00:36:29.978649+0000","last_fullsized":"2024-07-29T00:36:29.978649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918482+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:36:29.918482+0000","last_peered":"2024-07-29T00:36:29.918482+0000","last_clean":"2024-07-29T00:36:29.918482+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:36:29.918482+0000","last_undegraded":"2024-07-29T00:36:29.918482+0000","last_fullsized":"2024-07-29T00:36:29.918482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":377,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.847995+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:36:30.847995+0000","last_peered":"2024-07-29T00:36:30.847995+0000","last_clean":"2024-07-29T00:36:30.847995+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:36:30.847995+0000","last_undegraded":"2024-07-29T00:36:30.847995+0000","last_fullsized":"2024-07-29T00:36:30.847995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66470,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":208,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030244+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:36:30.030244+0000","last_peered":"2024-07-29T00:36:30.030244+0000","last_clean":"2024-07-29T00:36:30.030244+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:36:30.030244+0000","last_undegraded":"2024-07-29T00:36:30.030244+0000","last_fullsized":"2024-07-29T00:36:30.030244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119352+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:36:30.119352+0000","last_peered":"2024-07-29T00:36:30.119352+0000","last_clean":"2024-07-29T00:36:30.119352+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:36:30.119352+0000","last_undegraded":"2024-07-29T00:36:30.119352+0000","last_fullsized":"2024-07-29T00:36:30.119352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002636+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:36:30.002636+0000","last_peered":"2024-07-29T00:36:30.002636+0000","last_clean":"2024-07-29T00:36:30.002636+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:36:30.002636+0000","last_undegraded":"2024-07-29T00:36:30.002636+0000","last_fullsized":"2024-07-29T00:36:30.002636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978452+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:36:29.978452+0000","last_peered":"2024-07-29T00:36:29.978452+0000","last_clean":"2024-07-29T00:36:29.978452+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:36:29.978452+0000","last_undegraded":"2024-07-29T00:36:29.978452+0000","last_fullsized":"2024-07-29T00:36:29.978452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978851+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:36:29.978851+0000","last_peered":"2024-07-29T00:36:29.978851+0000","last_clean":"2024-07-29T00:36:29.978851+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:36:29.978851+0000","last_undegraded":"2024-07-29T00:36:29.978851+0000","last_fullsized":"2024-07-29T00:36:29.978851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":183,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.264032+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:36:30.264032+0000","last_peered":"2024-07-29T00:36:30.264032+0000","last_clean":"2024-07-29T00:36:30.264032+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:36:30.264032+0000","last_undegraded":"2024-07-29T00:36:30.264032+0000","last_fullsized":"2024-07-29T00:36:30.264032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":5,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978642+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:36:29.978642+0000","last_peered":"2024-07-29T00:36:29.978642+0000","last_clean":"2024-07-29T00:36:29.978642+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:36:29.978642+0000","last_undegraded":"2024-07-29T00:36:29.978642+0000","last_fullsized":"2024-07-29T00:36:29.978642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029821+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:36:30.029821+0000","last_peered":"2024-07-29T00:36:30.029821+0000","last_clean":"2024-07-29T00:36:30.029821+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:36:30.029821+0000","last_undegraded":"2024-07-29T00:36:30.029821+0000","last_fullsized":"2024-07-29T00:36:30.029821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978327+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:36:29.978327+0000","last_peered":"2024-07-29T00:36:29.978327+0000","last_clean":"2024-07-29T00:36:29.978327+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:36:29.978327+0000","last_undegraded":"2024-07-29T00:36:29.978327+0000","last_fullsized":"2024-07-29T00:36:29.978327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030159+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:36:30.030159+0000","last_peered":"2024-07-29T00:36:30.030159+0000","last_clean":"2024-07-29T00:36:30.030159+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:36:30.030159+0000","last_undegraded":"2024-07-29T00:36:30.030159+0000","last_fullsized":"2024-07-29T00:36:30.030159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270695+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:36:30.270695+0000","last_peered":"2024-07-29T00:36:30.270695+0000","last_clean":"2024-07-29T00:36:30.270695+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:36:30.270695+0000","last_undegraded":"2024-07-29T00:36:30.270695+0000","last_fullsized":"2024-07-29T00:36:30.270695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003638+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:36:30.003638+0000","last_peered":"2024-07-29T00:36:30.003638+0000","last_clean":"2024-07-29T00:36:30.003638+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:36:30.003638+0000","last_undegraded":"2024-07-29T00:36:30.003638+0000","last_fullsized":"2024-07-29T00:36:30.003638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979263+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:36:29.979263+0000","last_peered":"2024-07-29T00:36:29.979263+0000","last_clean":"2024-07-29T00:36:29.979263+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:36:29.979263+0000","last_undegraded":"2024-07-29T00:36:29.979263+0000","last_fullsized":"2024-07-29T00:36:29.979263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.212250+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:36:30.212250+0000","last_peered":"2024-07-29T00:36:30.212250+0000","last_clean":"2024-07-29T00:36:30.212250+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:36:30.212250+0000","last_undegraded":"2024-07-29T00:36:30.212250+0000","last_fullsized":"2024-07-29T00:36:30.212250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978805+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:36:29.978805+0000","last_peered":"2024-07-29T00:36:29.978805+0000","last_clean":"2024-07-29T00:36:29.978805+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:36:29.978805+0000","last_undegraded":"2024-07-29T00:36:29.978805+0000","last_fullsized":"2024-07-29T00:36:29.978805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.844581+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:36:30.844581+0000","last_peered":"2024-07-29T00:36:30.844581+0000","last_clean":"2024-07-29T00:36:30.844581+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:36:30.844581+0000","last_undegraded":"2024-07-29T00:36:30.844581+0000","last_fullsized":"2024-07-29T00:36:30.844581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979102+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:36:29.979102+0000","last_peered":"2024-07-29T00:36:29.979102+0000","last_clean":"2024-07-29T00:36:29.979102+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:36:29.979102+0000","last_undegraded":"2024-07-29T00:36:29.979102+0000","last_fullsized":"2024-07-29T00:36:29.979102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978819+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:36:29.978819+0000","last_peered":"2024-07-29T00:36:29.978819+0000","last_clean":"2024-07-29T00:36:29.978819+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:36:29.978819+0000","last_undegraded":"2024-07-29T00:36:29.978819+0000","last_fullsized":"2024-07-29T00:36:29.978819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918549+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:36:29.918549+0000","last_peered":"2024-07-29T00:36:29.918549+0000","last_clean":"2024-07-29T00:36:29.918549+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:36:29.918549+0000","last_undegraded":"2024-07-29T00:36:29.918549+0000","last_fullsized":"2024-07-29T00:36:29.918549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006458+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:36:30.006458+0000","last_peered":"2024-07-29T00:36:30.006458+0000","last_clean":"2024-07-29T00:36:30.006458+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:36:30.006458+0000","last_undegraded":"2024-07-29T00:36:30.006458+0000","last_fullsized":"2024-07-29T00:36:30.006458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270459+0000","last_change":"2024-07-29T00:35:51.676982+0000","last_active":"2024-07-29T00:36:30.270459+0000","last_peered":"2024-07-29T00:36:30.270459+0000","last_clean":"2024-07-29T00:36:30.270459+0000","last_became_active":"2024-07-29T00:35:51.674179+0000","last_became_peered":"2024-07-29T00:35:51.674179+0000","last_unstale":"2024-07-29T00:36:30.270459+0000","last_undegraded":"2024-07-29T00:36:30.270459+0000","last_fullsized":"2024-07-29T00:36:30.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:24:23.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002590+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:36:30.002590+0000","last_peered":"2024-07-29T00:36:30.002590+0000","last_clean":"2024-07-29T00:36:30.002590+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:36:30.002590+0000","last_undegraded":"2024-07-29T00:36:30.002590+0000","last_fullsized":"2024-07-29T00:36:30.002590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003199+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:36:30.003199+0000","last_peered":"2024-07-29T00:36:30.003199+0000","last_clean":"2024-07-29T00:36:30.003199+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:36:30.003199+0000","last_undegraded":"2024-07-29T00:36:30.003199+0000","last_fullsized":"2024-07-29T00:36:30.003199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978450+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:36:29.978450+0000","last_peered":"2024-07-29T00:36:29.978450+0000","last_clean":"2024-07-29T00:36:29.978450+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:36:29.978450+0000","last_undegraded":"2024-07-29T00:36:29.978450+0000","last_fullsized":"2024-07-29T00:36:29.978450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979139+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:36:29.979139+0000","last_peered":"2024-07-29T00:36:29.979139+0000","last_clean":"2024-07-29T00:36:29.979139+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:36:29.979139+0000","last_undegraded":"2024-07-29T00:36:29.979139+0000","last_fullsized":"2024-07-29T00:36:29.979139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":895,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039767+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:36:30.039767+0000","last_peered":"2024-07-29T00:36:30.039767+0000","last_clean":"2024-07-29T00:36:30.039767+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:36:30.039767+0000","last_undegraded":"2024-07-29T00:36:30.039767+0000","last_fullsized":"2024-07-29T00:36:30.039767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921376+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:36:29.921376+0000","last_peered":"2024-07-29T00:36:29.921376+0000","last_clean":"2024-07-29T00:36:29.921376+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:36:29.921376+0000","last_undegraded":"2024-07-29T00:36:29.921376+0000","last_fullsized":"2024-07-29T00:36:29.921376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039877+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:36:30.039877+0000","last_peered":"2024-07-29T00:36:30.039877+0000","last_clean":"2024-07-29T00:36:30.039877+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:36:30.039877+0000","last_undegraded":"2024-07-29T00:36:30.039877+0000","last_fullsized":"2024-07-29T00:36:30.039877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030289+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:36:30.030289+0000","last_peered":"2024-07-29T00:36:30.030289+0000","last_clean":"2024-07-29T00:36:30.030289+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:36:30.030289+0000","last_undegraded":"2024-07-29T00:36:30.030289+0000","last_fullsized":"2024-07-29T00:36:30.030289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003485+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:36:30.003485+0000","last_peered":"2024-07-29T00:36:30.003485+0000","last_clean":"2024-07-29T00:36:30.003485+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:36:30.003485+0000","last_undegraded":"2024-07-29T00:36:30.003485+0000","last_fullsized":"2024-07-29T00:36:30.003485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'52","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.216413+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:36:30.216413+0000","last_peered":"2024-07-29T00:36:30.216413+0000","last_clean":"2024-07-29T00:36:30.216413+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:36:30.216413+0000","last_undegraded":"2024-07-29T00:36:30.216413+0000","last_fullsized":"2024-07-29T00:36:30.216413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006932+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:36:30.006932+0000","last_peered":"2024-07-29T00:36:30.006932+0000","last_clean":"2024-07-29T00:36:30.006932+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:36:30.006932+0000","last_undegraded":"2024-07-29T00:36:30.006932+0000","last_fullsized":"2024-07-29T00:36:30.006932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919572+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:36:29.919572+0000","last_peered":"2024-07-29T00:36:29.919572+0000","last_clean":"2024-07-29T00:36:29.919572+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:36:29.919572+0000","last_undegraded":"2024-07-29T00:36:29.919572+0000","last_fullsized":"2024-07-29T00:36:29.919572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122292+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:36:30.122292+0000","last_peered":"2024-07-29T00:36:30.122292+0000","last_clean":"2024-07-29T00:36:30.122292+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:36:30.122292+0000","last_undegraded":"2024-07-29T00:36:30.122292+0000","last_fullsized":"2024-07-29T00:36:30.122292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918120+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:36:29.918120+0000","last_peered":"2024-07-29T00:36:29.918120+0000","last_clean":"2024-07-29T00:36:29.918120+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:36:29.918120+0000","last_undegraded":"2024-07-29T00:36:29.918120+0000","last_fullsized":"2024-07-29T00:36:29.918120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029897+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:36:30.029897+0000","last_peered":"2024-07-29T00:36:30.029897+0000","last_clean":"2024-07-29T00:36:30.029897+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:36:30.029897+0000","last_undegraded":"2024-07-29T00:36:30.029897+0000","last_fullsized":"2024-07-29T00:36:30.029897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006317+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:36:30.006317+0000","last_peered":"2024-07-29T00:36:30.006317+0000","last_clean":"2024-07-29T00:36:30.006317+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:36:30.006317+0000","last_undegraded":"2024-07-29T00:36:30.006317+0000","last_fullsized":"2024-07-29T00:36:30.006317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270586+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:36:30.270586+0000","last_peered":"2024-07-29T00:36:30.270586+0000","last_clean":"2024-07-29T00:36:30.270586+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:36:30.270586+0000","last_undegraded":"2024-07-29T00:36:30.270586+0000","last_fullsized":"2024-07-29T00:36:30.270586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119490+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:36:30.119490+0000","last_peered":"2024-07-29T00:36:30.119490+0000","last_clean":"2024-07-29T00:36:30.119490+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:36:30.119490+0000","last_undegraded":"2024-07-29T00:36:30.119490+0000","last_fullsized":"2024-07-29T00:36:30.119490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006281+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:36:30.006281+0000","last_peered":"2024-07-29T00:36:30.006281+0000","last_clean":"2024-07-29T00:36:30.006281+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:36:30.006281+0000","last_undegraded":"2024-07-29T00:36:30.006281+0000","last_fullsized":"2024-07-29T00:36:30.006281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029962+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:36:30.029962+0000","last_peered":"2024-07-29T00:36:30.029962+0000","last_clean":"2024-07-29T00:36:30.029962+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:36:30.029962+0000","last_undegraded":"2024-07-29T00:36:30.029962+0000","last_fullsized":"2024-07-29T00:36:30.029962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978784+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:36:29.978784+0000","last_peered":"2024-07-29T00:36:29.978784+0000","last_clean":"2024-07-29T00:36:29.978784+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:36:29.978784+0000","last_undegraded":"2024-07-29T00:36:29.978784+0000","last_fullsized":"2024-07-29T00:36:29.978784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978667+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:36:29.978667+0000","last_peered":"2024-07-29T00:36:29.978667+0000","last_clean":"2024-07-29T00:36:29.978667+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:36:29.978667+0000","last_undegraded":"2024-07-29T00:36:29.978667+0000","last_fullsized":"2024-07-29T00:36:29.978667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030262+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:36:30.030262+0000","last_peered":"2024-07-29T00:36:30.030262+0000","last_clean":"2024-07-29T00:36:30.030262+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:36:30.030262+0000","last_undegraded":"2024-07-29T00:36:30.030262+0000","last_fullsized":"2024-07-29T00:36:30.030262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918288+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:36:29.918288+0000","last_peered":"2024-07-29T00:36:29.918288+0000","last_clean":"2024-07-29T00:36:29.918288+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:36:29.918288+0000","last_undegraded":"2024-07-29T00:36:29.918288+0000","last_fullsized":"2024-07-29T00:36:29.918288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270633+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:36:30.270633+0000","last_peered":"2024-07-29T00:36:30.270633+0000","last_clean":"2024-07-29T00:36:30.270633+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:36:30.270633+0000","last_undegraded":"2024-07-29T00:36:30.270633+0000","last_fullsized":"2024-07-29T00:36:30.270633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122784+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:36:30.122784+0000","last_peered":"2024-07-29T00:36:30.122784+0000","last_clean":"2024-07-29T00:36:30.122784+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:36:30.122784+0000","last_undegraded":"2024-07-29T00:36:30.122784+0000","last_fullsized":"2024-07-29T00:36:30.122784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006887+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:36:30.006887+0000","last_peered":"2024-07-29T00:36:30.006887+0000","last_clean":"2024-07-29T00:36:30.006887+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:36:30.006887+0000","last_undegraded":"2024-07-29T00:36:30.006887+0000","last_fullsized":"2024-07-29T00:36:30.006887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921570+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:36:29.921570+0000","last_peered":"2024-07-29T00:36:29.921570+0000","last_clean":"2024-07-29T00:36:29.921570+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:36:29.921570+0000","last_undegraded":"2024-07-29T00:36:29.921570+0000","last_fullsized":"2024-07-29T00:36:29.921570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":185,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921917+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:36:29.921917+0000","last_peered":"2024-07-29T00:36:29.921917+0000","last_clean":"2024-07-29T00:36:29.921917+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:36:29.921917+0000","last_undegraded":"2024-07-29T00:36:29.921917+0000","last_fullsized":"2024-07-29T00:36:29.921917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600886,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1476,"num_read_kb":867,"num_write":379,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":1802658,"data_compressed":85489,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416429,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":492125,"data_compressed":11554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678065,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1291549,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":501069,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":894663,"data_compressed":33607,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1094412,"data_compressed":50309,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":492079,"data_compressed":11554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1225632,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":562665,"data_compressed":24044,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1025427,"data_compressed":39763,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":501529,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":568476,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":502905,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":70431,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135200,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":201117,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:36:36.833 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-07-29T00:36:36.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-29T00:36:37.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.046+0000 7fb75ca00640 1 -- 172.21.15.175:0/763434867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 msgr2=0x7fb75810db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:37.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.046+0000 7fb75ca00640 1 --2- 172.21.15.175:0/763434867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75810db10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb740009900 tx=0x7fb74002a480 comp rx=0 tx=0).stop 2024-07-29T00:36:37.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 -- 172.21.15.175:0/763434867 shutdown_connections 2024-07-29T00:36:37.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 --2- 172.21.15.175:0/763434867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75810e050 0x7fb758110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 --2- 172.21.15.175:0/763434867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75810db10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 --2- 172.21.15.175:0/763434867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb758074290 0x7fb758074670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 -- 172.21.15.175:0/763434867 >> 172.21.15.175:0/763434867 conn(0x7fb7580fd4b0 msgr2=0x7fb7580ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:37.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 -- 172.21.15.175:0/763434867 shutdown_connections 2024-07-29T00:36:37.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.047+0000 7fb75ca00640 1 -- 172.21.15.175:0/763434867 wait complete. 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 Processor -- start 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 -- start start 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 0x7fb75819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb75810e050 0x7fb75819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb758113dd0 con 0x7fb758074bb0 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb758113c50 con 0x7fb758074290 2024-07-29T00:36:37.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb75ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb758113f50 con 0x7fb75810e050 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.048+0000 7fb74fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 0x7fb75819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb74fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57334/0 (socket says 172.21.15.175:57334) 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 0x7fb75819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:44624/0 (socket says 172.21.15.175:44624) 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb74fe00640 1 -- 172.21.15.175:0/307619945 learned_addr learned my addr 172.21.15.175:0/307619945 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb756800640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb75810e050 0x7fb75819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 -- 172.21.15.175:0/307619945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb75810e050 msgr2=0x7fb75819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb75810e050 0x7fb75819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 -- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 msgr2=0x7fb75819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 -- 172.21.15.175:0/307619945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819f330 con 0x7fb758074290 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb74fe00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:37.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.049+0000 7fb755e00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 0x7fb75819a340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9f0 tx=0x7fb74800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:37.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.050+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800ccc0 con 0x7fb758074290 2024-07-29T00:36:37.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.050+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb748005020 con 0x7fb758074290 2024-07-29T00:36:37.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.050+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480049e0 con 0x7fb758074290 2024-07-29T00:36:37.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.050+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819f620 con 0x7fb758074290 2024-07-29T00:36:37.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.050+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a6eb0 con 0x7fb758074290 2024-07-29T00:36:37.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.051+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758074670 con 0x7fb758074290 2024-07-29T00:36:37.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.053+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7fb748005560 con 0x7fb758074290 2024-07-29T00:36:37.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.054+0000 7fb74ea00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb73c0a2e10 0x7fb73c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:37.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.054+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb7480cf580 con 0x7fb758074290 2024-07-29T00:36:37.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.054+0000 7fb74fe00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb73c0a2e10 0x7fb73c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:37.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.055+0000 7fb74fe00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb73c0a2e10 0x7fb73c0a52d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb7400096c0 tx=0x7fb7400046c0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:37.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.058+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb7480051d0 con 0x7fb758074290 2024-07-29T00:36:37.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:36 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:36 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.259+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb758062320 con 0x7fb73c0a2e10 2024-07-29T00:36:37.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.267+0000 7fb74ea00640 1 -- 172.21.15.175:0/307619945 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258137 (secure 0 0 0) 0x7fb758062320 con 0x7fb73c0a2e10 2024-07-29T00:36:37.271 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:37.272 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:36:37.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.272+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb73c0a2e10 msgr2=0x7fb73c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.272+0000 7fb75ca00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb73c0a2e10 0x7fb73c0a52d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb7400096c0 tx=0x7fb7400046c0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.272+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 msgr2=0x7fb75819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.272+0000 7fb75ca00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 0x7fb75819a340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9f0 tx=0x7fb74800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.272+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 shutdown_connections 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb75810e050 0x7fb75819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb73c0a2e10 0x7fb73c0a52d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb758074bb0 0x7fb75819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 --2- 172.21.15.175:0/307619945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb758074290 0x7fb75819a340 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 >> 172.21.15.175:0/307619945 conn(0x7fb7580fd4b0 msgr2=0x7fb7580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:37.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 shutdown_connections 2024-07-29T00:36:37.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:37.273+0000 7fb75ca00640 1 -- 172.21.15.175:0/307619945 wait complete. 2024-07-29T00:36:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.conf 2024-07-29T00:36:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: Updating smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:36 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:37.291 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-07-29T00:36:35.939808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191488,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1612,"num_read_kb":986,"num_write":562,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357528,"kb_used_data":12488,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584296,"statfs":{"total":1151940427776,"available":1151574319104,"internally_reserved":0,"allocated":12787712,"data_stored":9152531,"data_compressed":341605,"data_compressed_allocated":458752,"data_compressed_original":3457024,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.986219"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270840+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:36:30.270840+0000","last_peered":"2024-07-29T00:36:30.270840+0000","last_clean":"2024-07-29T00:36:30.270840+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:36:30.270840+0000","last_undegraded":"2024-07-29T00:36:30.270840+0000","last_fullsized":"2024-07-29T00:36:30.270840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122176+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:36:30.122176+0000","last_peered":"2024-07-29T00:36:30.122176+0000","last_clean":"2024-07-29T00:36:30.122176+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:36:30.122176+0000","last_undegraded":"2024-07-29T00:36:30.122176+0000","last_fullsized":"2024-07-29T00:36:30.122176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121890+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:36:30.121890+0000","last_peered":"2024-07-29T00:36:30.121890+0000","last_clean":"2024-07-29T00:36:30.121890+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:36:30.121890+0000","last_undegraded":"2024-07-29T00:36:30.121890+0000","last_fullsized":"2024-07-29T00:36:30.121890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919430+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:36:29.919430+0000","last_peered":"2024-07-29T00:36:29.919430+0000","last_clean":"2024-07-29T00:36:29.919430+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:36:29.919430+0000","last_undegraded":"2024-07-29T00:36:29.919430+0000","last_fullsized":"2024-07-29T00:36:29.919430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030174+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:36:30.030174+0000","last_peered":"2024-07-29T00:36:30.030174+0000","last_clean":"2024-07-29T00:36:30.030174+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:36:30.030174+0000","last_undegraded":"2024-07-29T00:36:30.030174+0000","last_fullsized":"2024-07-29T00:36:30.030174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039832+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:36:30.039832+0000","last_peered":"2024-07-29T00:36:30.039832+0000","last_clean":"2024-07-29T00:36:30.039832+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:36:30.039832+0000","last_undegraded":"2024-07-29T00:36:30.039832+0000","last_fullsized":"2024-07-29T00:36:30.039832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978714+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:36:29.978714+0000","last_peered":"2024-07-29T00:36:29.978714+0000","last_clean":"2024-07-29T00:36:29.978714+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:36:29.978714+0000","last_undegraded":"2024-07-29T00:36:29.978714+0000","last_fullsized":"2024-07-29T00:36:29.978714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006206+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:36:30.006206+0000","last_peered":"2024-07-29T00:36:30.006206+0000","last_clean":"2024-07-29T00:36:30.006206+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:36:30.006206+0000","last_undegraded":"2024-07-29T00:36:30.006206+0000","last_fullsized":"2024-07-29T00:36:30.006206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918987+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:36:29.918987+0000","last_peered":"2024-07-29T00:36:29.918987+0000","last_clean":"2024-07-29T00:36:29.918987+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:36:29.918987+0000","last_undegraded":"2024-07-29T00:36:29.918987+0000","last_fullsized":"2024-07-29T00:36:29.918987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.040027+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:36:30.040027+0000","last_peered":"2024-07-29T00:36:30.040027+0000","last_clean":"2024-07-29T00:36:30.040027+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:36:30.040027+0000","last_undegraded":"2024-07-29T00:36:30.040027+0000","last_fullsized":"2024-07-29T00:36:30.040027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978875+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:36:29.978875+0000","last_peered":"2024-07-29T00:36:29.978875+0000","last_clean":"2024-07-29T00:36:29.978875+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:36:29.978875+0000","last_undegraded":"2024-07-29T00:36:29.978875+0000","last_fullsized":"2024-07-29T00:36:29.978875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919501+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:36:29.919501+0000","last_peered":"2024-07-29T00:36:29.919501+0000","last_clean":"2024-07-29T00:36:29.919501+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:36:29.919501+0000","last_undegraded":"2024-07-29T00:36:29.919501+0000","last_fullsized":"2024-07-29T00:36:29.919501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003149+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:36:30.003149+0000","last_peered":"2024-07-29T00:36:30.003149+0000","last_clean":"2024-07-29T00:36:30.003149+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:36:30.003149+0000","last_undegraded":"2024-07-29T00:36:30.003149+0000","last_fullsized":"2024-07-29T00:36:30.003149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122517+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:36:30.122517+0000","last_peered":"2024-07-29T00:36:30.122517+0000","last_clean":"2024-07-29T00:36:30.122517+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:36:30.122517+0000","last_undegraded":"2024-07-29T00:36:30.122517+0000","last_fullsized":"2024-07-29T00:36:30.122517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121450+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:36:30.121450+0000","last_peered":"2024-07-29T00:36:30.121450+0000","last_clean":"2024-07-29T00:36:30.121450+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:36:30.121450+0000","last_undegraded":"2024-07-29T00:36:30.121450+0000","last_fullsized":"2024-07-29T00:36:30.121450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918029+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:36:29.918029+0000","last_peered":"2024-07-29T00:36:29.918029+0000","last_clean":"2024-07-29T00:36:29.918029+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:36:29.918029+0000","last_undegraded":"2024-07-29T00:36:29.918029+0000","last_fullsized":"2024-07-29T00:36:29.918029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270846+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:36:30.270846+0000","last_peered":"2024-07-29T00:36:30.270846+0000","last_clean":"2024-07-29T00:36:30.270846+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:36:30.270846+0000","last_undegraded":"2024-07-29T00:36:30.270846+0000","last_fullsized":"2024-07-29T00:36:30.270846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921670+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:36:29.921670+0000","last_peered":"2024-07-29T00:36:29.921670+0000","last_clean":"2024-07-29T00:36:29.921670+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:36:29.921670+0000","last_undegraded":"2024-07-29T00:36:29.921670+0000","last_fullsized":"2024-07-29T00:36:29.921670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119558+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:36:30.119558+0000","last_peered":"2024-07-29T00:36:30.119558+0000","last_clean":"2024-07-29T00:36:30.119558+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:36:30.119558+0000","last_undegraded":"2024-07-29T00:36:30.119558+0000","last_fullsized":"2024-07-29T00:36:30.119558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270504+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:36:30.270504+0000","last_peered":"2024-07-29T00:36:30.270504+0000","last_clean":"2024-07-29T00:36:30.270504+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:36:30.270504+0000","last_undegraded":"2024-07-29T00:36:30.270504+0000","last_fullsized":"2024-07-29T00:36:30.270504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003134+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:36:30.003134+0000","last_peered":"2024-07-29T00:36:30.003134+0000","last_clean":"2024-07-29T00:36:30.003134+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:36:30.003134+0000","last_undegraded":"2024-07-29T00:36:30.003134+0000","last_fullsized":"2024-07-29T00:36:30.003134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918840+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:36:29.918840+0000","last_peered":"2024-07-29T00:36:29.918840+0000","last_clean":"2024-07-29T00:36:29.918840+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:36:29.918840+0000","last_undegraded":"2024-07-29T00:36:29.918840+0000","last_fullsized":"2024-07-29T00:36:29.918840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006207+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:36:30.006207+0000","last_peered":"2024-07-29T00:36:30.006207+0000","last_clean":"2024-07-29T00:36:30.006207+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:36:30.006207+0000","last_undegraded":"2024-07-29T00:36:30.006207+0000","last_fullsized":"2024-07-29T00:36:30.006207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978362+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:36:29.978362+0000","last_peered":"2024-07-29T00:36:29.978362+0000","last_clean":"2024-07-29T00:36:29.978362+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:36:29.978362+0000","last_undegraded":"2024-07-29T00:36:29.978362+0000","last_fullsized":"2024-07-29T00:36:29.978362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006816+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:36:30.006816+0000","last_peered":"2024-07-29T00:36:30.006816+0000","last_clean":"2024-07-29T00:36:30.006816+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:36:30.006816+0000","last_undegraded":"2024-07-29T00:36:30.006816+0000","last_fullsized":"2024-07-29T00:36:30.006816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119656+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:36:30.119656+0000","last_peered":"2024-07-29T00:36:30.119656+0000","last_clean":"2024-07-29T00:36:30.119656+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:36:30.119656+0000","last_undegraded":"2024-07-29T00:36:30.119656+0000","last_fullsized":"2024-07-29T00:36:30.119656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270796+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:36:30.270796+0000","last_peered":"2024-07-29T00:36:30.270796+0000","last_clean":"2024-07-29T00:36:30.270796+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:36:30.270796+0000","last_undegraded":"2024-07-29T00:36:30.270796+0000","last_fullsized":"2024-07-29T00:36:30.270796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003369+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:36:30.003369+0000","last_peered":"2024-07-29T00:36:30.003369+0000","last_clean":"2024-07-29T00:36:30.003369+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:36:30.003369+0000","last_undegraded":"2024-07-29T00:36:30.003369+0000","last_fullsized":"2024-07-29T00:36:30.003369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121371+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:36:30.121371+0000","last_peered":"2024-07-29T00:36:30.121371+0000","last_clean":"2024-07-29T00:36:30.121371+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:36:30.121371+0000","last_undegraded":"2024-07-29T00:36:30.121371+0000","last_fullsized":"2024-07-29T00:36:30.121371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119269+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:36:30.119269+0000","last_peered":"2024-07-29T00:36:30.119269+0000","last_clean":"2024-07-29T00:36:30.119269+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:36:30.119269+0000","last_undegraded":"2024-07-29T00:36:30.119269+0000","last_fullsized":"2024-07-29T00:36:30.119269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119276+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:36:30.119276+0000","last_peered":"2024-07-29T00:36:30.119276+0000","last_clean":"2024-07-29T00:36:30.119276+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:36:30.119276+0000","last_undegraded":"2024-07-29T00:36:30.119276+0000","last_fullsized":"2024-07-29T00:36:30.119276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270536+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:36:30.270536+0000","last_peered":"2024-07-29T00:36:30.270536+0000","last_clean":"2024-07-29T00:36:30.270536+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:36:30.270536+0000","last_undegraded":"2024-07-29T00:36:30.270536+0000","last_fullsized":"2024-07-29T00:36:30.270536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029964+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:36:30.029964+0000","last_peered":"2024-07-29T00:36:30.029964+0000","last_clean":"2024-07-29T00:36:30.029964+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:36:30.029964+0000","last_undegraded":"2024-07-29T00:36:30.029964+0000","last_fullsized":"2024-07-29T00:36:30.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":186,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:34.892782+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:36:34.892782+0000","last_peered":"2024-07-29T00:36:34.892782+0000","last_clean":"2024-07-29T00:36:34.892782+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:36:34.892782+0000","last_undegraded":"2024-07-29T00:36:34.892782+0000","last_fullsized":"2024-07-29T00:36:34.892782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978592+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:36:29.978592+0000","last_peered":"2024-07-29T00:36:29.978592+0000","last_clean":"2024-07-29T00:36:29.978592+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:36:29.978592+0000","last_undegraded":"2024-07-29T00:36:29.978592+0000","last_fullsized":"2024-07-29T00:36:29.978592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039762+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:36:30.039762+0000","last_peered":"2024-07-29T00:36:30.039762+0000","last_clean":"2024-07-29T00:36:30.039762+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:36:30.039762+0000","last_undegraded":"2024-07-29T00:36:30.039762+0000","last_fullsized":"2024-07-29T00:36:30.039762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002737+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:36:30.002737+0000","last_peered":"2024-07-29T00:36:30.002737+0000","last_clean":"2024-07-29T00:36:30.002737+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:36:30.002737+0000","last_undegraded":"2024-07-29T00:36:30.002737+0000","last_fullsized":"2024-07-29T00:36:30.002737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006358+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:36:30.006358+0000","last_peered":"2024-07-29T00:36:30.006358+0000","last_clean":"2024-07-29T00:36:30.006358+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:36:30.006358+0000","last_undegraded":"2024-07-29T00:36:30.006358+0000","last_fullsized":"2024-07-29T00:36:30.006358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978697+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:36:29.978697+0000","last_peered":"2024-07-29T00:36:29.978697+0000","last_clean":"2024-07-29T00:36:29.978697+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:36:29.978697+0000","last_undegraded":"2024-07-29T00:36:29.978697+0000","last_fullsized":"2024-07-29T00:36:29.978697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.920897+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:36:29.920897+0000","last_peered":"2024-07-29T00:36:29.920897+0000","last_clean":"2024-07-29T00:36:29.920897+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:36:29.920897+0000","last_undegraded":"2024-07-29T00:36:29.920897+0000","last_fullsized":"2024-07-29T00:36:29.920897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.844192+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:36:30.844192+0000","last_peered":"2024-07-29T00:36:30.844192+0000","last_clean":"2024-07-29T00:36:30.844192+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:36:30.844192+0000","last_undegraded":"2024-07-29T00:36:30.844192+0000","last_fullsized":"2024-07-29T00:36:30.844192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":275,"num_read_kb":275,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918684+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:36:29.918684+0000","last_peered":"2024-07-29T00:36:29.918684+0000","last_clean":"2024-07-29T00:36:29.918684+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:36:29.918684+0000","last_undegraded":"2024-07-29T00:36:29.918684+0000","last_fullsized":"2024-07-29T00:36:29.918684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979141+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:36:29.979141+0000","last_peered":"2024-07-29T00:36:29.979141+0000","last_clean":"2024-07-29T00:36:29.979141+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:36:29.979141+0000","last_undegraded":"2024-07-29T00:36:29.979141+0000","last_fullsized":"2024-07-29T00:36:29.979141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122906+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:36:30.122906+0000","last_peered":"2024-07-29T00:36:30.122906+0000","last_clean":"2024-07-29T00:36:30.122906+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:36:30.122906+0000","last_undegraded":"2024-07-29T00:36:30.122906+0000","last_fullsized":"2024-07-29T00:36:30.122906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978649+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:36:29.978649+0000","last_peered":"2024-07-29T00:36:29.978649+0000","last_clean":"2024-07-29T00:36:29.978649+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:36:29.978649+0000","last_undegraded":"2024-07-29T00:36:29.978649+0000","last_fullsized":"2024-07-29T00:36:29.978649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918482+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:36:29.918482+0000","last_peered":"2024-07-29T00:36:29.918482+0000","last_clean":"2024-07-29T00:36:29.918482+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:36:29.918482+0000","last_undegraded":"2024-07-29T00:36:29.918482+0000","last_fullsized":"2024-07-29T00:36:29.918482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":377,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.847995+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:36:30.847995+0000","last_peered":"2024-07-29T00:36:30.847995+0000","last_clean":"2024-07-29T00:36:30.847995+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:36:30.847995+0000","last_undegraded":"2024-07-29T00:36:30.847995+0000","last_fullsized":"2024-07-29T00:36:30.847995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66470,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":208,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030244+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:36:30.030244+0000","last_peered":"2024-07-29T00:36:30.030244+0000","last_clean":"2024-07-29T00:36:30.030244+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:36:30.030244+0000","last_undegraded":"2024-07-29T00:36:30.030244+0000","last_fullsized":"2024-07-29T00:36:30.030244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119352+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:36:30.119352+0000","last_peered":"2024-07-29T00:36:30.119352+0000","last_clean":"2024-07-29T00:36:30.119352+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:36:30.119352+0000","last_undegraded":"2024-07-29T00:36:30.119352+0000","last_fullsized":"2024-07-29T00:36:30.119352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002636+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:36:30.002636+0000","last_peered":"2024-07-29T00:36:30.002636+0000","last_clean":"2024-07-29T00:36:30.002636+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:36:30.002636+0000","last_undegraded":"2024-07-29T00:36:30.002636+0000","last_fullsized":"2024-07-29T00:36:30.002636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978452+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:36:29.978452+0000","last_peered":"2024-07-29T00:36:29.978452+0000","last_clean":"2024-07-29T00:36:29.978452+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:36:29.978452+0000","last_undegraded":"2024-07-29T00:36:29.978452+0000","last_fullsized":"2024-07-29T00:36:29.978452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978851+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:36:29.978851+0000","last_peered":"2024-07-29T00:36:29.978851+0000","last_clean":"2024-07-29T00:36:29.978851+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:36:29.978851+0000","last_undegraded":"2024-07-29T00:36:29.978851+0000","last_fullsized":"2024-07-29T00:36:29.978851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":183,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.264032+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:36:30.264032+0000","last_peered":"2024-07-29T00:36:30.264032+0000","last_clean":"2024-07-29T00:36:30.264032+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:36:30.264032+0000","last_undegraded":"2024-07-29T00:36:30.264032+0000","last_fullsized":"2024-07-29T00:36:30.264032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":5,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978642+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:36:29.978642+0000","last_peered":"2024-07-29T00:36:29.978642+0000","last_clean":"2024-07-29T00:36:29.978642+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:36:29.978642+0000","last_undegraded":"2024-07-29T00:36:29.978642+0000","last_fullsized":"2024-07-29T00:36:29.978642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029821+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:36:30.029821+0000","last_peered":"2024-07-29T00:36:30.029821+0000","last_clean":"2024-07-29T00:36:30.029821+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:36:30.029821+0000","last_undegraded":"2024-07-29T00:36:30.029821+0000","last_fullsized":"2024-07-29T00:36:30.029821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978327+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:36:29.978327+0000","last_peered":"2024-07-29T00:36:29.978327+0000","last_clean":"2024-07-29T00:36:29.978327+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:36:29.978327+0000","last_undegraded":"2024-07-29T00:36:29.978327+0000","last_fullsized":"2024-07-29T00:36:29.978327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030159+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:36:30.030159+0000","last_peered":"2024-07-29T00:36:30.030159+0000","last_clean":"2024-07-29T00:36:30.030159+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:36:30.030159+0000","last_undegraded":"2024-07-29T00:36:30.030159+0000","last_fullsized":"2024-07-29T00:36:30.030159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270695+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:36:30.270695+0000","last_peered":"2024-07-29T00:36:30.270695+0000","last_clean":"2024-07-29T00:36:30.270695+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:36:30.270695+0000","last_undegraded":"2024-07-29T00:36:30.270695+0000","last_fullsized":"2024-07-29T00:36:30.270695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003638+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:36:30.003638+0000","last_peered":"2024-07-29T00:36:30.003638+0000","last_clean":"2024-07-29T00:36:30.003638+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:36:30.003638+0000","last_undegraded":"2024-07-29T00:36:30.003638+0000","last_fullsized":"2024-07-29T00:36:30.003638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979263+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:36:29.979263+0000","last_peered":"2024-07-29T00:36:29.979263+0000","last_clean":"2024-07-29T00:36:29.979263+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:36:29.979263+0000","last_undegraded":"2024-07-29T00:36:29.979263+0000","last_fullsized":"2024-07-29T00:36:29.979263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.212250+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:36:30.212250+0000","last_peered":"2024-07-29T00:36:30.212250+0000","last_clean":"2024-07-29T00:36:30.212250+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:36:30.212250+0000","last_undegraded":"2024-07-29T00:36:30.212250+0000","last_fullsized":"2024-07-29T00:36:30.212250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978805+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:36:29.978805+0000","last_peered":"2024-07-29T00:36:29.978805+0000","last_clean":"2024-07-29T00:36:29.978805+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:36:29.978805+0000","last_undegraded":"2024-07-29T00:36:29.978805+0000","last_fullsized":"2024-07-29T00:36:29.978805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.844581+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:36:30.844581+0000","last_peered":"2024-07-29T00:36:30.844581+0000","last_clean":"2024-07-29T00:36:30.844581+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:36:30.844581+0000","last_undegraded":"2024-07-29T00:36:30.844581+0000","last_fullsized":"2024-07-29T00:36:30.844581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979102+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:36:29.979102+0000","last_peered":"2024-07-29T00:36:29.979102+0000","last_clean":"2024-07-29T00:36:29.979102+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:36:29.979102+0000","last_undegraded":"2024-07-29T00:36:29.979102+0000","last_fullsized":"2024-07-29T00:36:29.979102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978819+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:36:29.978819+0000","last_peered":"2024-07-29T00:36:29.978819+0000","last_clean":"2024-07-29T00:36:29.978819+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:36:29.978819+0000","last_undegraded":"2024-07-29T00:36:29.978819+0000","last_fullsized":"2024-07-29T00:36:29.978819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918549+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:36:29.918549+0000","last_peered":"2024-07-29T00:36:29.918549+0000","last_clean":"2024-07-29T00:36:29.918549+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:36:29.918549+0000","last_undegraded":"2024-07-29T00:36:29.918549+0000","last_fullsized":"2024-07-29T00:36:29.918549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006458+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:36:30.006458+0000","last_peered":"2024-07-29T00:36:30.006458+0000","last_clean":"2024-07-29T00:36:30.006458+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:36:30.006458+0000","last_undegraded":"2024-07-29T00:36:30.006458+0000","last_fullsized":"2024-07-29T00:36:30.006458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270459+0000","last_change":"2024-07-29T00:35:51.676982+0000","last_active":"2024-07-29T00:36:30.270459+0000","last_peered":"2024-07-29T00:36:30.270459+0000","last_clean":"2024-07-29T00:36:30.270459+0000","last_became_active":"2024-07-29T00:35:51.674179+0000","last_became_peered":"2024-07-29T00:35:51.674179+0000","last_unstale":"2024-07-29T00:36:30.270459+0000","last_undegraded":"2024-07-29T00:36:30.270459+0000","last_fullsized":"2024-07-29T00:36:30.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:24:23.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002590+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:36:30.002590+0000","last_peered":"2024-07-29T00:36:30.002590+0000","last_clean":"2024-07-29T00:36:30.002590+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:36:30.002590+0000","last_undegraded":"2024-07-29T00:36:30.002590+0000","last_fullsized":"2024-07-29T00:36:30.002590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003199+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:36:30.003199+0000","last_peered":"2024-07-29T00:36:30.003199+0000","last_clean":"2024-07-29T00:36:30.003199+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:36:30.003199+0000","last_undegraded":"2024-07-29T00:36:30.003199+0000","last_fullsized":"2024-07-29T00:36:30.003199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978450+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:36:29.978450+0000","last_peered":"2024-07-29T00:36:29.978450+0000","last_clean":"2024-07-29T00:36:29.978450+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:36:29.978450+0000","last_undegraded":"2024-07-29T00:36:29.978450+0000","last_fullsized":"2024-07-29T00:36:29.978450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979139+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:36:29.979139+0000","last_peered":"2024-07-29T00:36:29.979139+0000","last_clean":"2024-07-29T00:36:29.979139+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:36:29.979139+0000","last_undegraded":"2024-07-29T00:36:29.979139+0000","last_fullsized":"2024-07-29T00:36:29.979139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":895,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039767+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:36:30.039767+0000","last_peered":"2024-07-29T00:36:30.039767+0000","last_clean":"2024-07-29T00:36:30.039767+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:36:30.039767+0000","last_undegraded":"2024-07-29T00:36:30.039767+0000","last_fullsized":"2024-07-29T00:36:30.039767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921376+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:36:29.921376+0000","last_peered":"2024-07-29T00:36:29.921376+0000","last_clean":"2024-07-29T00:36:29.921376+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:36:29.921376+0000","last_undegraded":"2024-07-29T00:36:29.921376+0000","last_fullsized":"2024-07-29T00:36:29.921376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039877+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:36:30.039877+0000","last_peered":"2024-07-29T00:36:30.039877+0000","last_clean":"2024-07-29T00:36:30.039877+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:36:30.039877+0000","last_undegraded":"2024-07-29T00:36:30.039877+0000","last_fullsized":"2024-07-29T00:36:30.039877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030289+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:36:30.030289+0000","last_peered":"2024-07-29T00:36:30.030289+0000","last_clean":"2024-07-29T00:36:30.030289+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:36:30.030289+0000","last_undegraded":"2024-07-29T00:36:30.030289+0000","last_fullsized":"2024-07-29T00:36:30.030289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003485+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:36:30.003485+0000","last_peered":"2024-07-29T00:36:30.003485+0000","last_clean":"2024-07-29T00:36:30.003485+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:36:30.003485+0000","last_undegraded":"2024-07-29T00:36:30.003485+0000","last_fullsized":"2024-07-29T00:36:30.003485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'52","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.216413+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:36:30.216413+0000","last_peered":"2024-07-29T00:36:30.216413+0000","last_clean":"2024-07-29T00:36:30.216413+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:36:30.216413+0000","last_undegraded":"2024-07-29T00:36:30.216413+0000","last_fullsized":"2024-07-29T00:36:30.216413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006932+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:36:30.006932+0000","last_peered":"2024-07-29T00:36:30.006932+0000","last_clean":"2024-07-29T00:36:30.006932+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:36:30.006932+0000","last_undegraded":"2024-07-29T00:36:30.006932+0000","last_fullsized":"2024-07-29T00:36:30.006932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919572+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:36:29.919572+0000","last_peered":"2024-07-29T00:36:29.919572+0000","last_clean":"2024-07-29T00:36:29.919572+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:36:29.919572+0000","last_undegraded":"2024-07-29T00:36:29.919572+0000","last_fullsized":"2024-07-29T00:36:29.919572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122292+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:36:30.122292+0000","last_peered":"2024-07-29T00:36:30.122292+0000","last_clean":"2024-07-29T00:36:30.122292+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:36:30.122292+0000","last_undegraded":"2024-07-29T00:36:30.122292+0000","last_fullsized":"2024-07-29T00:36:30.122292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918120+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:36:29.918120+0000","last_peered":"2024-07-29T00:36:29.918120+0000","last_clean":"2024-07-29T00:36:29.918120+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:36:29.918120+0000","last_undegraded":"2024-07-29T00:36:29.918120+0000","last_fullsized":"2024-07-29T00:36:29.918120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029897+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:36:30.029897+0000","last_peered":"2024-07-29T00:36:30.029897+0000","last_clean":"2024-07-29T00:36:30.029897+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:36:30.029897+0000","last_undegraded":"2024-07-29T00:36:30.029897+0000","last_fullsized":"2024-07-29T00:36:30.029897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006317+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:36:30.006317+0000","last_peered":"2024-07-29T00:36:30.006317+0000","last_clean":"2024-07-29T00:36:30.006317+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:36:30.006317+0000","last_undegraded":"2024-07-29T00:36:30.006317+0000","last_fullsized":"2024-07-29T00:36:30.006317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270586+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:36:30.270586+0000","last_peered":"2024-07-29T00:36:30.270586+0000","last_clean":"2024-07-29T00:36:30.270586+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:36:30.270586+0000","last_undegraded":"2024-07-29T00:36:30.270586+0000","last_fullsized":"2024-07-29T00:36:30.270586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119490+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:36:30.119490+0000","last_peered":"2024-07-29T00:36:30.119490+0000","last_clean":"2024-07-29T00:36:30.119490+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:36:30.119490+0000","last_undegraded":"2024-07-29T00:36:30.119490+0000","last_fullsized":"2024-07-29T00:36:30.119490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006281+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:36:30.006281+0000","last_peered":"2024-07-29T00:36:30.006281+0000","last_clean":"2024-07-29T00:36:30.006281+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:36:30.006281+0000","last_undegraded":"2024-07-29T00:36:30.006281+0000","last_fullsized":"2024-07-29T00:36:30.006281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029962+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:36:30.029962+0000","last_peered":"2024-07-29T00:36:30.029962+0000","last_clean":"2024-07-29T00:36:30.029962+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:36:30.029962+0000","last_undegraded":"2024-07-29T00:36:30.029962+0000","last_fullsized":"2024-07-29T00:36:30.029962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978784+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:36:29.978784+0000","last_peered":"2024-07-29T00:36:29.978784+0000","last_clean":"2024-07-29T00:36:29.978784+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:36:29.978784+0000","last_undegraded":"2024-07-29T00:36:29.978784+0000","last_fullsized":"2024-07-29T00:36:29.978784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978667+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:36:29.978667+0000","last_peered":"2024-07-29T00:36:29.978667+0000","last_clean":"2024-07-29T00:36:29.978667+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:36:29.978667+0000","last_undegraded":"2024-07-29T00:36:29.978667+0000","last_fullsized":"2024-07-29T00:36:29.978667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030262+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:36:30.030262+0000","last_peered":"2024-07-29T00:36:30.030262+0000","last_clean":"2024-07-29T00:36:30.030262+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:36:30.030262+0000","last_undegraded":"2024-07-29T00:36:30.030262+0000","last_fullsized":"2024-07-29T00:36:30.030262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918288+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:36:29.918288+0000","last_peered":"2024-07-29T00:36:29.918288+0000","last_clean":"2024-07-29T00:36:29.918288+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:36:29.918288+0000","last_undegraded":"2024-07-29T00:36:29.918288+0000","last_fullsized":"2024-07-29T00:36:29.918288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270633+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:36:30.270633+0000","last_peered":"2024-07-29T00:36:30.270633+0000","last_clean":"2024-07-29T00:36:30.270633+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:36:30.270633+0000","last_undegraded":"2024-07-29T00:36:30.270633+0000","last_fullsized":"2024-07-29T00:36:30.270633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122784+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:36:30.122784+0000","last_peered":"2024-07-29T00:36:30.122784+0000","last_clean":"2024-07-29T00:36:30.122784+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:36:30.122784+0000","last_undegraded":"2024-07-29T00:36:30.122784+0000","last_fullsized":"2024-07-29T00:36:30.122784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006887+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:36:30.006887+0000","last_peered":"2024-07-29T00:36:30.006887+0000","last_clean":"2024-07-29T00:36:30.006887+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:36:30.006887+0000","last_undegraded":"2024-07-29T00:36:30.006887+0000","last_fullsized":"2024-07-29T00:36:30.006887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921570+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:36:29.921570+0000","last_peered":"2024-07-29T00:36:29.921570+0000","last_clean":"2024-07-29T00:36:29.921570+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:36:29.921570+0000","last_undegraded":"2024-07-29T00:36:29.921570+0000","last_fullsized":"2024-07-29T00:36:29.921570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":185,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921917+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:36:29.921917+0000","last_peered":"2024-07-29T00:36:29.921917+0000","last_clean":"2024-07-29T00:36:29.921917+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:36:29.921917+0000","last_undegraded":"2024-07-29T00:36:29.921917+0000","last_fullsized":"2024-07-29T00:36:29.921917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600886,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1476,"num_read_kb":867,"num_write":379,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":1802658,"data_compressed":85489,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416429,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":492125,"data_compressed":11554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678065,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1291549,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":501069,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":894663,"data_compressed":33607,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1094412,"data_compressed":50309,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":492079,"data_compressed":11554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1225632,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":562665,"data_compressed":24044,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1025427,"data_compressed":39763,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":501529,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":568476,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":502905,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":70431,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135200,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":201117,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:36:37.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:36:37.780 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 1 op/s 2024-07-29T00:36:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:37 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 1 op/s 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:38.164 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:38.164 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:38.164 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:37 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 1 op/s 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: Updating smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/config/ceph.client.admin.keyring 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:38.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:37 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:38.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 -- 172.21.15.175:0/438011190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c70104260 msgr2=0x7f3c70110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:38.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 --2- 172.21.15.175:0/438011190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c70104260 0x7f3c70110790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009cd0 tx=0x7f3c6802a8e0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 -- 172.21.15.175:0/438011190 shutdown_connections 2024-07-29T00:36:38.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 --2- 172.21.15.175:0/438011190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c70104260 0x7f3c70110790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 --2- 172.21.15.175:0/438011190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701038c0 0x7f3c70103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 --2- 172.21.15.175:0/438011190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c701098c0 0x7f3c70109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 -- 172.21.15.175:0/438011190 >> 172.21.15.175:0/438011190 conn(0x7f3c700ff5c0 msgr2=0x7f3c701019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:38.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.473+0000 7f3c6f200640 1 -- 172.21.15.175:0/438011190 shutdown_connections 2024-07-29T00:36:38.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.474+0000 7f3c6f200640 1 -- 172.21.15.175:0/438011190 wait complete. 2024-07-29T00:36:38.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.475+0000 7f3c6f200640 1 Processor -- start 2024-07-29T00:36:38.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.475+0000 7f3c6f200640 1 -- start start 2024-07-29T00:36:38.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 0x7f3c7019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c70104260 0x7f3c7019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701098c0 0x7f3c7019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70113d10 con 0x7f3c701098c0 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70113b90 con 0x7f3c70104260 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70113e90 con 0x7f3c701038c0 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 0x7f3c7019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 0x7f3c7019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33722/0 (socket says 172.21.15.175:33722) 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.476+0000 7f3c6de00640 1 -- 172.21.15.175:0/3793541486 learned_addr learned my addr 172.21.15.175:0/3793541486 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6e800640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701098c0 0x7f3c7019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:38.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6d400640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c70104260 0x7f3c7019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:38.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6de00640 1 -- 172.21.15.175:0/3793541486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c70104260 msgr2=0x7f3c7019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:38.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6de00640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c70104260 0x7f3c7019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6de00640 1 -- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701098c0 msgr2=0x7f3c7019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6de00640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701098c0 0x7f3c7019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6de00640 1 -- 172.21.15.175:0/3793541486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7019f140 con 0x7f3c701038c0 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6e800640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701098c0 0x7f3c7019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c6de00640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 0x7f3c7019a190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9f0 tx=0x7f3c6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.477+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000cd80 con 0x7f3c701038c0 2024-07-29T00:36:38.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.478+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c60010910 con 0x7f3c701038c0 2024-07-29T00:36:38.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.478+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60018990 con 0x7f3c701038c0 2024-07-29T00:36:38.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.478+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7019f430 con 0x7f3c701038c0 2024-07-29T00:36:38.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.478+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a6d10 con 0x7f3c701038c0 2024-07-29T00:36:38.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.479+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70103d20 con 0x7f3c701038c0 2024-07-29T00:36:38.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.481+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f3c60010ac0 con 0x7f3c701038c0 2024-07-29T00:36:38.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.482+0000 7f3c66a00640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c540a2d80 0x7f3c540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:38.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.483+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3c600cfe80 con 0x7f3c701038c0 2024-07-29T00:36:38.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.483+0000 7f3c6d400640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c540a2d80 0x7f3c540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:38.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.483+0000 7f3c6d400640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c540a2d80 0x7f3c540a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3c70101b70 tx=0x7f3c5c00b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.485+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3c6009c8b0 con 0x7f3c701038c0 2024-07-29T00:36:38.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.715+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c7019b4f0 con 0x7f3c701038c0 2024-07-29T00:36:38.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.716+0000 7f3c66a00640 1 -- 172.21.15.175:0/3793541486 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f3c6009c1b0 con 0x7f3c701038c0 2024-07-29T00:36:38.717 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:38.717 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:36:38.717 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:36:38.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c540a2d80 msgr2=0x7f3c540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:38.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c540a2d80 0x7f3c540a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3c70101b70 tx=0x7f3c5c00b410 comp rx=0 tx=0).stop 2024-07-29T00:36:38.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 msgr2=0x7f3c7019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:38.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 0x7f3c7019a190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9f0 tx=0x7f3c6000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 shutdown_connections 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c540a2d80 0x7f3c540a5240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c701098c0 0x7f3c7019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c70104260 0x7f3c7019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 --2- 172.21.15.175:0/3793541486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c701038c0 0x7f3c7019a190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 >> 172.21.15.175:0/3793541486 conn(0x7f3c700ff5c0 msgr2=0x7f3c700ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 shutdown_connections 2024-07-29T00:36:38.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:38.720+0000 7f3c6f200640 1 -- 172.21.15.175:0/3793541486 wait complete. 2024-07-29T00:36:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:38 smithi175 ceph-mon[32138]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:39.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:38 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3793541486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:38 smithi204 ceph-mon[37368]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:39.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:38 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3793541486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:39.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:38 smithi181 ceph-mon[36706]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:39.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:38 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3793541486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:39.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:36:40.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:39 smithi175 ceph-mon[32138]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-07-29T00:36:40.177 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:39 smithi181 ceph-mon[36706]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-07-29T00:36:40.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:39 smithi204 ceph-mon[37368]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-07-29T00:36:40.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 -- 172.21.15.175:0/3921269658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c4102060 msgr2=0x7f27c410e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 --2- 172.21.15.175:0/3921269658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c4102060 0x7f27c410e590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009ca0 tx=0x7f27bc02a8c0 comp rx=0 tx=0).stop 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 -- 172.21.15.175:0/3921269658 shutdown_connections 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 --2- 172.21.15.175:0/3921269658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c4102060 0x7f27c410e590 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 --2- 172.21.15.175:0/3921269658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 --2- 172.21.15.175:0/3921269658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c41076c0 0x7f27c4107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 -- 172.21.15.175:0/3921269658 >> 172.21.15.175:0/3921269658 conn(0x7f27c40fd400 msgr2=0x7f27c40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 -- 172.21.15.175:0/3921269658 shutdown_connections 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.861+0000 7f27caa00640 1 -- 172.21.15.175:0/3921269658 wait complete. 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.862+0000 7f27caa00640 1 Processor -- start 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.862+0000 7f27caa00640 1 -- start start 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27caa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:40.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27caa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4102060 0x7f27c4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c41076c0 0x7f27c419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27caa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111b10 con 0x7f27c41016c0 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27caa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111990 con 0x7f27c41076c0 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27caa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111c90 con 0x7f27c4102060 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27c9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27c9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60028/0 (socket says 172.21.15.175:60028) 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27c9600640 1 -- 172.21.15.175:0/27096780 learned_addr learned my addr 172.21.15.175:0/27096780 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27c8c00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4102060 0x7f27c4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.863+0000 7f27ca000640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c41076c0 0x7f27c419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c9600640 1 -- 172.21.15.175:0/27096780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4102060 msgr2=0x7f27c4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:40.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c9600640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4102060 0x7f27c4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c9600640 1 -- 172.21.15.175:0/27096780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c41076c0 msgr2=0x7f27c419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c9600640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c41076c0 0x7f27c419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c9600640 1 -- 172.21.15.175:0/27096780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c419ceb0 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c9600640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4197f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f27b400e9f0 tx=0x7f27b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b400cd80 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c419d1a0 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b4010910 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4018910 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.864+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a4a80 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.866+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c4101b20 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.868+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f27b4010ac0 con 0x7f27c41016c0 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.869+0000 7f27c2000640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f27a80a2d80 0x7f27a80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:40.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.869+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f27b40cf950 con 0x7f27c41016c0 2024-07-29T00:36:40.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.869+0000 7f27c8c00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f27a80a2d80 0x7f27a80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:40.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.870+0000 7f27c8c00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f27a80a2d80 0x7f27a80a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27c4106c40 tx=0x7f27b0005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:40.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:40.874+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f27b409bd30 con 0x7f27c41016c0 2024-07-29T00:36:41.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.066+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27c41a4de0 con 0x7f27c41016c0 2024-07-29T00:36:41.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.071+0000 7f27c2000640 1 -- 172.21.15.175:0/27096780 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+23219 (secure 0 0 0) 0x7f27b40a0030 con 0x7f27c41016c0 2024-07-29T00:36:41.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:41.072 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":92,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:36:29.909354+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1189171033":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/3246788131":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/257505941":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/2019123994":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/4008532755":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/3246788131":"2024-07-30T00:36:29.909303+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:36:41.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.073+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f27a80a2d80 msgr2=0x7f27a80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:41.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.073+0000 7f27caa00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f27a80a2d80 0x7f27a80a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27c4106c40 tx=0x7f27b0005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:41.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.073+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 msgr2=0x7f27c4197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:41.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.073+0000 7f27caa00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4197f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f27b400e9f0 tx=0x7f27b400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 shutdown_connections 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27c41076c0 0x7f27c419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4102060 0x7f27c4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f27a80a2d80 0x7f27a80a5240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 --2- 172.21.15.175:0/27096780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c41016c0 0x7f27c4197f00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 >> 172.21.15.175:0/27096780 conn(0x7f27c40fd400 msgr2=0x7f27c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 shutdown_connections 2024-07-29T00:36:41.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:41.074+0000 7f27caa00640 1 -- 172.21.15.175:0/27096780 wait complete. 2024-07-29T00:36:41.985 DEBUG:tasks.fwd_scrub:Yielding 2024-07-29T00:36:41.986 INFO:teuthology.run_tasks:Running task exec... 2024-07-29T00:36:41.994 INFO:teuthology.task.exec:Executing custom commands... 2024-07-29T00:36:41.995 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:36:41.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-07-29T00:36:41.995 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-07-29T00:36:41.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:36:41.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 -- 172.21.15.175:0/1144306271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba810bbf0 msgr2=0x7f1ba810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 --2- 172.21.15.175:0/1144306271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba810bbf0 0x7f1ba810e060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009900 tx=0x7f1ba402a480 comp rx=0 tx=0).stop 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 -- 172.21.15.175:0/1144306271 shutdown_connections 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 --2- 172.21.15.175:0/1144306271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba810bbf0 0x7f1ba810e060 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 --2- 172.21.15.175:0/1144306271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ba81030f0 0x7f1ba810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 --2- 172.21.15.175:0/1144306271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81027d0 0x7f1ba8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 -- 172.21.15.175:0/1144306271 >> 172.21.15.175:0/1144306271 conn(0x7f1ba80774b0 msgr2=0x7f1ba80fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 -- 172.21.15.175:0/1144306271 shutdown_connections 2024-07-29T00:36:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.186+0000 7f1baf200640 1 -- 172.21.15.175:0/1144306271 wait complete. 2024-07-29T00:36:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 Processor -- start 2024-07-29T00:36:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 -- start start 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 0x7f1ba819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81030f0 0x7f1ba819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ba810bbf0 0x7f1ba819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8111a10 con 0x7f1ba81027d0 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.187+0000 7f1baf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8111890 con 0x7f1ba81030f0 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1baf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8111b90 con 0x7f1ba810bbf0 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 0x7f1ba819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 0x7f1ba819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60052/0 (socket says 172.21.15.175:60052) 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 -- 172.21.15.175:0/3936652530 learned_addr learned my addr 172.21.15.175:0/3936652530 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bad400640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81030f0 0x7f1ba819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bae800640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ba810bbf0 0x7f1ba819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 -- 172.21.15.175:0/3936652530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ba810bbf0 msgr2=0x7f1ba819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ba810bbf0 0x7f1ba819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 -- 172.21.15.175:0/3936652530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81030f0 msgr2=0x7f1ba819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81030f0 0x7f1ba819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bade00640 1 -- 172.21.15.175:0/3936652530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba819f0e0 con 0x7f1ba81027d0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.188+0000 7f1bad400640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81030f0 0x7f1ba819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.189+0000 7f1bade00640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 0x7f1ba819a0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009d50 tx=0x7f1ba000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.189+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003b70 con 0x7f1ba81027d0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.189+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba0003d10 con 0x7f1ba81027d0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.189+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0010a20 con 0x7f1ba81027d0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.189+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba819f3d0 con 0x7f1ba81027d0 2024-07-29T00:36:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.189+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba81a6c60 con 0x7f1ba81027d0 2024-07-29T00:36:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.190+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba8102bb0 con 0x7f1ba81027d0 2024-07-29T00:36:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.193+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f1ba0010bc0 con 0x7f1ba81027d0 2024-07-29T00:36:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.194+0000 7f1b9ea00640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1b940a2e00 0x7f1b940a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.194+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1ba00cfca0 con 0x7f1ba81027d0 2024-07-29T00:36:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.194+0000 7f1bad400640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1b940a2e00 0x7f1b940a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.195+0000 7f1bad400640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1b940a2e00 0x7f1b940a52c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b980095f0 tx=0x7f1b98005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.196+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1ba009c7c0 con 0x7f1ba81027d0 2024-07-29T00:36:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:41 smithi181 ceph-mon[36706]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-07-29T00:36:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:41 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/27096780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:41 smithi175 ceph-mon[32138]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-07-29T00:36:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:41 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/27096780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:41 smithi204 ceph-mon[37368]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-07-29T00:36:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:41 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/27096780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:42.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.419+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f1ba819b440 con 0x7f1ba81027d0 2024-07-29T00:36:42.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.424+0000 7f1b9ea00640 1 -- 172.21.15.175:0/3936652530 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f1ba009c0c0 con 0x7f1ba81027d0 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.427+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1b940a2e00 msgr2=0x7f1b940a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.427+0000 7f1baf200640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1b940a2e00 0x7f1b940a52c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b980095f0 tx=0x7f1b98005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.427+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 msgr2=0x7f1ba819a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.427+0000 7f1baf200640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 0x7f1ba819a0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009d50 tx=0x7f1ba000c6a0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 shutdown_connections 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ba810bbf0 0x7f1ba819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba81030f0 0x7f1ba819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1b940a2e00 0x7f1b940a52c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 --2- 172.21.15.175:0/3936652530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba81027d0 0x7f1ba819a0f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 >> 172.21.15.175:0/3936652530 conn(0x7f1ba80774b0 msgr2=0x7f1ba8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:42.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 shutdown_connections 2024-07-29T00:36:42.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.428+0000 7f1baf200640 1 -- 172.21.15.175:0/3936652530 wait complete. 2024-07-29T00:36:42.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-07-29T00:36:42.470 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 -- 172.21.15.175:0/577384469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a601017b0 msgr2=0x7f6a60101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 --2- 172.21.15.175:0/577384469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a601017b0 0x7f6a60101c10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009900 tx=0x7f6a4c02a410 comp rx=0 tx=0).stop 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 -- 172.21.15.175:0/577384469 shutdown_connections 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 --2- 172.21.15.175:0/577384469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a60102150 0x7f6a6010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 --2- 172.21.15.175:0/577384469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a601017b0 0x7f6a60101c10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 --2- 172.21.15.175:0/577384469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601077c0 0x7f6a60107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.623+0000 7f6a65200640 1 -- 172.21.15.175:0/577384469 >> 172.21.15.175:0/577384469 conn(0x7f6a600fd4d0 msgr2=0x7f6a600ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:42.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.624+0000 7f6a65200640 1 -- 172.21.15.175:0/577384469 shutdown_connections 2024-07-29T00:36:42.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.624+0000 7f6a65200640 1 -- 172.21.15.175:0/577384469 wait complete. 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.624+0000 7f6a65200640 1 Processor -- start 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 -- start start 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 0x7f6a600741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a60102150 0x7f6a60074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a601077c0 0x7f6a60074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60111c50 con 0x7f6a601077c0 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60111ad0 con 0x7f6a60102150 2024-07-29T00:36:42.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a65200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60111dd0 con 0x7f6a601017b0 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a601077c0 0x7f6a60074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a5de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 0x7f6a600741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a5de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 0x7f6a600741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58692/0 (socket says 172.21.15.175:58692) 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a5de00640 1 -- 172.21.15.175:0/2471063620 learned_addr learned my addr 172.21.15.175:0/2471063620 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.625+0000 7f6a5d400640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a60102150 0x7f6a60074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5de00640 1 -- 172.21.15.175:0/2471063620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a60102150 msgr2=0x7f6a60074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5de00640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a60102150 0x7f6a60074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5de00640 1 -- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a601077c0 msgr2=0x7f6a60074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5de00640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a601077c0 0x7f6a60074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5de00640 1 -- 172.21.15.175:0/2471063620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a601a6540 con 0x7f6a601017b0 2024-07-29T00:36:42.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5d400640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a60102150 0x7f6a60074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:42.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a5de00640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 0x7f6a600741e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000e9f0 tx=0x7f6a5000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:42.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000cd80 con 0x7f6a601017b0 2024-07-29T00:36:42.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50005500 con 0x7f6a601017b0 2024-07-29T00:36:42.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a601a6830 con 0x7f6a601017b0 2024-07-29T00:36:42.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.626+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50010690 con 0x7f6a601017b0 2024-07-29T00:36:42.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.627+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a6d50 con 0x7f6a601017b0 2024-07-29T00:36:42.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.628+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a60102ef0 con 0x7f6a601017b0 2024-07-29T00:36:42.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.630+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f6a500046c0 con 0x7f6a601017b0 2024-07-29T00:36:42.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.631+0000 7f6a56a00640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6a440a2ec0 0x7f6a440a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:42.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.631+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6a500cfa30 con 0x7f6a601017b0 2024-07-29T00:36:42.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.631+0000 7f6a5d400640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6a440a2ec0 0x7f6a440a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:42.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.632+0000 7f6a5d400640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6a440a2ec0 0x7f6a440a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c000c00 tx=0x7f6a4c005a40 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:42.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.633+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6a5009c460 con 0x7f6a601017b0 2024-07-29T00:36:42.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.850+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6a60101c10 con 0x7f6a601017b0 2024-07-29T00:36:42.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.856+0000 7f6a56a00640 1 -- 172.21.15.175:0/2471063620 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f6a60101c10 con 0x7f6a601017b0 2024-07-29T00:36:42.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6a440a2ec0 msgr2=0x7f6a440a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6a440a2ec0 0x7f6a440a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c000c00 tx=0x7f6a4c005a40 comp rx=0 tx=0).stop 2024-07-29T00:36:42.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 msgr2=0x7f6a600741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:42.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 0x7f6a600741e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000e9f0 tx=0x7f6a5000edf0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 shutdown_connections 2024-07-29T00:36:42.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6a440a2ec0 0x7f6a440a5380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a601077c0 0x7f6a60074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a60102150 0x7f6a60074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 --2- 172.21.15.175:0/2471063620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a601017b0 0x7f6a600741e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:42.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.859+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 >> 172.21.15.175:0/2471063620 conn(0x7f6a600fd4d0 msgr2=0x7f6a600fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:42.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.860+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 shutdown_connections 2024-07-29T00:36:42.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:42.860+0000 7f6a65200640 1 -- 172.21.15.175:0/2471063620 wait complete. 2024-07-29T00:36:42.880 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-07-29T00:36:43.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.115+0000 7f128fc00640 1 -- 172.21.15.175:0/2107498935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288104420 msgr2=0x7f1288110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.115+0000 7f128fc00640 1 --2- 172.21.15.175:0/2107498935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288104420 0x7f1288110950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1280009cd0 tx=0x7f128002a8e0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 -- 172.21.15.175:0/3189044347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 msgr2=0x7f242810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 --2- 172.21.15.175:0/3189044347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f242810e060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2420009860 tx=0x7f242002a3d0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 -- 172.21.15.175:0/3189044347 shutdown_connections 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 --2- 172.21.15.175:0/3189044347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f242810e060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 --2- 172.21.15.175:0/3189044347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281030f0 0x7f242810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 --2- 172.21.15.175:0/3189044347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24281027d0 0x7f2428102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 -- 172.21.15.175:0/3189044347 >> 172.21.15.175:0/3189044347 conn(0x7f24280774b0 msgr2=0x7f24280fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 -- 172.21.15.175:0/2107498935 shutdown_connections 2024-07-29T00:36:43.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 --2- 172.21.15.175:0/2107498935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288104420 0x7f1288110950 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 --2- 172.21.15.175:0/2107498935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1288103a80 0x7f1288103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 --2- 172.21.15.175:0/2107498935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288109a80 0x7f1288109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 -- 172.21.15.175:0/2107498935 >> 172.21.15.175:0/2107498935 conn(0x7f12880ff780 msgr2=0x7f1288101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 -- 172.21.15.175:0/2107498935 shutdown_connections 2024-07-29T00:36:43.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f128fc00640 1 -- 172.21.15.175:0/2107498935 wait complete. 2024-07-29T00:36:43.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 -- 172.21.15.175:0/3189044347 shutdown_connections 2024-07-29T00:36:43.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.116+0000 7f242ec00640 1 -- 172.21.15.175:0/3189044347 wait complete. 2024-07-29T00:36:43.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 Processor -- start 2024-07-29T00:36:43.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 -- start start 2024-07-29T00:36:43.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f242ec00640 1 Processor -- start 2024-07-29T00:36:43.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f242ec00640 1 -- start start 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 0x7f128819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1288104420 0x7f128819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288109a80 0x7f128819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1288113ed0 con 0x7f1288104420 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1288113d50 con 0x7f1288109a80 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.117+0000 7f128fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1288114050 con 0x7f1288103a80 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 0x7f128819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 0x7f128819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58714/0 (socket says 172.21.15.175:58714) 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 -- 172.21.15.175:0/212744944 learned_addr learned my addr 172.21.15.175:0/212744944 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f1287e00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1288104420 0x7f128819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128dc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288109a80 0x7f128819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 0x7f2428100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24281030f0 0x7f2428100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f2428100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2428111980 con 0x7f24281030f0 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2428111800 con 0x7f242810bbf0 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2428111b00 con 0x7f24281027d0 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 0x7f2428100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 0x7f2428100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58722/0 (socket says 172.21.15.175:58722) 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242d800640 1 -- 172.21.15.175:0/42836050 learned_addr learned my addr 172.21.15.175:0/42836050 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f242e200640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f2428100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 -- 172.21.15.175:0/212744944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288109a80 msgr2=0x7f128819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288109a80 0x7f128819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 -- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1288104420 msgr2=0x7f128819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1288104420 0x7f128819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.118+0000 7f128d200640 1 -- 172.21.15.175:0/212744944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128819f2c0 con 0x7f1288103a80 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242d800640 1 -- 172.21.15.175:0/42836050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 msgr2=0x7f2428100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242d800640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f2428100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242d800640 1 -- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24281030f0 msgr2=0x7f2428100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242ce00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24281030f0 0x7f2428100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242d800640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24281030f0 0x7f2428100960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242d800640 1 -- 172.21.15.175:0/42836050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2428197be0 con 0x7f24281027d0 2024-07-29T00:36:43.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242e200640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f2428100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:43.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f128d200640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 0x7f128819a310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e9f0 tx=0x7f127c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f1285600640 1 -- 172.21.15.175:0/212744944 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c00cd80 con 0x7f1288103a80 2024-07-29T00:36:43.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f1285600640 1 -- 172.21.15.175:0/212744944 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c005500 con 0x7f1288103a80 2024-07-29T00:36:43.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f1285600640 1 -- 172.21.15.175:0/212744944 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c010690 con 0x7f1288103a80 2024-07-29T00:36:43.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f242d800640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 0x7f2428100420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f241c00efb0 tx=0x7f241c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128819f5b0 con 0x7f1288103a80 2024-07-29T00:36:43.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.120+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c01b070 con 0x7f24281027d0 2024-07-29T00:36:43.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.120+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2428197ed0 con 0x7f24281027d0 2024-07-29T00:36:43.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.120+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2428198470 con 0x7f24281027d0 2024-07-29T00:36:43.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.119+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12881a6e90 con 0x7f1288103a80 2024-07-29T00:36:43.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.120+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241c00ba50 con 0x7f24281027d0 2024-07-29T00:36:43.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.120+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c016c00 con 0x7f24281027d0 2024-07-29T00:36:43.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.121+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2408005290 con 0x7f24281027d0 2024-07-29T00:36:43.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.123+0000 7f1285600640 1 -- 172.21.15.175:0/212744944 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f127c0046c0 con 0x7f1288103a80 2024-07-29T00:36:43.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.123+0000 7f129262c740 1 Processor -- start 2024-07-29T00:36:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.123+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f241c01e070 con 0x7f24281027d0 2024-07-29T00:36:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 -- start start 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d15f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d23110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563f43c7b8a0 con 0x563f43d16460 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563f43c13710 con 0x563f43d15b40 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f129262c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563f43c77860 con 0x563f43d20d20 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f128d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f128dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d15f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f1287e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d23110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f128d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60114/0 (socket says 172.21.15.175:60114) 2024-07-29T00:36:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.124+0000 7f128d200640 1 -- 172.21.15.175:0/3371850374 learned_addr learned my addr 172.21.15.175:0/3371850374 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f2426000640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f24100a2e00 0x7f24100a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f128dc00640 1 -- 172.21.15.175:0/3371850374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 msgr2=0x563f43d23110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f241c0cf9b0 con 0x7f24281027d0 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f128dc00640 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d23110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f128dc00640 1 -- 172.21.15.175:0/3371850374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 msgr2=0x563f43d207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f128dc00640 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f128dc00640 1 -- 172.21.15.175:0/3371850374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f439ed6a0 con 0x563f43d15b40 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f1285600640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f12700a2e00 0x7f12700a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.125+0000 7f1285600640 1 -- 172.21.15.175:0/212744944 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f127c0cf920 con 0x7f1288103a80 2024-07-29T00:36:43.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.126+0000 7f242ce00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f24100a2e00 0x7f24100a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.126+0000 7f128d200640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f12700a2e00 0x7f12700a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.126+0000 7f128dc00640 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d15f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ca0 tx=0x7f1280009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75cf3f371b7de58 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.127+0000 7f242ce00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f24100a2e00 0x7f24100a52c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2428101eb0 tx=0x7f2414005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.126+0000 7f127a000640 1 -- 172.21.15.175:0/3371850374 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280049070 con 0x563f43d15b40 2024-07-29T00:36:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.126+0000 7f127a000640 1 -- 172.21.15.175:0/3371850374 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1280036040 con 0x563f43d15b40 2024-07-29T00:36:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.127+0000 7f127a000640 1 -- 172.21.15.175:0/3371850374 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280004040 con 0x563f43d15b40 2024-07-29T00:36:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.127+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f241c09c3e0 con 0x7f24281027d0 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.127+0000 7f128d200640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f12700a2e00 0x7f12700a52c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f127c0980b0 tx=0x7f127c0d5480 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.127+0000 7f129262c740 1 -- 172.21.15.175:0/3371850374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 msgr2=0x563f43d15f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.127+0000 7f129262c740 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d15f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ca0 tx=0x7f1280009a20 comp rx=0 tx=0).stop 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 -- 172.21.15.175:0/3371850374 shutdown_connections 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d23110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 --2- 172.21.15.175:0/3371850374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d15f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 -- 172.21.15.175:0/3371850374 >> 172.21.15.175:0/3371850374 conn(0x563f43d120d0 msgr2=0x563f43d124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 -- 172.21.15.175:0/3371850374 shutdown_connections 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 -- 172.21.15.175:0/3371850374 wait complete. 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.128+0000 7f129262c740 1 -- start start 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 Processor -- start 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d1dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d1e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d1e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563f43d23990 con 0x563f43d16460 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563f43c368c0 con 0x563f43d15b40 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563f43d23b10 con 0x563f43d20d20 2024-07-29T00:36:43.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.129+0000 7f129262c740 10 client.0 _ll_register_callbacks cb 0x563f43c22a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d1dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d1dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:50322/0 (socket says 172.21.15.175:50322) 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 -- 172.21.15.175:0/2089281099 learned_addr learned my addr 172.21.15.175:0/2089281099 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f1287e00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d1e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d1e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 -- 172.21.15.175:0/2089281099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 msgr2=0x563f43d1e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d1e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 -- 172.21.15.175:0/2089281099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 msgr2=0x563f43d1e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d1e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f128dc00640 1 -- 172.21.15.175:0/2089281099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f43c19bc0 con 0x563f43d15b40 2024-07-29T00:36:43.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.130+0000 7f1287e00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d1e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f128dc00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d1dcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1280047040 tx=0x7f1280004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92d2102bef33111 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f126ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f126ea00640 1 -- 172.21.15.175:0/2089281099 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280049070 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1280036040 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f126ea00640 1 -- 172.21.15.175:0/2089281099 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1280031ad0 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f126ea00640 1 -- 172.21.15.175:0/2089281099 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1280031cd0 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f129262c740 10 client.25268 fetch_fsmap learned FSMap version 29 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.131+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f43d23990 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f126ea00640 1 -- 172.21.15.175:0/2089281099 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1280036040 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f126ea00640 1 -- 172.21.15.175:0/2089281099 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1280030070 con 0x563f43d15b40 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f129262c740 10 client.25268 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f129262c740 10 client.25268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f129262c740 20 client.25268 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:43.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f129262c740 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43d39420 0x563f43d3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f129262c740 4 client.25268 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f43c3bc90 con 0x563f43d39420 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.132+0000 7f1287e00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43d39420 0x563f43d3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.133+0000 7f126ea00640 10 client.25268 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.133+0000 7f1287e00640 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43d39420 0x563f43d3b810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8408dfb0b50a4d7f server_cookie=6dead475f09b851a in_seq=0 out_seq=0 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.134+0000 7f126ea00640 1 -- 172.21.15.175:0/2089281099 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563f43c3bc90 con 0x563f43d39420 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.134+0000 7f126ea00640 10 client.25268 handle_command_reply: tid=0 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.134+0000 7f129262c740 1 client.25268 shutdown 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.134+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 msgr2=0x563f43d1dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.134+0000 7f129262c740 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d1dcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1280047040 tx=0x7f1280004a40 comp rx=0 tx=0).stop 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 shutdown_connections 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43d39420 0x563f43d3b810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d1e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d1e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 --2- 172.21.15.175:0/2089281099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d1dcd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f126ea00640 0 client.25268 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 >> 172.21.15.175:0/2089281099 conn(0x563f43d120d0 msgr2=0x563f43d17e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 shutdown_connections 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 1 -- 172.21.15.175:0/2089281099 wait complete. 2024-07-29T00:36:43.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.135+0000 7f129262c740 20 client.25268 trim_cache size 0 max 16384 2024-07-29T00:36:43.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 Processor -- start 2024-07-29T00:36:43.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 -- start start 2024-07-29T00:36:43.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 0x563f43d185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d16460 0x563f43d1fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d20d20 0x563f43d40ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ab480 con 0x563f43d20d20 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f12881acfc0 con 0x563f43d16460 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.154+0000 7f129262c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12881abb50 con 0x563f43d15b40 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 0x563f43d185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 0x563f43d185a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58756/0 (socket says 172.21.15.175:58756) 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 -- 172.21.15.175:0/3552430958 learned_addr learned my addr 172.21.15.175:0/3552430958 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d20d20 0x563f43d40ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f1287e00640 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d16460 0x563f43d1fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 -- 172.21.15.175:0/3552430958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d16460 msgr2=0x563f43d1fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d16460 0x563f43d1fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 -- 172.21.15.175:0/3552430958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d20d20 msgr2=0x563f43d40ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d20d20 0x563f43d40ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 -- 172.21.15.175:0/3552430958 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f43d0da90 con 0x563f43d15b40 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f1287e00640 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d16460 0x563f43d1fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.155+0000 7f128dc00640 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 0x563f43d185a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1280004fd0 tx=0x7f1280009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd9bd607cd956a1e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.156+0000 7f1279600640 1 -- 172.21.15.175:0/3552430958 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280049070 con 0x563f43d15b40 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.156+0000 7f1279600640 1 -- 172.21.15.175:0/3552430958 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1280036040 con 0x563f43d15b40 2024-07-29T00:36:43.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.156+0000 7f1279600640 1 -- 172.21.15.175:0/3552430958 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280004180 con 0x563f43d15b40 2024-07-29T00:36:43.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.156+0000 7f129262c740 1 -- 172.21.15.175:0/3552430958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 msgr2=0x563f43d185a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.156+0000 7f129262c740 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 0x563f43d185a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1280004fd0 tx=0x7f1280009a20 comp rx=0 tx=0).stop 2024-07-29T00:36:43.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 -- 172.21.15.175:0/3552430958 shutdown_connections 2024-07-29T00:36:43.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d20d20 0x563f43d40ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d16460 0x563f43d1fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 --2- 172.21.15.175:0/3552430958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d15b40 0x563f43d185a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 -- 172.21.15.175:0/3552430958 >> 172.21.15.175:0/3552430958 conn(0x563f43d120d0 msgr2=0x563f43d1f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 -- 172.21.15.175:0/3552430958 shutdown_connections 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 -- 172.21.15.175:0/3552430958 wait complete. 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.157+0000 7f129262c740 1 -- start start 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 Processor -- start 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d24610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d24b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ac8f0 con 0x563f43d16460 2024-07-29T00:36:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ac220 con 0x563f43d15b40 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12881a9480 con 0x563f43d20d20 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.158+0000 7f129262c740 10 client.0 _ll_register_callbacks cb 0x7f1280009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d24b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d24b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58772/0 (socket says 172.21.15.175:58772) 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 -- 172.21.15.175:0/2231551900 learned_addr learned my addr 172.21.15.175:0/2231551900 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f1287e00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d24610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 -- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 msgr2=0x563f43d292e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d292e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 -- 172.21.15.175:0/2231551900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 msgr2=0x563f43d24610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d24610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f128dc00640 1 -- 172.21.15.175:0/2231551900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f43d25190 con 0x563f43d20d20 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.159+0000 7f1287e00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d24610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f128dc00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d24b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1280031ef0 tx=0x7f1280003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ef623e2a7844c80 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f126ea00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:36:43.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f126ea00640 1 -- 172.21.15.175:0/2231551900 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280049070 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f126ea00640 1 -- 172.21.15.175:0/2231551900 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f128002e070 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1280036040 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f126ea00640 1 -- 172.21.15.175:0/2231551900 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1280031ad0 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f129262c740 10 client.25270 fetch_fsmap learned FSMap version 29 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.160+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f43da3ce0 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f126ea00640 1 -- 172.21.15.175:0/2231551900 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1280036040 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f126ea00640 1 -- 172.21.15.175:0/2231551900 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1280030050 con 0x563f43d20d20 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f129262c740 10 client.25270 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f129262c740 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f129262c740 20 client.25270 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:43.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f129262c740 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43da41b0 0x563f43da65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f129262c740 4 client.25270 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:36:43.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f43c88e10 con 0x563f43da41b0 2024-07-29T00:36:43.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.161+0000 7f1287e00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43da41b0 0x563f43da65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.162+0000 7f1287e00640 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43da41b0 0x563f43da65a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0fa75e5d3d7f server_cookie=980069a426301e83 in_seq=0 out_seq=0 2024-07-29T00:36:43.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.162+0000 7f126ea00640 10 client.25270 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.163+0000 7f126ea00640 1 -- 172.21.15.175:0/2231551900 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f43c88e10 con 0x563f43da41b0 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.163+0000 7f126ea00640 10 client.25270 handle_command_reply: tid=0 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.163+0000 7f129262c740 1 client.25270 shutdown 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 msgr2=0x563f43d24b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d24b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1280031ef0 tx=0x7f1280003d20 comp rx=0 tx=0).stop 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 shutdown_connections 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563f43da41b0 0x563f43da65a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f43d20d20 0x563f43d24b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f43d16460 0x563f43d24610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f126ea00640 0 client.25270 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 --2- 172.21.15.175:0/2231551900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f43d15b40 0x563f43d292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 >> 172.21.15.175:0/2231551900 conn(0x563f43d120d0 msgr2=0x563f43d11230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.164+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 shutdown_connections 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.165+0000 7f129262c740 1 -- 172.21.15.175:0/2231551900 wait complete. 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.165+0000 7f129262c740 20 client.25270 trim_cache size 0 max 16384 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "2a9f3795-42e1-4c45-885d-35ce50b77968", 2024-07-29T00:36:43.167 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f12700a2e00 msgr2=0x7f12700a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f12700a2e00 0x7f12700a52c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f127c0980b0 tx=0x7f127c0d5480 comp rx=0 tx=0).stop 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 msgr2=0x7f128819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 0x7f128819a310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e9f0 tx=0x7f127c00edf0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 shutdown_connections 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1288109a80 0x7f128819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f12700a2e00 0x7f12700a52c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1288104420 0x7f128819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 --2- 172.21.15.175:0/212744944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1288103a80 0x7f128819a310 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 >> 172.21.15.175:0/212744944 conn(0x7f12880ff780 msgr2=0x7f12880ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 shutdown_connections 2024-07-29T00:36:43.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.166+0000 7f128fc00640 1 -- 172.21.15.175:0/212744944 wait complete. 2024-07-29T00:36:43.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.369+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f2408005580 con 0x7f24281027d0 2024-07-29T00:36:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.377+0000 7f2426000640 1 -- 172.21.15.175:0/42836050 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f241c09bce0 con 0x7f24281027d0 2024-07-29T00:36:43.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f24100a2e00 msgr2=0x7f24100a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f24100a2e00 0x7f24100a52c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2428101eb0 tx=0x7f2414005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:43.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 msgr2=0x7f2428100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 0x7f2428100420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f241c00efb0 tx=0x7f241c00c800 comp rx=0 tx=0).stop 2024-07-29T00:36:43.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 shutdown_connections 2024-07-29T00:36:43.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242810bbf0 0x7f2428100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f24100a2e00 0x7f24100a52c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24281030f0 0x7f2428100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 --2- 172.21.15.175:0/42836050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24281027d0 0x7f2428100420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 >> 172.21.15.175:0/42836050 conn(0x7f24280774b0 msgr2=0x7f24280fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 shutdown_connections 2024-07-29T00:36:43.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.381+0000 7f242ec00640 1 -- 172.21.15.175:0/42836050 wait complete. 2024-07-29T00:36:43.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 -- 172.21.15.175:0/1411750383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c074280 msgr2=0x7f684c074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 --2- 172.21.15.175:0/1411750383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c074280 0x7f684c074660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f683c009900 tx=0x7f683c02a480 comp rx=0 tx=0).stop 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 -- 172.21.15.175:0/1411750383 shutdown_connections 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 --2- 172.21.15.175:0/1411750383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c1125b0 0x7f684c1149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 --2- 172.21.15.175:0/1411750383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c074ba0 0x7f684c112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 --2- 172.21.15.175:0/1411750383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c074280 0x7f684c074660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 -- 172.21.15.175:0/1411750383 >> 172.21.15.175:0/1411750383 conn(0x7f684c0fd2c0 msgr2=0x7f684c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 -- 172.21.15.175:0/1411750383 shutdown_connections 2024-07-29T00:36:43.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.586+0000 7f6850a00640 1 -- 172.21.15.175:0/1411750383 wait complete. 2024-07-29T00:36:43.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.587+0000 7f6850a00640 1 Processor -- start 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 -- start start 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 0x7f684c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c074ba0 0x7f684c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c1125b0 0x7f684c1a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f684c118000 con 0x7f684c074280 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f684c117e80 con 0x7f684c074ba0 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6850a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f684c118180 con 0x7f684c1125b0 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f684b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 0x7f684c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f684b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 0x7f684c19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60164/0 (socket says 172.21.15.175:60164) 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f684b400640 1 -- 172.21.15.175:0/2588208853 learned_addr learned my addr 172.21.15.175:0/2588208853 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f684be00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c1125b0 0x7f684c1a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.588+0000 7f6842a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c074ba0 0x7f684c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684b400640 1 -- 172.21.15.175:0/2588208853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c1125b0 msgr2=0x7f684c1a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684b400640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c1125b0 0x7f684c1a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684b400640 1 -- 172.21.15.175:0/2588208853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c074ba0 msgr2=0x7f684c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684b400640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c074ba0 0x7f684c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684b400640 1 -- 172.21.15.175:0/2588208853 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c1a3520 con 0x7f684c074280 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684be00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c1125b0 0x7f684c1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f684b400640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 0x7f684c19e530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f683c004770 tx=0x7f683c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.589+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c03f070 con 0x7f684c074280 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.590+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c1a37b0 con 0x7f684c074280 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.590+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f683c031ca0 con 0x7f684c074280 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.590+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c049060 con 0x7f684c074280 2024-07-29T00:36:43.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.590+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c1ab0a0 con 0x7f684c074280 2024-07-29T00:36:43.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.591+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c074660 con 0x7f684c074280 2024-07-29T00:36:43.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.593+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f683c042070 con 0x7f684c074280 2024-07-29T00:36:43.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.594+0000 7f6848c00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68300a2d80 0x7f68300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:43.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.594+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f683c0f0090 con 0x7f684c074280 2024-07-29T00:36:43.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.594+0000 7f6842a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68300a2d80 0x7f68300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:43.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.595+0000 7f6842a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68300a2d80 0x7f68300a5240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f684c0ffae0 tx=0x7f6834005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:43.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.598+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f683c0bc470 con 0x7f684c074280 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3936652530' entity='client.admin' 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.662 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.663 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:43 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-07-29T00:36:43.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3936652530' entity='client.admin' 2024-07-29T00:36:43.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.751 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:43 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3936652530' entity='client.admin' 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' 2024-07-29T00:36:43.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:36:43.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:36:43.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:36:43.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:43 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:36:43.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.839+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f684c062320 con 0x7f68300a2d80 2024-07-29T00:36:43.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.956+0000 7f6848c00640 1 -- 172.21.15.175:0/2588208853 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f684c062320 con 0x7f68300a2d80 2024-07-29T00:36:43.956 INFO:teuthology.orchestra.run.smithi175.stdout:Schedule set for path / 2024-07-29T00:36:43.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68300a2d80 msgr2=0x7f68300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68300a2d80 0x7f68300a5240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f684c0ffae0 tx=0x7f6834005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 msgr2=0x7f684c19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 0x7f684c19e530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f683c004770 tx=0x7f683c003c10 comp rx=0 tx=0).stop 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 shutdown_connections 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c1125b0 0x7f684c1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c074ba0 0x7f684c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68300a2d80 0x7f68300a5240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 --2- 172.21.15.175:0/2588208853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f684c074280 0x7f684c19e530 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.959+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 >> 172.21.15.175:0/2588208853 conn(0x7f684c0fd2c0 msgr2=0x7f684c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.960+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 shutdown_connections 2024-07-29T00:36:43.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:43.960+0000 7f6850a00640 1 -- 172.21.15.175:0/2588208853 wait complete. 2024-07-29T00:36:43.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-07-29T00:36:44.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:36:44.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 -- 172.21.15.175:0/2240611284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520103170 msgr2=0x7fd52010b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 --2- 172.21.15.175:0/2240611284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520103170 0x7fd52010b650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009900 tx=0x7fd50c02a480 comp rx=0 tx=0).stop 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 -- 172.21.15.175:0/2240611284 shutdown_connections 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 --2- 172.21.15.175:0/2240611284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd52010bc20 0x7fd52010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 --2- 172.21.15.175:0/2240611284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520103170 0x7fd52010b650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 --2- 172.21.15.175:0/2240611284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520102850 0x7fd520102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 -- 172.21.15.175:0/2240611284 >> 172.21.15.175:0/2240611284 conn(0x7fd5200774b0 msgr2=0x7fd5200fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 -- 172.21.15.175:0/2240611284 shutdown_connections 2024-07-29T00:36:44.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.168+0000 7fd526a00640 1 -- 172.21.15.175:0/2240611284 wait complete. 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.169+0000 7fd526a00640 1 Processor -- start 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.169+0000 7fd526a00640 1 -- start start 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.169+0000 7fd526a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520102850 0x7fd52019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.169+0000 7fd526a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 0x7fd52019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.169+0000 7fd526a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd52010bc20 0x7fd52019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.169+0000 7fd526a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd520111a10 con 0x7fd520103170 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd526a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd520111890 con 0x7fd520102850 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd526a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd520111b90 con 0x7fd52010bc20 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 0x7fd52019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 0x7fd52019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60188/0 (socket says 172.21.15.175:60188) 2024-07-29T00:36:44.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 -- 172.21.15.175:0/1346504815 learned_addr learned my addr 172.21.15.175:0/1346504815 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd525600640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520102850 0x7fd52019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd526000640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd52010bc20 0x7fd52019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 -- 172.21.15.175:0/1346504815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd52010bc20 msgr2=0x7fd52019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd52010bc20 0x7fd52019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 -- 172.21.15.175:0/1346504815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520102850 msgr2=0x7fd52019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520102850 0x7fd52019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.170+0000 7fd524c00640 1 -- 172.21.15.175:0/1346504815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52019f0d0 con 0x7fd520103170 2024-07-29T00:36:44.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.171+0000 7fd524c00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 0x7fd52019a620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd50c004770 tx=0x7fd50c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:44.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.171+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c03f070 con 0x7fd520103170 2024-07-29T00:36:44.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.171+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52019f360 con 0x7fd520103170 2024-07-29T00:36:44.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.171+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd50c031ca0 con 0x7fd520103170 2024-07-29T00:36:44.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.171+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c035030 con 0x7fd520103170 2024-07-29T00:36:44.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.171+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5201a6c50 con 0x7fd520103170 2024-07-29T00:36:44.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.173+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd520102c30 con 0x7fd520103170 2024-07-29T00:36:44.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.175+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7fd50c042070 con 0x7fd520103170 2024-07-29T00:36:44.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.176+0000 7fd51e000640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5080a2d80 0x7fd5080a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.176+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd50c0efcf0 con 0x7fd520103170 2024-07-29T00:36:44.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.176+0000 7fd525600640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5080a2d80 0x7fd5080a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.177+0000 7fd525600640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5080a2d80 0x7fd5080a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd518005fd0 tx=0x7fd518009760 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:44.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.184+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd50c0bc0d0 con 0x7fd520103170 2024-07-29T00:36:44.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.412+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd5200fd1f0 con 0x7fd5080a2d80 2024-07-29T00:36:44.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.498+0000 7fd51e000640 1 -- 172.21.15.175:0/1346504815 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd5200fd1f0 con 0x7fd5080a2d80 2024-07-29T00:36:44.498 INFO:teuthology.orchestra.run.smithi175.stdout:Retention added to path / 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5080a2d80 msgr2=0x7fd5080a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5080a2d80 0x7fd5080a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd518005fd0 tx=0x7fd518009760 comp rx=0 tx=0).stop 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 msgr2=0x7fd52019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 0x7fd52019a620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd50c004770 tx=0x7fd50c003c10 comp rx=0 tx=0).stop 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 shutdown_connections 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd52010bc20 0x7fd52019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5080a2d80 0x7fd5080a5240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd520103170 0x7fd52019a620 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 --2- 172.21.15.175:0/1346504815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520102850 0x7fd52019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 >> 172.21.15.175:0/1346504815 conn(0x7fd5200774b0 msgr2=0x7fd5200fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:44.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 shutdown_connections 2024-07-29T00:36:44.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.501+0000 7fd526a00640 1 -- 172.21.15.175:0/1346504815 wait complete. 2024-07-29T00:36:44.504 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:44.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-29T00:36:44.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:44 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:36:44.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:44 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:36:44.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:44 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:36:44.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:44 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:36:44.679 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:44 smithi175 ceph-mon[32138]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:44.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 -- 172.21.15.175:0/2996554415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc107680 msgr2=0x7f17bc107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 --2- 172.21.15.175:0/2996554415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc107680 0x7f17bc107a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009900 tx=0x7f17b002a410 comp rx=0 tx=0).stop 2024-07-29T00:36:44.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 -- 172.21.15.175:0/2996554415 shutdown_connections 2024-07-29T00:36:44.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 --2- 172.21.15.175:0/2996554415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc101f60 0x7f17bc10e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 --2- 172.21.15.175:0/2996554415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17bc1015c0 0x7f17bc101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 --2- 172.21.15.175:0/2996554415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc107680 0x7f17bc107a60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 -- 172.21.15.175:0/2996554415 >> 172.21.15.175:0/2996554415 conn(0x7f17bc0fd2c0 msgr2=0x7f17bc0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:44.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 -- 172.21.15.175:0/2996554415 shutdown_connections 2024-07-29T00:36:44.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.699+0000 7f17c2600640 1 -- 172.21.15.175:0/2996554415 wait complete. 2024-07-29T00:36:44.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.700+0000 7f17c2600640 1 Processor -- start 2024-07-29T00:36:44.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.700+0000 7f17c2600640 1 -- start start 2024-07-29T00:36:44.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.700+0000 7f17c2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc1015c0 0x7f17bc0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.700+0000 7f17c2600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 0x7f17bc074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c2600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17bc107680 0x7f17bc074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c2600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111a20 con 0x7f17bc101f60 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1118a0 con 0x7f17bc1015c0 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c2600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111ba0 con 0x7f17bc107680 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 0x7f17bc074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc1015c0 0x7f17bc0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 0x7f17bc074720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60210/0 (socket says 172.21.15.175:60210) 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 -- 172.21.15.175:0/2049146815 learned_addr learned my addr 172.21.15.175:0/2049146815 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c1c00640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17bc107680 0x7f17bc074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 -- 172.21.15.175:0/2049146815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17bc107680 msgr2=0x7f17bc074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17bc107680 0x7f17bc074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 -- 172.21.15.175:0/2049146815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc1015c0 msgr2=0x7f17bc0741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc1015c0 0x7f17bc0741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17bbe00640 1 -- 172.21.15.175:0/2049146815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc1a6370 con 0x7f17bc101f60 2024-07-29T00:36:44.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.701+0000 7f17c1200640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc1015c0 0x7f17bc0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:44.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.702+0000 7f17bbe00640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 0x7f17bc074720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17a800e9f0 tx=0x7f17a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:44.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.702+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800cd80 con 0x7f17bc101f60 2024-07-29T00:36:44.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.702+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8005500 con 0x7f17bc101f60 2024-07-29T00:36:44.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.702+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc1a6660 con 0x7f17bc101f60 2024-07-29T00:36:44.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.702+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8010690 con 0x7f17bc101f60 2024-07-29T00:36:44.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.702+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc1a6b60 con 0x7f17bc101f60 2024-07-29T00:36:44.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.703+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17bc101a20 con 0x7f17bc101f60 2024-07-29T00:36:44.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.705+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f17a80046c0 con 0x7f17bc101f60 2024-07-29T00:36:44.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.707+0000 7f17b9600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17a40a2d80 0x7f17a40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:44.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.707+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f17a80cf9f0 con 0x7f17bc101f60 2024-07-29T00:36:44.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.707+0000 7f17c1200640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17a40a2d80 0x7f17a40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:44.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.708+0000 7f17c1200640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17a40a2d80 0x7f17a40a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000c00 tx=0x7f17b0005b50 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:44.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.711+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f17a809c480 con 0x7f17bc101f60 2024-07-29T00:36:44.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:44 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:36:44.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:44 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:36:44.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:44 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:36:44.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:44 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:36:44.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:44 smithi204 ceph-mon[37368]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:44.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:44 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:36:44.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:44 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:36:44.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:44 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:36:44.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:44 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:36:44.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:44 smithi181 ceph-mon[36706]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:44.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.944+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f17bc062320 con 0x7f17a40a2d80 2024-07-29T00:36:44.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.956+0000 7f17b9600640 1 -- 172.21.15.175:0/2049146815 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f17bc062320 con 0x7f17a40a2d80 2024-07-29T00:36:44.956 INFO:teuthology.orchestra.run.smithi175.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-29T00:00:00", "created": "2024-07-29T00:36:43", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-07-29T00:36:44.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17a40a2d80 msgr2=0x7f17a40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17a40a2d80 0x7f17a40a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000c00 tx=0x7f17b0005b50 comp rx=0 tx=0).stop 2024-07-29T00:36:44.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 msgr2=0x7f17bc074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:44.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 0x7f17bc074720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17a800e9f0 tx=0x7f17a800edf0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 shutdown_connections 2024-07-29T00:36:44.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17bc107680 0x7f17bc074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17a40a2d80 0x7f17a40a5240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17bc101f60 0x7f17bc074720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 --2- 172.21.15.175:0/2049146815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc1015c0 0x7f17bc0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:44.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 >> 172.21.15.175:0/2049146815 conn(0x7f17bc0fd2c0 msgr2=0x7f17bc0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:44.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 shutdown_connections 2024-07-29T00:36:44.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:44.959+0000 7f17c2600640 1 -- 172.21.15.175:0/2049146815 wait complete. 2024-07-29T00:36:44.978 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-29T00:36:45.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 -- 172.21.15.175:0/4227087870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 msgr2=0x7fa58c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 --2- 172.21.15.175:0/4227087870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 0x7fa58c102e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa580009be0 tx=0x7fa58002a750 comp rx=0 tx=0).stop 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 -- 172.21.15.175:0/4227087870 shutdown_connections 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 --2- 172.21.15.175:0/4227087870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 0x7fa58c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 --2- 172.21.15.175:0/4227087870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 --2- 172.21.15.175:0/4227087870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 0x7fa58c102e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.139+0000 7fa593e00640 1 -- 172.21.15.175:0/4227087870 >> 172.21.15.175:0/4227087870 conn(0x7fa58c0fc490 msgr2=0x7fa58c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.140+0000 7fa593e00640 1 -- 172.21.15.175:0/4227087870 shutdown_connections 2024-07-29T00:36:45.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.140+0000 7fa593e00640 1 -- 172.21.15.175:0/4227087870 wait complete. 2024-07-29T00:36:45.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.141+0000 7fa593e00640 1 Processor -- start 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.141+0000 7fa593e00640 1 -- start start 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa593e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 0x7fa58c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa593e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa593e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 0x7fa58c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa593e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c111c60 con 0x7fa58c103350 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa593e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c111ae0 con 0x7fa58c10be60 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa593e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c111de0 con 0x7fa58c102a30 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa590a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa590a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60230/0 (socket says 172.21.15.175:60230) 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa590a00640 1 -- 172.21.15.175:0/4156766937 learned_addr learned my addr 172.21.15.175:0/4156766937 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa591400640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 0x7fa58c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa591e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 0x7fa58c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa590a00640 1 -- 172.21.15.175:0/4156766937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 msgr2=0x7fa58c19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.142+0000 7fa590a00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 0x7fa58c19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa590a00640 1 -- 172.21.15.175:0/4156766937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 msgr2=0x7fa58c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa590a00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 0x7fa58c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa590a00640 1 -- 172.21.15.175:0/4156766937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c19f2d0 con 0x7fa58c103350 2024-07-29T00:36:45.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa591e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 0x7fa58c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa590a00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c19a860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e910 tx=0x7fa57800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa58a000640 1 -- 172.21.15.175:0/4156766937 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57800cd80 con 0x7fa58c103350 2024-07-29T00:36:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c19f5c0 con 0x7fa58c103350 2024-07-29T00:36:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa58a000640 1 -- 172.21.15.175:0/4156766937 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5780035f0 con 0x7fa58c103350 2024-07-29T00:36:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa58a000640 1 -- 172.21.15.175:0/4156766937 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578010670 con 0x7fa58c103350 2024-07-29T00:36:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.143+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c1a6ea0 con 0x7fa58c103350 2024-07-29T00:36:45.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa58a000640 1 -- 172.21.15.175:0/4156766937 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7fa5780037a0 con 0x7fa58c103350 2024-07-29T00:36:45.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 Processor -- start 2024-07-29T00:36:45.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 -- start start 2024-07-29T00:36:45.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde36f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556cfdd8ead0 con 0x556cfde2a250 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556cfdd26af0 con 0x556cfde34b10 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.147+0000 7fa59683a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556cfdd8aa90 con 0x556cfde29930 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa591e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa590a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde36f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa591e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde29d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58852/0 (socket says 172.21.15.175:58852) 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa591e00640 1 -- 172.21.15.175:0/4012491387 learned_addr learned my addr 172.21.15.175:0/4012491387 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde345d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60232/0 (socket says 172.21.15.175:60232) 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa590a00640 1 -- 172.21.15.175:0/4012491387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 msgr2=0x556cfde29d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa590a00640 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde29d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa590a00640 1 -- 172.21.15.175:0/4012491387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 msgr2=0x556cfde345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa590a00640 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa590a00640 1 -- 172.21.15.175:0/4012491387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfdb003c0 con 0x556cfde34b10 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.148+0000 7fa591e00640 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa590a00640 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde36f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5780ccea0 tx=0x7fa5780d5e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deff238f0519d06 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa58a000640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5740a2e00 0x7fa5740a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa57e600640 1 -- 172.21.15.175:0/4012491387 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5780de070 con 0x556cfde34b10 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa57e600640 1 -- 172.21.15.175:0/4012491387 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5780d7d20 con 0x556cfde34b10 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa57e600640 1 -- 172.21.15.175:0/4012491387 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57809b520 con 0x556cfde34b10 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa58a000640 1 -- 172.21.15.175:0/4156766937 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa5780cf890 con 0x7fa58c103350 2024-07-29T00:36:45.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.149+0000 7fa591e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5740a2e00 0x7fa5740a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa591e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5740a2e00 0x7fa5740a52c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa584005f60 tx=0x7fa584005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 -- 172.21.15.175:0/4012491387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 msgr2=0x556cfde36f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde36f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5780ccea0 tx=0x7fa5780d5e40 comp rx=0 tx=0).stop 2024-07-29T00:36:45.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 -- 172.21.15.175:0/4012491387 shutdown_connections 2024-07-29T00:36:45.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde36f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 --2- 172.21.15.175:0/4012491387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 -- 172.21.15.175:0/4012491387 >> 172.21.15.175:0/4012491387 conn(0x556cfde25ec0 msgr2=0x556cfde262a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.150+0000 7fa59683a740 1 -- 172.21.15.175:0/4012491387 shutdown_connections 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.151+0000 7fa59683a740 1 -- 172.21.15.175:0/4012491387 wait complete. 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.151+0000 7fa59683a740 1 -- start start 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.151+0000 7fa59683a740 1 Processor -- start 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde31f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde32440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556cfde376e0 con 0x556cfde29930 2024-07-29T00:36:45.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556cfdd49bc0 con 0x556cfde2a250 2024-07-29T00:36:45.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556cfde37860 con 0x556cfde34b10 2024-07-29T00:36:45.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa59683a740 10 client.0 _ll_register_callbacks cb 0x556cfdd35d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde319c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60248/0 (socket says 172.21.15.175:60248) 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa591400640 1 -- 172.21.15.175:0/2553885499 learned_addr learned my addr 172.21.15.175:0/2553885499 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa590a00640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde31f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.152+0000 7fa591e00640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde32440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa591400640 1 -- 172.21.15.175:0/2553885499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 msgr2=0x556cfde32440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa591400640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde32440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa591400640 1 -- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 msgr2=0x556cfde31f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa591400640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde31f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa591400640 1 -- 172.21.15.175:0/2553885499 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfde40ba0 con 0x556cfde29930 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa590a00640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde31f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa591400640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde319c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa580033000 tx=0x7fa580030d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9530ea7ea824e8a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa572a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa572a00640 1 -- 172.21.15.175:0/2553885499 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580036040 con 0x556cfde29930 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa572a00640 1 -- 172.21.15.175:0/2553885499 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa5800305a0 con 0x556cfde29930 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.153+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5780d7d20 con 0x556cfde29930 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.154+0000 7fa572a00640 1 -- 172.21.15.175:0/2553885499 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa580030ed0 con 0x556cfde29930 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.154+0000 7fa59683a740 10 client.15201 fetch_fsmap learned FSMap version 29 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.154+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556cfde43530 con 0x556cfde29930 2024-07-29T00:36:45.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.154+0000 7fa572a00640 1 -- 172.21.15.175:0/2553885499 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5780d7d20 con 0x556cfde29930 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa572a00640 1 -- 172.21.15.175:0/2553885499 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa580004640 con 0x556cfde29930 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa59683a740 10 client.15201 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa59683a740 10 client.15201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa59683a740 20 client.15201 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa59683a740 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfde449f0 0x556cfde46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa59683a740 4 client.15201 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cfdd9c040 con 0x556cfde449f0 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.155+0000 7fa590a00640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfde449f0 0x556cfde46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.156+0000 7fa590a00640 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfde449f0 0x556cfde46e60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11eccae35a45644d server_cookie=5948dc7d97e556f5 in_seq=0 out_seq=0 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.156+0000 7fa572a00640 10 client.15201 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.156+0000 7fa572a00640 1 -- 172.21.15.175:0/2553885499 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556cfdd9c040 con 0x556cfde449f0 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.156+0000 7fa572a00640 10 client.15201 handle_command_reply: tid=0 2024-07-29T00:36:45.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 client.15201 shutdown 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 msgr2=0x556cfde319c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde319c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa580033000 tx=0x7fa580030d30 comp rx=0 tx=0).stop 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 shutdown_connections 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfde449f0 0x556cfde46e60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde32440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde31f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa572a00640 0 client.15201 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:45.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 --2- 172.21.15.175:0/2553885499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde319c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 >> 172.21.15.175:0/2553885499 conn(0x556cfde25ec0 msgr2=0x556cfde2bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 shutdown_connections 2024-07-29T00:36:45.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 1 -- 172.21.15.175:0/2553885499 wait complete. 2024-07-29T00:36:45.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.157+0000 7fa59683a740 20 client.15201 trim_cache size 0 max 16384 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 -- 172.21.15.175:0/2967694696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 msgr2=0x7f693c107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 --2- 172.21.15.175:0/2967694696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 0x7f693c107bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f692c009890 tx=0x7f692c02a360 comp rx=0 tx=0).stop 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 -- 172.21.15.175:0/2967694696 shutdown_connections 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 --2- 172.21.15.175:0/2967694696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 --2- 172.21.15.175:0/2967694696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c1017c0 0x7f693c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 --2- 172.21.15.175:0/2967694696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 0x7f693c107bb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 -- 172.21.15.175:0/2967694696 >> 172.21.15.175:0/2967694696 conn(0x7f693c0fd4c0 msgr2=0x7f693c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 -- 172.21.15.175:0/2967694696 shutdown_connections 2024-07-29T00:36:45.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.165+0000 7f6943200640 1 -- 172.21.15.175:0/2967694696 wait complete. 2024-07-29T00:36:45.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.166+0000 7f6943200640 1 Processor -- start 2024-07-29T00:36:45.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.166+0000 7f6943200640 1 -- start start 2024-07-29T00:36:45.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.166+0000 7f6943200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c1017c0 0x7f693c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f6943200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f6943200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 0x7f693c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f6943200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f693c111c10 con 0x7f693c1017c0 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f6943200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f693c111a90 con 0x7f693c102160 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f6943200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f693c111d90 con 0x7f693c1077d0 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c1017c0 0x7f693c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f6941200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 0x7f693c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:50438/0 (socket says 172.21.15.175:50438) 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 -- 172.21.15.175:0/3464946298 learned_addr learned my addr 172.21.15.175:0/3464946298 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 -- 172.21.15.175:0/3464946298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 msgr2=0x7f693c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 0x7f693c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 -- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c1017c0 msgr2=0x7f693c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c1017c0 0x7f693c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.167+0000 7f693b400640 1 -- 172.21.15.175:0/3464946298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c19d090 con 0x7f693c102160 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.168+0000 7f693b400640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c1985e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f692800e9c0 tx=0x7f692800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.168+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692800ccc0 con 0x7f693c102160 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.168+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6928005500 con 0x7f693c102160 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.168+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c19d380 con 0x7f693c102160 2024-07-29T00:36:45.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.168+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928010430 con 0x7f693c102160 2024-07-29T00:36:45.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.168+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a4c10 con 0x7f693c102160 2024-07-29T00:36:45.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.169+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c101c20 con 0x7f693c102160 2024-07-29T00:36:45.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.171+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f69280046c0 con 0x7f693c102160 2024-07-29T00:36:45.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.172+0000 7f6938c00640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f69200a2e40 0x7f69200a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.173+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f69280cfb60 con 0x7f693c102160 2024-07-29T00:36:45.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.173+0000 7f693be00640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f69200a2e40 0x7f69200a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.174+0000 7f693be00640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f69200a2e40 0x7f69200a5300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f692c02a950 tx=0x7f692c0056d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.174+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6928053df0 con 0x7f693c102160 2024-07-29T00:36:45.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 Processor -- start 2024-07-29T00:36:45.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 -- start start 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde32290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1af8f0 con 0x556cfde2a250 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b1430 con 0x556cfde34b10 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa59683a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1affc0 con 0x556cfde29930 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa591400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde32290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.178+0000 7fa590a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde32290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58874/0 (socket says 172.21.15.175:58874) 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 -- 172.21.15.175:0/1594815211 learned_addr learned my addr 172.21.15.175:0/1594815211 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:45.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591e00640 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 -- 172.21.15.175:0/1594815211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 msgr2=0x556cfde4cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde4cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 -- 172.21.15.175:0/1594815211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 msgr2=0x556cfde4c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde4c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 -- 172.21.15.175:0/1594815211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfde2d600 con 0x556cfde29930 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591e00640 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.179+0000 7fa591400640 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde32290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa580002230 tx=0x7fa580005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d65911ae89ced44c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa57dc00640 1 -- 172.21.15.175:0/1594815211 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580053030 con 0x556cfde29930 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa57dc00640 1 -- 172.21.15.175:0/1594815211 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5800476e0 con 0x556cfde29930 2024-07-29T00:36:45.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa57dc00640 1 -- 172.21.15.175:0/1594815211 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580036040 con 0x556cfde29930 2024-07-29T00:36:45.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 -- 172.21.15.175:0/1594815211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 msgr2=0x556cfde32290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde32290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa580002230 tx=0x7fa580005e50 comp rx=0 tx=0).stop 2024-07-29T00:36:45.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 -- 172.21.15.175:0/1594815211 shutdown_connections 2024-07-29T00:36:45.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde34b10 0x556cfde4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde2a250 0x556cfde4c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 --2- 172.21.15.175:0/1594815211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde29930 0x556cfde32290 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.180+0000 7fa59683a740 1 -- 172.21.15.175:0/1594815211 >> 172.21.15.175:0/1594815211 conn(0x556cfde25ec0 msgr2=0x556cfde30af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.181+0000 7fa59683a740 1 -- 172.21.15.175:0/1594815211 shutdown_connections 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.181+0000 7fa59683a740 1 -- 172.21.15.175:0/1594815211 wait complete. 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.181+0000 7fa59683a740 1 -- start start 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.181+0000 7fa59683a740 1 Processor -- start 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde3cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde3d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde2e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b0d60 con 0x556cfde29930 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b0690 con 0x556cfde2a250 2024-07-29T00:36:45.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1ad8d0 con 0x556cfde34b10 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa59683a740 10 client.0 _ll_register_callbacks cb 0x7fa580030a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde3cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa591400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde3cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60294/0 (socket says 172.21.15.175:60294) 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa591400640 1 -- 172.21.15.175:0/3802593082 learned_addr learned my addr 172.21.15.175:0/3802593082 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa591e00640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde2e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa590a00640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde3d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa591400640 1 -- 172.21.15.175:0/3802593082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 msgr2=0x556cfde2e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.182+0000 7fa591400640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde2e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa591400640 1 -- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 msgr2=0x556cfde3d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa591400640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde3d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa591400640 1 -- 172.21.15.175:0/3802593082 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfde2e8b0 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa590a00640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde3d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa591400640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde3cc90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa5800475e0 tx=0x7fa580005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cca5a68dec6155 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa572a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa572a00640 1 -- 172.21.15.175:0/3802593082 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58005d060 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa572a00640 1 -- 172.21.15.175:0/3802593082 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa5800305a0 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.183+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5800476e0 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa572a00640 1 -- 172.21.15.175:0/3802593082 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa580053040 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 10 client.15213 fetch_fsmap learned FSMap version 29 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556cfdeaf020 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa572a00640 1 -- 172.21.15.175:0/3802593082 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5800476e0 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa572a00640 1 -- 172.21.15.175:0/3802593082 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa580031440 con 0x556cfde29930 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 10 client.15213 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 10 client.15213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:36:45.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 20 client.15213 populate_metadata read hostname 'smithi175' 2024-07-29T00:36:45.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfdeaf5d0 0x556cfdeb19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:45.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 4 client.15213 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:36:45.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.184+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556cfdd96550 con 0x556cfdeaf5d0 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.185+0000 7fa590a00640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfdeaf5d0 0x556cfdeb19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.185+0000 7fa590a00640 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfdeaf5d0 0x556cfdeb19f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b096c3ba31d6153 server_cookie=9583d62b2eb68ffc in_seq=0 out_seq=0 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.185+0000 7fa572a00640 10 client.15213 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.185+0000 7fa572a00640 1 -- 172.21.15.175:0/3802593082 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556cfdd96550 con 0x556cfdeaf5d0 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.185+0000 7fa572a00640 10 client.15213 handle_command_reply: tid=0 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.185+0000 7fa59683a740 1 client.15213 shutdown 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 msgr2=0x556cfde3cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde3cc90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa5800475e0 tx=0x7fa580005ec0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 shutdown_connections 2024-07-29T00:36:45.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556cfdeaf5d0 0x556cfdeb19f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa572a00640 0 client.15213 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cfde34b10 0x556cfde2e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556cfde2a250 0x556cfde3d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 --2- 172.21.15.175:0/3802593082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cfde29930 0x556cfde3cc90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 >> 172.21.15.175:0/3802593082 conn(0x556cfde25ec0 msgr2=0x556cfde2b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 shutdown_connections 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 1 -- 172.21.15.175:0/3802593082 wait complete. 2024-07-29T00:36:45.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.186+0000 7fa59683a740 20 client.15213 trim_cache size 0 max 16384 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: "2a9f3795-42e1-4c45-885d-35ce50b77968": { 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "2a9f3795-42e1-4c45-885d-35ce50b77968", 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:36:45.191 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5740a2e00 msgr2=0x7fa5740a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5740a2e00 0x7fa5740a52c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa584005f60 tx=0x7fa584005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 msgr2=0x7fa58c19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c19a860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e910 tx=0x7fa57800edd0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 shutdown_connections 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa58c10be60 0x7fa58c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5740a2e00 0x7fa5740a52c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa58c103350 0x7fa58c19a860 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 --2- 172.21.15.175:0/4156766937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa58c102a30 0x7fa58c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 >> 172.21.15.175:0/4156766937 conn(0x7fa58c0fc490 msgr2=0x7fa58c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 shutdown_connections 2024-07-29T00:36:45.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.187+0000 7fa593e00640 1 -- 172.21.15.175:0/4156766937 wait complete. 2024-07-29T00:36:45.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.423+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f693c0fe320 con 0x7f69200a2e40 2024-07-29T00:36:45.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.435+0000 7f6938c00640 1 -- 172.21.15.175:0/3464946298 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f693c0fe320 con 0x7f69200a2e40 2024-07-29T00:36:45.436 INFO:teuthology.orchestra.run.smithi175.stdout:/ 1m 6m3h 2024-07-29T00:36:45.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f69200a2e40 msgr2=0x7f69200a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f69200a2e40 0x7f69200a5300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f692c02a950 tx=0x7f692c0056d0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 msgr2=0x7f693c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:45.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c1985e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f692800e9c0 tx=0x7f692800ee80 comp rx=0 tx=0).stop 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 shutdown_connections 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f693c1077d0 0x7f693c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c102160 0x7f693c1985e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f69200a2e40 0x7f69200a5300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 --2- 172.21.15.175:0/3464946298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f693c1017c0 0x7f693c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 >> 172.21.15.175:0/3464946298 conn(0x7f693c0fd4c0 msgr2=0x7f693c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 shutdown_connections 2024-07-29T00:36:45.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:45.438+0000 7f6943200640 1 -- 172.21.15.175:0/3464946298 wait complete. 2024-07-29T00:36:45.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-07-29T00:36:45.535 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-07-29T00:36:45.546 INFO:teuthology.run_tasks:Running task quiescer... 2024-07-29T00:36:45.562 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-29T00:36:45.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:36:45.710 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:45 smithi175 ceph-mon[32138]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-07-29T00:36:45.710 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:45 smithi175 ceph-mon[32138]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:45.749 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:45 smithi204 ceph-mon[37368]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-07-29T00:36:45.749 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:45 smithi204 ceph-mon[37368]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:45.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:45 smithi181 ceph-mon[36706]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-07-29T00:36:45.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:45 smithi181 ceph-mon[36706]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:46.277 INFO:tasks.cephfs.filesystem:scrub status for tag:2a9f3795-42e1-4c45-885d-35ce50b77968 - {'path': '/', 'tag': '2a9f3795-42e1-4c45-885d-35ce50b77968', 'options': 'recursive,force'} 2024-07-29T00:36:46.481 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:46.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:46 smithi175 ceph-mon[32138]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:46.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:46 smithi175 ceph-mon[32138]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:46.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:46 smithi181 ceph-mon[36706]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:46.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:46 smithi181 ceph-mon[36706]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:46.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:46 smithi204 ceph-mon[37368]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:46.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:46 smithi204 ceph-mon[37368]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 -- 172.21.15.175:0/2142616354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc798104260 msgr2=0x7fc798110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 --2- 172.21.15.175:0/2142616354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc798104260 0x7fc798110790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc790009cd0 tx=0x7fc79002a8f0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 -- 172.21.15.175:0/2142616354 shutdown_connections 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 --2- 172.21.15.175:0/2142616354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc798104260 0x7fc798110790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 --2- 172.21.15.175:0/2142616354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7981038c0 0x7fc798103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 --2- 172.21.15.175:0/2142616354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7981098c0 0x7fc798109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 -- 172.21.15.175:0/2142616354 >> 172.21.15.175:0/2142616354 conn(0x7fc7980ff5c0 msgr2=0x7fc7981019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:47.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 -- 172.21.15.175:0/2142616354 shutdown_connections 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.195+0000 7fc79e600640 1 -- 172.21.15.175:0/2142616354 wait complete. 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.196+0000 7fc79e600640 1 Processor -- start 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.196+0000 7fc79e600640 1 -- start start 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 0x7fc79819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc798104260 0x7fc79819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7981098c0 0x7fc79819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc798113d10 con 0x7fc7981038c0 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc798113b90 con 0x7fc798104260 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc798113e90 con 0x7fc7981098c0 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 0x7fc79819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 0x7fc79819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60306/0 (socket says 172.21.15.175:60306) 2024-07-29T00:36:47.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79d200640 1 -- 172.21.15.175:0/462310539 learned_addr learned my addr 172.21.15.175:0/462310539 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc79dc00640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7981098c0 0x7fc79819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.197+0000 7fc797e00640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc798104260 0x7fc79819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79d200640 1 -- 172.21.15.175:0/462310539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7981098c0 msgr2=0x7fc79819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79d200640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7981098c0 0x7fc79819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79d200640 1 -- 172.21.15.175:0/462310539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc798104260 msgr2=0x7fc79819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79d200640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc798104260 0x7fc79819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79d200640 1 -- 172.21.15.175:0/462310539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79819f100 con 0x7fc7981038c0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc797e00640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc798104260 0x7fc79819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79d200640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 0x7fc79819a150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00d950 tx=0x7fc78c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c017070 con 0x7fc7981038c0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78c009c30 con 0x7fc7981038c0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79819f3f0 con 0x7fc7981038c0 2024-07-29T00:36:47.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00f6b0 con 0x7fc7981038c0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.198+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7981a6cd0 con 0x7fc7981038c0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.200+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc798103d20 con 0x7fc7981038c0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.201+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7fc78c0109d0 con 0x7fc7981038c0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.203+0000 7fc795600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc7800a2d80 0x7fc7800a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.203+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fc78c0cf1a0 con 0x7fc7981038c0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.203+0000 7fc797e00640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc7800a2d80 0x7fc7800a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.204+0000 7fc797e00640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc7800a2d80 0x7fc7800a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc798101ac0 tx=0x7fc784005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:47.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.207+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc78c009de0 con 0x7fc7981038c0 2024-07-29T00:36:47.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.391+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc798100350 con 0x7fc7800a2d80 2024-07-29T00:36:47.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.408+0000 7fc795600640 1 -- 172.21.15.175:0/462310539 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258121 (secure 0 0 0) 0x7fc798100350 con 0x7fc7800a2d80 2024-07-29T00:36:47.410 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:47.414 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc7800a2d80 msgr2=0x7fc7800a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc7800a2d80 0x7fc7800a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc798101ac0 tx=0x7fc784005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 msgr2=0x7fc79819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 0x7fc79819a150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00d950 tx=0x7fc78c00de10 comp rx=0 tx=0).stop 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 shutdown_connections 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7981098c0 0x7fc79819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc798104260 0x7fc79819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc7800a2d80 0x7fc7800a5240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 --2- 172.21.15.175:0/462310539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7981038c0 0x7fc79819a150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 >> 172.21.15.175:0/462310539 conn(0x7fc7980ff5c0 msgr2=0x7fc7980ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:47.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 shutdown_connections 2024-07-29T00:36:47.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:47.415+0000 7fc79e600640 1 -- 172.21.15.175:0/462310539 wait complete. 2024-07-29T00:36:47.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:47 smithi175 ceph-mon[32138]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-07-29T00:36:47.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:47 smithi181 ceph-mon[36706]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-07-29T00:36:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:47 smithi204 ceph-mon[37368]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-07-29T00:36:48.417 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-07-29T00:36:45.944523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388120,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1847,"num_read_kb":1276,"num_write":609,"num_write_kb":4873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":981,"ondisk_log_size":981,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361764,"kb_used_data":12568,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580060,"statfs":{"total":1151940427776,"available":1151569981440,"internally_reserved":0,"allocated":12869632,"data_stored":9770460,"data_compressed":375611,"data_compressed_allocated":503808,"data_compressed_original":4050944,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":240,"num_read_kb":290,"num_write":47,"num_write_kb":466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005150"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270840+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:36:30.270840+0000","last_peered":"2024-07-29T00:36:30.270840+0000","last_clean":"2024-07-29T00:36:30.270840+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:36:30.270840+0000","last_undegraded":"2024-07-29T00:36:30.270840+0000","last_fullsized":"2024-07-29T00:36:30.270840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122176+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:36:30.122176+0000","last_peered":"2024-07-29T00:36:30.122176+0000","last_clean":"2024-07-29T00:36:30.122176+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:36:30.122176+0000","last_undegraded":"2024-07-29T00:36:30.122176+0000","last_fullsized":"2024-07-29T00:36:30.122176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121890+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:36:30.121890+0000","last_peered":"2024-07-29T00:36:30.121890+0000","last_clean":"2024-07-29T00:36:30.121890+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:36:30.121890+0000","last_undegraded":"2024-07-29T00:36:30.121890+0000","last_fullsized":"2024-07-29T00:36:30.121890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919430+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:36:29.919430+0000","last_peered":"2024-07-29T00:36:29.919430+0000","last_clean":"2024-07-29T00:36:29.919430+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:36:29.919430+0000","last_undegraded":"2024-07-29T00:36:29.919430+0000","last_fullsized":"2024-07-29T00:36:29.919430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030174+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:36:30.030174+0000","last_peered":"2024-07-29T00:36:30.030174+0000","last_clean":"2024-07-29T00:36:30.030174+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:36:30.030174+0000","last_undegraded":"2024-07-29T00:36:30.030174+0000","last_fullsized":"2024-07-29T00:36:30.030174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039832+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:36:30.039832+0000","last_peered":"2024-07-29T00:36:30.039832+0000","last_clean":"2024-07-29T00:36:30.039832+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:36:30.039832+0000","last_undegraded":"2024-07-29T00:36:30.039832+0000","last_fullsized":"2024-07-29T00:36:30.039832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978714+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:36:29.978714+0000","last_peered":"2024-07-29T00:36:29.978714+0000","last_clean":"2024-07-29T00:36:29.978714+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:36:29.978714+0000","last_undegraded":"2024-07-29T00:36:29.978714+0000","last_fullsized":"2024-07-29T00:36:29.978714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006206+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:36:30.006206+0000","last_peered":"2024-07-29T00:36:30.006206+0000","last_clean":"2024-07-29T00:36:30.006206+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:36:30.006206+0000","last_undegraded":"2024-07-29T00:36:30.006206+0000","last_fullsized":"2024-07-29T00:36:30.006206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918987+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:36:29.918987+0000","last_peered":"2024-07-29T00:36:29.918987+0000","last_clean":"2024-07-29T00:36:29.918987+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:36:29.918987+0000","last_undegraded":"2024-07-29T00:36:29.918987+0000","last_fullsized":"2024-07-29T00:36:29.918987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.040027+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:36:30.040027+0000","last_peered":"2024-07-29T00:36:30.040027+0000","last_clean":"2024-07-29T00:36:30.040027+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:36:30.040027+0000","last_undegraded":"2024-07-29T00:36:30.040027+0000","last_fullsized":"2024-07-29T00:36:30.040027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978875+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:36:29.978875+0000","last_peered":"2024-07-29T00:36:29.978875+0000","last_clean":"2024-07-29T00:36:29.978875+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:36:29.978875+0000","last_undegraded":"2024-07-29T00:36:29.978875+0000","last_fullsized":"2024-07-29T00:36:29.978875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919501+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:36:29.919501+0000","last_peered":"2024-07-29T00:36:29.919501+0000","last_clean":"2024-07-29T00:36:29.919501+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:36:29.919501+0000","last_undegraded":"2024-07-29T00:36:29.919501+0000","last_fullsized":"2024-07-29T00:36:29.919501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003149+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:36:30.003149+0000","last_peered":"2024-07-29T00:36:30.003149+0000","last_clean":"2024-07-29T00:36:30.003149+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:36:30.003149+0000","last_undegraded":"2024-07-29T00:36:30.003149+0000","last_fullsized":"2024-07-29T00:36:30.003149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122517+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:36:30.122517+0000","last_peered":"2024-07-29T00:36:30.122517+0000","last_clean":"2024-07-29T00:36:30.122517+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:36:30.122517+0000","last_undegraded":"2024-07-29T00:36:30.122517+0000","last_fullsized":"2024-07-29T00:36:30.122517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121450+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:36:30.121450+0000","last_peered":"2024-07-29T00:36:30.121450+0000","last_clean":"2024-07-29T00:36:30.121450+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:36:30.121450+0000","last_undegraded":"2024-07-29T00:36:30.121450+0000","last_fullsized":"2024-07-29T00:36:30.121450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918029+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:36:29.918029+0000","last_peered":"2024-07-29T00:36:29.918029+0000","last_clean":"2024-07-29T00:36:29.918029+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:36:29.918029+0000","last_undegraded":"2024-07-29T00:36:29.918029+0000","last_fullsized":"2024-07-29T00:36:29.918029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270846+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:36:30.270846+0000","last_peered":"2024-07-29T00:36:30.270846+0000","last_clean":"2024-07-29T00:36:30.270846+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:36:30.270846+0000","last_undegraded":"2024-07-29T00:36:30.270846+0000","last_fullsized":"2024-07-29T00:36:30.270846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921670+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:36:29.921670+0000","last_peered":"2024-07-29T00:36:29.921670+0000","last_clean":"2024-07-29T00:36:29.921670+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:36:29.921670+0000","last_undegraded":"2024-07-29T00:36:29.921670+0000","last_fullsized":"2024-07-29T00:36:29.921670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119558+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:36:30.119558+0000","last_peered":"2024-07-29T00:36:30.119558+0000","last_clean":"2024-07-29T00:36:30.119558+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:36:30.119558+0000","last_undegraded":"2024-07-29T00:36:30.119558+0000","last_fullsized":"2024-07-29T00:36:30.119558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270504+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:36:30.270504+0000","last_peered":"2024-07-29T00:36:30.270504+0000","last_clean":"2024-07-29T00:36:30.270504+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:36:30.270504+0000","last_undegraded":"2024-07-29T00:36:30.270504+0000","last_fullsized":"2024-07-29T00:36:30.270504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003134+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:36:30.003134+0000","last_peered":"2024-07-29T00:36:30.003134+0000","last_clean":"2024-07-29T00:36:30.003134+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:36:30.003134+0000","last_undegraded":"2024-07-29T00:36:30.003134+0000","last_fullsized":"2024-07-29T00:36:30.003134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918840+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:36:29.918840+0000","last_peered":"2024-07-29T00:36:29.918840+0000","last_clean":"2024-07-29T00:36:29.918840+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:36:29.918840+0000","last_undegraded":"2024-07-29T00:36:29.918840+0000","last_fullsized":"2024-07-29T00:36:29.918840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006207+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:36:30.006207+0000","last_peered":"2024-07-29T00:36:30.006207+0000","last_clean":"2024-07-29T00:36:30.006207+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:36:30.006207+0000","last_undegraded":"2024-07-29T00:36:30.006207+0000","last_fullsized":"2024-07-29T00:36:30.006207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978362+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:36:29.978362+0000","last_peered":"2024-07-29T00:36:29.978362+0000","last_clean":"2024-07-29T00:36:29.978362+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:36:29.978362+0000","last_undegraded":"2024-07-29T00:36:29.978362+0000","last_fullsized":"2024-07-29T00:36:29.978362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006816+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:36:30.006816+0000","last_peered":"2024-07-29T00:36:30.006816+0000","last_clean":"2024-07-29T00:36:30.006816+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:36:30.006816+0000","last_undegraded":"2024-07-29T00:36:30.006816+0000","last_fullsized":"2024-07-29T00:36:30.006816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119656+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:36:30.119656+0000","last_peered":"2024-07-29T00:36:30.119656+0000","last_clean":"2024-07-29T00:36:30.119656+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:36:30.119656+0000","last_undegraded":"2024-07-29T00:36:30.119656+0000","last_fullsized":"2024-07-29T00:36:30.119656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270796+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:36:30.270796+0000","last_peered":"2024-07-29T00:36:30.270796+0000","last_clean":"2024-07-29T00:36:30.270796+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:36:30.270796+0000","last_undegraded":"2024-07-29T00:36:30.270796+0000","last_fullsized":"2024-07-29T00:36:30.270796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003369+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:36:30.003369+0000","last_peered":"2024-07-29T00:36:30.003369+0000","last_clean":"2024-07-29T00:36:30.003369+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:36:30.003369+0000","last_undegraded":"2024-07-29T00:36:30.003369+0000","last_fullsized":"2024-07-29T00:36:30.003369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121371+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:36:30.121371+0000","last_peered":"2024-07-29T00:36:30.121371+0000","last_clean":"2024-07-29T00:36:30.121371+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:36:30.121371+0000","last_undegraded":"2024-07-29T00:36:30.121371+0000","last_fullsized":"2024-07-29T00:36:30.121371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119269+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:36:30.119269+0000","last_peered":"2024-07-29T00:36:30.119269+0000","last_clean":"2024-07-29T00:36:30.119269+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:36:30.119269+0000","last_undegraded":"2024-07-29T00:36:30.119269+0000","last_fullsized":"2024-07-29T00:36:30.119269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119276+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:36:30.119276+0000","last_peered":"2024-07-29T00:36:30.119276+0000","last_clean":"2024-07-29T00:36:30.119276+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:36:30.119276+0000","last_undegraded":"2024-07-29T00:36:30.119276+0000","last_fullsized":"2024-07-29T00:36:30.119276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270536+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:36:30.270536+0000","last_peered":"2024-07-29T00:36:30.270536+0000","last_clean":"2024-07-29T00:36:30.270536+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:36:30.270536+0000","last_undegraded":"2024-07-29T00:36:30.270536+0000","last_fullsized":"2024-07-29T00:36:30.270536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029964+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:36:30.029964+0000","last_peered":"2024-07-29T00:36:30.029964+0000","last_clean":"2024-07-29T00:36:30.029964+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:36:30.029964+0000","last_undegraded":"2024-07-29T00:36:30.029964+0000","last_fullsized":"2024-07-29T00:36:30.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":196,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.906520+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:36:44.906520+0000","last_peered":"2024-07-29T00:36:44.906520+0000","last_clean":"2024-07-29T00:36:44.906520+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:36:44.906520+0000","last_undegraded":"2024-07-29T00:36:44.906520+0000","last_fullsized":"2024-07-29T00:36:44.906520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978592+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:36:29.978592+0000","last_peered":"2024-07-29T00:36:29.978592+0000","last_clean":"2024-07-29T00:36:29.978592+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:36:29.978592+0000","last_undegraded":"2024-07-29T00:36:29.978592+0000","last_fullsized":"2024-07-29T00:36:29.978592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039762+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:36:30.039762+0000","last_peered":"2024-07-29T00:36:30.039762+0000","last_clean":"2024-07-29T00:36:30.039762+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:36:30.039762+0000","last_undegraded":"2024-07-29T00:36:30.039762+0000","last_fullsized":"2024-07-29T00:36:30.039762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002737+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:36:30.002737+0000","last_peered":"2024-07-29T00:36:30.002737+0000","last_clean":"2024-07-29T00:36:30.002737+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:36:30.002737+0000","last_undegraded":"2024-07-29T00:36:30.002737+0000","last_fullsized":"2024-07-29T00:36:30.002737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006358+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:36:30.006358+0000","last_peered":"2024-07-29T00:36:30.006358+0000","last_clean":"2024-07-29T00:36:30.006358+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:36:30.006358+0000","last_undegraded":"2024-07-29T00:36:30.006358+0000","last_fullsized":"2024-07-29T00:36:30.006358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978697+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:36:29.978697+0000","last_peered":"2024-07-29T00:36:29.978697+0000","last_clean":"2024-07-29T00:36:29.978697+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:36:29.978697+0000","last_undegraded":"2024-07-29T00:36:29.978697+0000","last_fullsized":"2024-07-29T00:36:29.978697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.920897+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:36:29.920897+0000","last_peered":"2024-07-29T00:36:29.920897+0000","last_clean":"2024-07-29T00:36:29.920897+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:36:29.920897+0000","last_undegraded":"2024-07-29T00:36:29.920897+0000","last_fullsized":"2024-07-29T00:36:29.920897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":334,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:40.858264+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:36:40.858264+0000","last_peered":"2024-07-29T00:36:40.858264+0000","last_clean":"2024-07-29T00:36:40.858264+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:36:40.858264+0000","last_undegraded":"2024-07-29T00:36:40.858264+0000","last_fullsized":"2024-07-29T00:36:40.858264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":295,"num_read_kb":295,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918684+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:36:29.918684+0000","last_peered":"2024-07-29T00:36:29.918684+0000","last_clean":"2024-07-29T00:36:29.918684+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:36:29.918684+0000","last_undegraded":"2024-07-29T00:36:29.918684+0000","last_fullsized":"2024-07-29T00:36:29.918684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979141+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:36:29.979141+0000","last_peered":"2024-07-29T00:36:29.979141+0000","last_clean":"2024-07-29T00:36:29.979141+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:36:29.979141+0000","last_undegraded":"2024-07-29T00:36:29.979141+0000","last_fullsized":"2024-07-29T00:36:29.979141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122906+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:36:30.122906+0000","last_peered":"2024-07-29T00:36:30.122906+0000","last_clean":"2024-07-29T00:36:30.122906+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:36:30.122906+0000","last_undegraded":"2024-07-29T00:36:30.122906+0000","last_fullsized":"2024-07-29T00:36:30.122906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978649+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:36:29.978649+0000","last_peered":"2024-07-29T00:36:29.978649+0000","last_clean":"2024-07-29T00:36:29.978649+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:36:29.978649+0000","last_undegraded":"2024-07-29T00:36:29.978649+0000","last_fullsized":"2024-07-29T00:36:29.978649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918482+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:36:29.918482+0000","last_peered":"2024-07-29T00:36:29.918482+0000","last_clean":"2024-07-29T00:36:29.918482+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:36:29.918482+0000","last_undegraded":"2024-07-29T00:36:29.918482+0000","last_fullsized":"2024-07-29T00:36:29.918482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":397,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:40.862791+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:36:40.862791+0000","last_peered":"2024-07-29T00:36:40.862791+0000","last_clean":"2024-07-29T00:36:40.862791+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:36:40.862791+0000","last_undegraded":"2024-07-29T00:36:40.862791+0000","last_fullsized":"2024-07-29T00:36:40.862791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66470,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":306,"num_read_kb":218,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030244+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:36:30.030244+0000","last_peered":"2024-07-29T00:36:30.030244+0000","last_clean":"2024-07-29T00:36:30.030244+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:36:30.030244+0000","last_undegraded":"2024-07-29T00:36:30.030244+0000","last_fullsized":"2024-07-29T00:36:30.030244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119352+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:36:30.119352+0000","last_peered":"2024-07-29T00:36:30.119352+0000","last_clean":"2024-07-29T00:36:30.119352+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:36:30.119352+0000","last_undegraded":"2024-07-29T00:36:30.119352+0000","last_fullsized":"2024-07-29T00:36:30.119352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002636+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:36:30.002636+0000","last_peered":"2024-07-29T00:36:30.002636+0000","last_clean":"2024-07-29T00:36:30.002636+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:36:30.002636+0000","last_undegraded":"2024-07-29T00:36:30.002636+0000","last_fullsized":"2024-07-29T00:36:30.002636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978452+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:36:29.978452+0000","last_peered":"2024-07-29T00:36:29.978452+0000","last_clean":"2024-07-29T00:36:29.978452+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:36:29.978452+0000","last_undegraded":"2024-07-29T00:36:29.978452+0000","last_fullsized":"2024-07-29T00:36:29.978452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978851+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:36:29.978851+0000","last_peered":"2024-07-29T00:36:29.978851+0000","last_clean":"2024-07-29T00:36:29.978851+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:36:29.978851+0000","last_undegraded":"2024-07-29T00:36:29.978851+0000","last_fullsized":"2024-07-29T00:36:29.978851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":194,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:39.023459+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:36:39.023459+0000","last_peered":"2024-07-29T00:36:39.023459+0000","last_clean":"2024-07-29T00:36:39.023459+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:36:39.023459+0000","last_undegraded":"2024-07-29T00:36:39.023459+0000","last_fullsized":"2024-07-29T00:36:39.023459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978642+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:36:29.978642+0000","last_peered":"2024-07-29T00:36:29.978642+0000","last_clean":"2024-07-29T00:36:29.978642+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:36:29.978642+0000","last_undegraded":"2024-07-29T00:36:29.978642+0000","last_fullsized":"2024-07-29T00:36:29.978642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029821+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:36:30.029821+0000","last_peered":"2024-07-29T00:36:30.029821+0000","last_clean":"2024-07-29T00:36:30.029821+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:36:30.029821+0000","last_undegraded":"2024-07-29T00:36:30.029821+0000","last_fullsized":"2024-07-29T00:36:30.029821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978327+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:36:29.978327+0000","last_peered":"2024-07-29T00:36:29.978327+0000","last_clean":"2024-07-29T00:36:29.978327+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:36:29.978327+0000","last_undegraded":"2024-07-29T00:36:29.978327+0000","last_fullsized":"2024-07-29T00:36:29.978327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030159+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:36:30.030159+0000","last_peered":"2024-07-29T00:36:30.030159+0000","last_clean":"2024-07-29T00:36:30.030159+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:36:30.030159+0000","last_undegraded":"2024-07-29T00:36:30.030159+0000","last_fullsized":"2024-07-29T00:36:30.030159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270695+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:36:30.270695+0000","last_peered":"2024-07-29T00:36:30.270695+0000","last_clean":"2024-07-29T00:36:30.270695+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:36:30.270695+0000","last_undegraded":"2024-07-29T00:36:30.270695+0000","last_fullsized":"2024-07-29T00:36:30.270695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003638+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:36:30.003638+0000","last_peered":"2024-07-29T00:36:30.003638+0000","last_clean":"2024-07-29T00:36:30.003638+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:36:30.003638+0000","last_undegraded":"2024-07-29T00:36:30.003638+0000","last_fullsized":"2024-07-29T00:36:30.003638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979263+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:36:29.979263+0000","last_peered":"2024-07-29T00:36:29.979263+0000","last_clean":"2024-07-29T00:36:29.979263+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:36:29.979263+0000","last_undegraded":"2024-07-29T00:36:29.979263+0000","last_fullsized":"2024-07-29T00:36:29.979263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'67","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.952930+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:36:44.952930+0000","last_peered":"2024-07-29T00:36:44.952930+0000","last_clean":"2024-07-29T00:36:44.952930+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:36:44.952930+0000","last_undegraded":"2024-07-29T00:36:44.952930+0000","last_fullsized":"2024-07-29T00:36:44.952930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978805+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:36:29.978805+0000","last_peered":"2024-07-29T00:36:29.978805+0000","last_clean":"2024-07-29T00:36:29.978805+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:36:29.978805+0000","last_undegraded":"2024-07-29T00:36:29.978805+0000","last_fullsized":"2024-07-29T00:36:29.978805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":139,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:39.857152+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:36:39.857152+0000","last_peered":"2024-07-29T00:36:39.857152+0000","last_clean":"2024-07-29T00:36:39.857152+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:36:39.857152+0000","last_undegraded":"2024-07-29T00:36:39.857152+0000","last_fullsized":"2024-07-29T00:36:39.857152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979102+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:36:29.979102+0000","last_peered":"2024-07-29T00:36:29.979102+0000","last_clean":"2024-07-29T00:36:29.979102+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:36:29.979102+0000","last_undegraded":"2024-07-29T00:36:29.979102+0000","last_fullsized":"2024-07-29T00:36:29.979102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":408,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:32.081815+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:36:32.081815+0000","last_peered":"2024-07-29T00:36:32.081815+0000","last_clean":"2024-07-29T00:36:32.081815+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:36:32.081815+0000","last_undegraded":"2024-07-29T00:36:32.081815+0000","last_fullsized":"2024-07-29T00:36:32.081815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918549+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:36:29.918549+0000","last_peered":"2024-07-29T00:36:29.918549+0000","last_clean":"2024-07-29T00:36:29.918549+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:36:29.918549+0000","last_undegraded":"2024-07-29T00:36:29.918549+0000","last_fullsized":"2024-07-29T00:36:29.918549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006458+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:36:30.006458+0000","last_peered":"2024-07-29T00:36:30.006458+0000","last_clean":"2024-07-29T00:36:30.006458+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:36:30.006458+0000","last_undegraded":"2024-07-29T00:36:30.006458+0000","last_fullsized":"2024-07-29T00:36:30.006458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270459+0000","last_change":"2024-07-29T00:35:51.676982+0000","last_active":"2024-07-29T00:36:30.270459+0000","last_peered":"2024-07-29T00:36:30.270459+0000","last_clean":"2024-07-29T00:36:30.270459+0000","last_became_active":"2024-07-29T00:35:51.674179+0000","last_became_peered":"2024-07-29T00:35:51.674179+0000","last_unstale":"2024-07-29T00:36:30.270459+0000","last_undegraded":"2024-07-29T00:36:30.270459+0000","last_fullsized":"2024-07-29T00:36:30.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:24:23.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002590+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:36:30.002590+0000","last_peered":"2024-07-29T00:36:30.002590+0000","last_clean":"2024-07-29T00:36:30.002590+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:36:30.002590+0000","last_undegraded":"2024-07-29T00:36:30.002590+0000","last_fullsized":"2024-07-29T00:36:30.002590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":193,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.031746+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:36:44.031746+0000","last_peered":"2024-07-29T00:36:44.031746+0000","last_clean":"2024-07-29T00:36:44.031746+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:36:44.031746+0000","last_undegraded":"2024-07-29T00:36:44.031746+0000","last_fullsized":"2024-07-29T00:36:44.031746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978450+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:36:29.978450+0000","last_peered":"2024-07-29T00:36:29.978450+0000","last_clean":"2024-07-29T00:36:29.978450+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:36:29.978450+0000","last_undegraded":"2024-07-29T00:36:29.978450+0000","last_fullsized":"2024-07-29T00:36:29.978450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":193,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.004283+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:36:44.004283+0000","last_peered":"2024-07-29T00:36:44.004283+0000","last_clean":"2024-07-29T00:36:44.004283+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:36:44.004283+0000","last_undegraded":"2024-07-29T00:36:44.004283+0000","last_fullsized":"2024-07-29T00:36:44.004283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":895,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039767+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:36:30.039767+0000","last_peered":"2024-07-29T00:36:30.039767+0000","last_clean":"2024-07-29T00:36:30.039767+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:36:30.039767+0000","last_undegraded":"2024-07-29T00:36:30.039767+0000","last_fullsized":"2024-07-29T00:36:30.039767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921376+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:36:29.921376+0000","last_peered":"2024-07-29T00:36:29.921376+0000","last_clean":"2024-07-29T00:36:29.921376+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:36:29.921376+0000","last_undegraded":"2024-07-29T00:36:29.921376+0000","last_fullsized":"2024-07-29T00:36:29.921376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039877+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:36:30.039877+0000","last_peered":"2024-07-29T00:36:30.039877+0000","last_clean":"2024-07-29T00:36:30.039877+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:36:30.039877+0000","last_undegraded":"2024-07-29T00:36:30.039877+0000","last_fullsized":"2024-07-29T00:36:30.039877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030289+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:36:30.030289+0000","last_peered":"2024-07-29T00:36:30.030289+0000","last_clean":"2024-07-29T00:36:30.030289+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:36:30.030289+0000","last_undegraded":"2024-07-29T00:36:30.030289+0000","last_fullsized":"2024-07-29T00:36:30.030289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003485+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:36:30.003485+0000","last_peered":"2024-07-29T00:36:30.003485+0000","last_clean":"2024-07-29T00:36:30.003485+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:36:30.003485+0000","last_undegraded":"2024-07-29T00:36:30.003485+0000","last_fullsized":"2024-07-29T00:36:30.003485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'52","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.216413+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:36:30.216413+0000","last_peered":"2024-07-29T00:36:30.216413+0000","last_clean":"2024-07-29T00:36:30.216413+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:36:30.216413+0000","last_undegraded":"2024-07-29T00:36:30.216413+0000","last_fullsized":"2024-07-29T00:36:30.216413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006932+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:36:30.006932+0000","last_peered":"2024-07-29T00:36:30.006932+0000","last_clean":"2024-07-29T00:36:30.006932+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:36:30.006932+0000","last_undegraded":"2024-07-29T00:36:30.006932+0000","last_fullsized":"2024-07-29T00:36:30.006932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919572+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:36:29.919572+0000","last_peered":"2024-07-29T00:36:29.919572+0000","last_clean":"2024-07-29T00:36:29.919572+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:36:29.919572+0000","last_undegraded":"2024-07-29T00:36:29.919572+0000","last_fullsized":"2024-07-29T00:36:29.919572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122292+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:36:30.122292+0000","last_peered":"2024-07-29T00:36:30.122292+0000","last_clean":"2024-07-29T00:36:30.122292+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:36:30.122292+0000","last_undegraded":"2024-07-29T00:36:30.122292+0000","last_fullsized":"2024-07-29T00:36:30.122292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918120+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:36:29.918120+0000","last_peered":"2024-07-29T00:36:29.918120+0000","last_clean":"2024-07-29T00:36:29.918120+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:36:29.918120+0000","last_undegraded":"2024-07-29T00:36:29.918120+0000","last_fullsized":"2024-07-29T00:36:29.918120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029897+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:36:30.029897+0000","last_peered":"2024-07-29T00:36:30.029897+0000","last_clean":"2024-07-29T00:36:30.029897+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:36:30.029897+0000","last_undegraded":"2024-07-29T00:36:30.029897+0000","last_fullsized":"2024-07-29T00:36:30.029897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006317+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:36:30.006317+0000","last_peered":"2024-07-29T00:36:30.006317+0000","last_clean":"2024-07-29T00:36:30.006317+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:36:30.006317+0000","last_undegraded":"2024-07-29T00:36:30.006317+0000","last_fullsized":"2024-07-29T00:36:30.006317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270586+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:36:30.270586+0000","last_peered":"2024-07-29T00:36:30.270586+0000","last_clean":"2024-07-29T00:36:30.270586+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:36:30.270586+0000","last_undegraded":"2024-07-29T00:36:30.270586+0000","last_fullsized":"2024-07-29T00:36:30.270586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119490+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:36:30.119490+0000","last_peered":"2024-07-29T00:36:30.119490+0000","last_clean":"2024-07-29T00:36:30.119490+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:36:30.119490+0000","last_undegraded":"2024-07-29T00:36:30.119490+0000","last_fullsized":"2024-07-29T00:36:30.119490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006281+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:36:30.006281+0000","last_peered":"2024-07-29T00:36:30.006281+0000","last_clean":"2024-07-29T00:36:30.006281+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:36:30.006281+0000","last_undegraded":"2024-07-29T00:36:30.006281+0000","last_fullsized":"2024-07-29T00:36:30.006281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029962+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:36:30.029962+0000","last_peered":"2024-07-29T00:36:30.029962+0000","last_clean":"2024-07-29T00:36:30.029962+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:36:30.029962+0000","last_undegraded":"2024-07-29T00:36:30.029962+0000","last_fullsized":"2024-07-29T00:36:30.029962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978784+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:36:29.978784+0000","last_peered":"2024-07-29T00:36:29.978784+0000","last_clean":"2024-07-29T00:36:29.978784+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:36:29.978784+0000","last_undegraded":"2024-07-29T00:36:29.978784+0000","last_fullsized":"2024-07-29T00:36:29.978784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978667+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:36:29.978667+0000","last_peered":"2024-07-29T00:36:29.978667+0000","last_clean":"2024-07-29T00:36:29.978667+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:36:29.978667+0000","last_undegraded":"2024-07-29T00:36:29.978667+0000","last_fullsized":"2024-07-29T00:36:29.978667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030262+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:36:30.030262+0000","last_peered":"2024-07-29T00:36:30.030262+0000","last_clean":"2024-07-29T00:36:30.030262+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:36:30.030262+0000","last_undegraded":"2024-07-29T00:36:30.030262+0000","last_fullsized":"2024-07-29T00:36:30.030262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918288+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:36:29.918288+0000","last_peered":"2024-07-29T00:36:29.918288+0000","last_clean":"2024-07-29T00:36:29.918288+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:36:29.918288+0000","last_undegraded":"2024-07-29T00:36:29.918288+0000","last_fullsized":"2024-07-29T00:36:29.918288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270633+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:36:30.270633+0000","last_peered":"2024-07-29T00:36:30.270633+0000","last_clean":"2024-07-29T00:36:30.270633+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:36:30.270633+0000","last_undegraded":"2024-07-29T00:36:30.270633+0000","last_fullsized":"2024-07-29T00:36:30.270633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122784+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:36:30.122784+0000","last_peered":"2024-07-29T00:36:30.122784+0000","last_clean":"2024-07-29T00:36:30.122784+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:36:30.122784+0000","last_undegraded":"2024-07-29T00:36:30.122784+0000","last_fullsized":"2024-07-29T00:36:30.122784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006887+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:36:30.006887+0000","last_peered":"2024-07-29T00:36:30.006887+0000","last_clean":"2024-07-29T00:36:30.006887+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:36:30.006887+0000","last_undegraded":"2024-07-29T00:36:30.006887+0000","last_fullsized":"2024-07-29T00:36:30.006887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921570+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:36:29.921570+0000","last_peered":"2024-07-29T00:36:29.921570+0000","last_clean":"2024-07-29T00:36:29.921570+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:36:29.921570+0000","last_undegraded":"2024-07-29T00:36:29.921570+0000","last_fullsized":"2024-07-29T00:36:29.921570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":195,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:38.993474+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:36:38.993474+0000","last_peered":"2024-07-29T00:36:38.993474+0000","last_clean":"2024-07-29T00:36:38.993474+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:36:38.993474+0000","last_undegraded":"2024-07-29T00:36:38.993474+0000","last_fullsized":"2024-07-29T00:36:38.993474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797518,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1610,"num_read_kb":946,"num_write":406,"num_write_kb":1933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":1999290,"data_compressed":94535,"data_compressed_allocated":122880,"data_compressed_original":1789952,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":501919,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678067,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1291549,"data_compressed":56465,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841593,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":894285,"data_compressed":33607,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":894663,"data_compressed":33607,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332164,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1095972,"data_compressed":50309,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":501093,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":1422264,"data_compressed":65511,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":570378,"data_compressed":24044,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1025427,"data_compressed":39763,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":501529,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":568476,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":502905,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":70431,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331832,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":201117,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:36:48.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:36:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:48 smithi181 ceph-mon[36706]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:48 smithi175 ceph-mon[32138]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:48 smithi204 ceph-mon[37368]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:48.890 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:49.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 -- 172.21.15.175:0/585997271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468104240 msgr2=0x7f7468110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:49.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 --2- 172.21.15.175:0/585997271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468104240 0x7f7468110770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7460009cd0 tx=0x7f746002a8f0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 -- 172.21.15.175:0/585997271 shutdown_connections 2024-07-29T00:36:49.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 --2- 172.21.15.175:0/585997271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468104240 0x7f7468110770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 --2- 172.21.15.175:0/585997271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74681038a0 0x7f7468103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 --2- 172.21.15.175:0/585997271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 0x7f7468109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.538+0000 7f746de00640 1 -- 172.21.15.175:0/585997271 >> 172.21.15.175:0/585997271 conn(0x7f74680ff560 msgr2=0x7f7468101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.539+0000 7f746de00640 1 -- 172.21.15.175:0/585997271 shutdown_connections 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.539+0000 7f746de00640 1 -- 172.21.15.175:0/585997271 wait complete. 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 Processor -- start 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 -- start start 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 0x7f746819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7468104240 0x7f746819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 0x7f746819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7468113ca0 con 0x7f74681038a0 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7468113b20 con 0x7f74681098a0 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7468113e20 con 0x7f7468104240 2024-07-29T00:36:49.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 0x7f746819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 0x7f746819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38440/0 (socket says 172.21.15.175:38440) 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746ca00640 1 -- 172.21.15.175:0/2011109879 learned_addr learned my addr 172.21.15.175:0/2011109879 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f7467e00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7468104240 0x7f746819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.540+0000 7f746d400640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 0x7f746819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746ca00640 1 -- 172.21.15.175:0/2011109879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7468104240 msgr2=0x7f746819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746ca00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7468104240 0x7f746819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746ca00640 1 -- 172.21.15.175:0/2011109879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 msgr2=0x7f746819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746ca00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 0x7f746819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746ca00640 1 -- 172.21.15.175:0/2011109879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746819f130 con 0x7f74681038a0 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746d400640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 0x7f746819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:49.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f746ca00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 0x7f746819a180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f745c00e9f0 tx=0x7f745c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c00cd80 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745c010910 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.541+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c018990 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.542+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746819f420 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.542+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a6d00 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.543+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7468104fe0 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.545+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f745c010ac0 con 0x7f74681038a0 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.546+0000 7f7465600640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f74500a2e00 0x7f74500a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:49.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.546+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f745c0cf840 con 0x7f74681038a0 2024-07-29T00:36:49.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.546+0000 7f7467e00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f74500a2e00 0x7f74500a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:49.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.547+0000 7f7467e00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f74500a2e00 0x7f74500a52c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7468101b90 tx=0x7f7454005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:49.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.550+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f745c09bc20 con 0x7f74681038a0 2024-07-29T00:36:49.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.737+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7468062320 con 0x7f74500a2e00 2024-07-29T00:36:49.738 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:49 smithi175 ceph-mon[32138]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 53 op/s 2024-07-29T00:36:49.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:49 smithi181 ceph-mon[36706]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 53 op/s 2024-07-29T00:36:49.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.758+0000 7f7465600640 1 -- 172.21.15.175:0/2011109879 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258136 (secure 0 0 0) 0x7f7468062320 con 0x7f74500a2e00 2024-07-29T00:36:49.763 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:49.764 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:36:49.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f74500a2e00 msgr2=0x7f74500a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:49.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f74500a2e00 0x7f74500a52c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7468101b90 tx=0x7f7454005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:49.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 msgr2=0x7f746819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:49.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 0x7f746819a180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f745c00e9f0 tx=0x7f745c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 shutdown_connections 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74681098a0 0x7f746819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7468104240 0x7f746819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f74500a2e00 0x7f74500a52c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 --2- 172.21.15.175:0/2011109879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74681038a0 0x7f746819a180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 >> 172.21.15.175:0/2011109879 conn(0x7f74680ff560 msgr2=0x7f74680ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 shutdown_connections 2024-07-29T00:36:49.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:49.764+0000 7f746de00640 1 -- 172.21.15.175:0/2011109879 wait complete. 2024-07-29T00:36:49.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:49 smithi204 ceph-mon[37368]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 53 op/s 2024-07-29T00:36:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:50 smithi175 ceph-mon[32138]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:50.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:50 smithi181 ceph-mon[36706]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:50.823 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-07-29T00:36:47.945498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388120,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2114,"num_read_kb":1505,"num_write":708,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1034,"ondisk_log_size":1034,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361816,"kb_used_data":12616,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580008,"statfs":{"total":1151940427776,"available":1151569928192,"internally_reserved":0,"allocated":12918784,"data_stored":10163724,"data_compressed":394126,"data_compressed_allocated":528384,"data_compressed_original":4419584,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":502,"num_read_kb":519,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005689"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270840+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:36:30.270840+0000","last_peered":"2024-07-29T00:36:30.270840+0000","last_clean":"2024-07-29T00:36:30.270840+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:36:30.270840+0000","last_undegraded":"2024-07-29T00:36:30.270840+0000","last_fullsized":"2024-07-29T00:36:30.270840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122176+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:36:30.122176+0000","last_peered":"2024-07-29T00:36:30.122176+0000","last_clean":"2024-07-29T00:36:30.122176+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:36:30.122176+0000","last_undegraded":"2024-07-29T00:36:30.122176+0000","last_fullsized":"2024-07-29T00:36:30.122176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121890+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:36:30.121890+0000","last_peered":"2024-07-29T00:36:30.121890+0000","last_clean":"2024-07-29T00:36:30.121890+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:36:30.121890+0000","last_undegraded":"2024-07-29T00:36:30.121890+0000","last_fullsized":"2024-07-29T00:36:30.121890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919430+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:36:29.919430+0000","last_peered":"2024-07-29T00:36:29.919430+0000","last_clean":"2024-07-29T00:36:29.919430+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:36:29.919430+0000","last_undegraded":"2024-07-29T00:36:29.919430+0000","last_fullsized":"2024-07-29T00:36:29.919430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030174+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:36:30.030174+0000","last_peered":"2024-07-29T00:36:30.030174+0000","last_clean":"2024-07-29T00:36:30.030174+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:36:30.030174+0000","last_undegraded":"2024-07-29T00:36:30.030174+0000","last_fullsized":"2024-07-29T00:36:30.030174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039832+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:36:30.039832+0000","last_peered":"2024-07-29T00:36:30.039832+0000","last_clean":"2024-07-29T00:36:30.039832+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:36:30.039832+0000","last_undegraded":"2024-07-29T00:36:30.039832+0000","last_fullsized":"2024-07-29T00:36:30.039832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978714+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:36:29.978714+0000","last_peered":"2024-07-29T00:36:29.978714+0000","last_clean":"2024-07-29T00:36:29.978714+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:36:29.978714+0000","last_undegraded":"2024-07-29T00:36:29.978714+0000","last_fullsized":"2024-07-29T00:36:29.978714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006206+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:36:30.006206+0000","last_peered":"2024-07-29T00:36:30.006206+0000","last_clean":"2024-07-29T00:36:30.006206+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:36:30.006206+0000","last_undegraded":"2024-07-29T00:36:30.006206+0000","last_fullsized":"2024-07-29T00:36:30.006206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918987+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:36:29.918987+0000","last_peered":"2024-07-29T00:36:29.918987+0000","last_clean":"2024-07-29T00:36:29.918987+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:36:29.918987+0000","last_undegraded":"2024-07-29T00:36:29.918987+0000","last_fullsized":"2024-07-29T00:36:29.918987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.040027+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:36:30.040027+0000","last_peered":"2024-07-29T00:36:30.040027+0000","last_clean":"2024-07-29T00:36:30.040027+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:36:30.040027+0000","last_undegraded":"2024-07-29T00:36:30.040027+0000","last_fullsized":"2024-07-29T00:36:30.040027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978875+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:36:29.978875+0000","last_peered":"2024-07-29T00:36:29.978875+0000","last_clean":"2024-07-29T00:36:29.978875+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:36:29.978875+0000","last_undegraded":"2024-07-29T00:36:29.978875+0000","last_fullsized":"2024-07-29T00:36:29.978875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919501+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:36:29.919501+0000","last_peered":"2024-07-29T00:36:29.919501+0000","last_clean":"2024-07-29T00:36:29.919501+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:36:29.919501+0000","last_undegraded":"2024-07-29T00:36:29.919501+0000","last_fullsized":"2024-07-29T00:36:29.919501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003149+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:36:30.003149+0000","last_peered":"2024-07-29T00:36:30.003149+0000","last_clean":"2024-07-29T00:36:30.003149+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:36:30.003149+0000","last_undegraded":"2024-07-29T00:36:30.003149+0000","last_fullsized":"2024-07-29T00:36:30.003149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122517+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:36:30.122517+0000","last_peered":"2024-07-29T00:36:30.122517+0000","last_clean":"2024-07-29T00:36:30.122517+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:36:30.122517+0000","last_undegraded":"2024-07-29T00:36:30.122517+0000","last_fullsized":"2024-07-29T00:36:30.122517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121450+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:36:30.121450+0000","last_peered":"2024-07-29T00:36:30.121450+0000","last_clean":"2024-07-29T00:36:30.121450+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:36:30.121450+0000","last_undegraded":"2024-07-29T00:36:30.121450+0000","last_fullsized":"2024-07-29T00:36:30.121450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918029+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:36:29.918029+0000","last_peered":"2024-07-29T00:36:29.918029+0000","last_clean":"2024-07-29T00:36:29.918029+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:36:29.918029+0000","last_undegraded":"2024-07-29T00:36:29.918029+0000","last_fullsized":"2024-07-29T00:36:29.918029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270846+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:36:30.270846+0000","last_peered":"2024-07-29T00:36:30.270846+0000","last_clean":"2024-07-29T00:36:30.270846+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:36:30.270846+0000","last_undegraded":"2024-07-29T00:36:30.270846+0000","last_fullsized":"2024-07-29T00:36:30.270846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921670+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:36:29.921670+0000","last_peered":"2024-07-29T00:36:29.921670+0000","last_clean":"2024-07-29T00:36:29.921670+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:36:29.921670+0000","last_undegraded":"2024-07-29T00:36:29.921670+0000","last_fullsized":"2024-07-29T00:36:29.921670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119558+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:36:30.119558+0000","last_peered":"2024-07-29T00:36:30.119558+0000","last_clean":"2024-07-29T00:36:30.119558+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:36:30.119558+0000","last_undegraded":"2024-07-29T00:36:30.119558+0000","last_fullsized":"2024-07-29T00:36:30.119558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270504+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:36:30.270504+0000","last_peered":"2024-07-29T00:36:30.270504+0000","last_clean":"2024-07-29T00:36:30.270504+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:36:30.270504+0000","last_undegraded":"2024-07-29T00:36:30.270504+0000","last_fullsized":"2024-07-29T00:36:30.270504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003134+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:36:30.003134+0000","last_peered":"2024-07-29T00:36:30.003134+0000","last_clean":"2024-07-29T00:36:30.003134+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:36:30.003134+0000","last_undegraded":"2024-07-29T00:36:30.003134+0000","last_fullsized":"2024-07-29T00:36:30.003134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918840+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:36:29.918840+0000","last_peered":"2024-07-29T00:36:29.918840+0000","last_clean":"2024-07-29T00:36:29.918840+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:36:29.918840+0000","last_undegraded":"2024-07-29T00:36:29.918840+0000","last_fullsized":"2024-07-29T00:36:29.918840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006207+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:36:30.006207+0000","last_peered":"2024-07-29T00:36:30.006207+0000","last_clean":"2024-07-29T00:36:30.006207+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:36:30.006207+0000","last_undegraded":"2024-07-29T00:36:30.006207+0000","last_fullsized":"2024-07-29T00:36:30.006207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978362+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:36:29.978362+0000","last_peered":"2024-07-29T00:36:29.978362+0000","last_clean":"2024-07-29T00:36:29.978362+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:36:29.978362+0000","last_undegraded":"2024-07-29T00:36:29.978362+0000","last_fullsized":"2024-07-29T00:36:29.978362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006816+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:36:30.006816+0000","last_peered":"2024-07-29T00:36:30.006816+0000","last_clean":"2024-07-29T00:36:30.006816+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:36:30.006816+0000","last_undegraded":"2024-07-29T00:36:30.006816+0000","last_fullsized":"2024-07-29T00:36:30.006816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119656+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:36:30.119656+0000","last_peered":"2024-07-29T00:36:30.119656+0000","last_clean":"2024-07-29T00:36:30.119656+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:36:30.119656+0000","last_undegraded":"2024-07-29T00:36:30.119656+0000","last_fullsized":"2024-07-29T00:36:30.119656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270796+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:36:30.270796+0000","last_peered":"2024-07-29T00:36:30.270796+0000","last_clean":"2024-07-29T00:36:30.270796+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:36:30.270796+0000","last_undegraded":"2024-07-29T00:36:30.270796+0000","last_fullsized":"2024-07-29T00:36:30.270796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003369+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:36:30.003369+0000","last_peered":"2024-07-29T00:36:30.003369+0000","last_clean":"2024-07-29T00:36:30.003369+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:36:30.003369+0000","last_undegraded":"2024-07-29T00:36:30.003369+0000","last_fullsized":"2024-07-29T00:36:30.003369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121371+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:36:30.121371+0000","last_peered":"2024-07-29T00:36:30.121371+0000","last_clean":"2024-07-29T00:36:30.121371+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:36:30.121371+0000","last_undegraded":"2024-07-29T00:36:30.121371+0000","last_fullsized":"2024-07-29T00:36:30.121371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119269+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:36:30.119269+0000","last_peered":"2024-07-29T00:36:30.119269+0000","last_clean":"2024-07-29T00:36:30.119269+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:36:30.119269+0000","last_undegraded":"2024-07-29T00:36:30.119269+0000","last_fullsized":"2024-07-29T00:36:30.119269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119276+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:36:30.119276+0000","last_peered":"2024-07-29T00:36:30.119276+0000","last_clean":"2024-07-29T00:36:30.119276+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:36:30.119276+0000","last_undegraded":"2024-07-29T00:36:30.119276+0000","last_fullsized":"2024-07-29T00:36:30.119276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270536+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:36:30.270536+0000","last_peered":"2024-07-29T00:36:30.270536+0000","last_clean":"2024-07-29T00:36:30.270536+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:36:30.270536+0000","last_undegraded":"2024-07-29T00:36:30.270536+0000","last_fullsized":"2024-07-29T00:36:30.270536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029964+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:36:30.029964+0000","last_peered":"2024-07-29T00:36:30.029964+0000","last_clean":"2024-07-29T00:36:30.029964+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:36:30.029964+0000","last_undegraded":"2024-07-29T00:36:30.029964+0000","last_fullsized":"2024-07-29T00:36:30.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":196,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.906520+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:36:44.906520+0000","last_peered":"2024-07-29T00:36:44.906520+0000","last_clean":"2024-07-29T00:36:44.906520+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:36:44.906520+0000","last_undegraded":"2024-07-29T00:36:44.906520+0000","last_fullsized":"2024-07-29T00:36:44.906520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978592+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:36:29.978592+0000","last_peered":"2024-07-29T00:36:29.978592+0000","last_clean":"2024-07-29T00:36:29.978592+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:36:29.978592+0000","last_undegraded":"2024-07-29T00:36:29.978592+0000","last_fullsized":"2024-07-29T00:36:29.978592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039762+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:36:30.039762+0000","last_peered":"2024-07-29T00:36:30.039762+0000","last_clean":"2024-07-29T00:36:30.039762+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:36:30.039762+0000","last_undegraded":"2024-07-29T00:36:30.039762+0000","last_fullsized":"2024-07-29T00:36:30.039762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002737+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:36:30.002737+0000","last_peered":"2024-07-29T00:36:30.002737+0000","last_clean":"2024-07-29T00:36:30.002737+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:36:30.002737+0000","last_undegraded":"2024-07-29T00:36:30.002737+0000","last_fullsized":"2024-07-29T00:36:30.002737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006358+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:36:30.006358+0000","last_peered":"2024-07-29T00:36:30.006358+0000","last_clean":"2024-07-29T00:36:30.006358+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:36:30.006358+0000","last_undegraded":"2024-07-29T00:36:30.006358+0000","last_fullsized":"2024-07-29T00:36:30.006358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978697+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:36:29.978697+0000","last_peered":"2024-07-29T00:36:29.978697+0000","last_clean":"2024-07-29T00:36:29.978697+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:36:29.978697+0000","last_undegraded":"2024-07-29T00:36:29.978697+0000","last_fullsized":"2024-07-29T00:36:29.978697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.920897+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:36:29.920897+0000","last_peered":"2024-07-29T00:36:29.920897+0000","last_clean":"2024-07-29T00:36:29.920897+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:36:29.920897+0000","last_undegraded":"2024-07-29T00:36:29.920897+0000","last_fullsized":"2024-07-29T00:36:29.920897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":344,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:45.864975+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:36:45.864975+0000","last_peered":"2024-07-29T00:36:45.864975+0000","last_clean":"2024-07-29T00:36:45.864975+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:36:45.864975+0000","last_undegraded":"2024-07-29T00:36:45.864975+0000","last_fullsized":"2024-07-29T00:36:45.864975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":305,"num_read_kb":305,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918684+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:36:29.918684+0000","last_peered":"2024-07-29T00:36:29.918684+0000","last_clean":"2024-07-29T00:36:29.918684+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:36:29.918684+0000","last_undegraded":"2024-07-29T00:36:29.918684+0000","last_fullsized":"2024-07-29T00:36:29.918684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979141+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:36:29.979141+0000","last_peered":"2024-07-29T00:36:29.979141+0000","last_clean":"2024-07-29T00:36:29.979141+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:36:29.979141+0000","last_undegraded":"2024-07-29T00:36:29.979141+0000","last_fullsized":"2024-07-29T00:36:29.979141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122906+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:36:30.122906+0000","last_peered":"2024-07-29T00:36:30.122906+0000","last_clean":"2024-07-29T00:36:30.122906+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:36:30.122906+0000","last_undegraded":"2024-07-29T00:36:30.122906+0000","last_fullsized":"2024-07-29T00:36:30.122906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978649+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:36:29.978649+0000","last_peered":"2024-07-29T00:36:29.978649+0000","last_clean":"2024-07-29T00:36:29.978649+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:36:29.978649+0000","last_undegraded":"2024-07-29T00:36:29.978649+0000","last_fullsized":"2024-07-29T00:36:29.978649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918482+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:36:29.918482+0000","last_peered":"2024-07-29T00:36:29.918482+0000","last_clean":"2024-07-29T00:36:29.918482+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:36:29.918482+0000","last_undegraded":"2024-07-29T00:36:29.918482+0000","last_fullsized":"2024-07-29T00:36:29.918482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":408,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:45.869732+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:36:45.869732+0000","last_peered":"2024-07-29T00:36:45.869732+0000","last_clean":"2024-07-29T00:36:45.869732+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:36:45.869732+0000","last_undegraded":"2024-07-29T00:36:45.869732+0000","last_fullsized":"2024-07-29T00:36:45.869732+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66470,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":317,"num_read_kb":224,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030244+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:36:30.030244+0000","last_peered":"2024-07-29T00:36:30.030244+0000","last_clean":"2024-07-29T00:36:30.030244+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:36:30.030244+0000","last_undegraded":"2024-07-29T00:36:30.030244+0000","last_fullsized":"2024-07-29T00:36:30.030244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119352+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:36:30.119352+0000","last_peered":"2024-07-29T00:36:30.119352+0000","last_clean":"2024-07-29T00:36:30.119352+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:36:30.119352+0000","last_undegraded":"2024-07-29T00:36:30.119352+0000","last_fullsized":"2024-07-29T00:36:30.119352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002636+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:36:30.002636+0000","last_peered":"2024-07-29T00:36:30.002636+0000","last_clean":"2024-07-29T00:36:30.002636+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:36:30.002636+0000","last_undegraded":"2024-07-29T00:36:30.002636+0000","last_fullsized":"2024-07-29T00:36:30.002636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978452+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:36:29.978452+0000","last_peered":"2024-07-29T00:36:29.978452+0000","last_clean":"2024-07-29T00:36:29.978452+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:36:29.978452+0000","last_undegraded":"2024-07-29T00:36:29.978452+0000","last_fullsized":"2024-07-29T00:36:29.978452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978851+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:36:29.978851+0000","last_peered":"2024-07-29T00:36:29.978851+0000","last_clean":"2024-07-29T00:36:29.978851+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:36:29.978851+0000","last_undegraded":"2024-07-29T00:36:29.978851+0000","last_fullsized":"2024-07-29T00:36:29.978851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":199,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.032497+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:36:44.032497+0000","last_peered":"2024-07-29T00:36:44.032497+0000","last_clean":"2024-07-29T00:36:44.032497+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:36:44.032497+0000","last_undegraded":"2024-07-29T00:36:44.032497+0000","last_fullsized":"2024-07-29T00:36:44.032497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978642+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:36:29.978642+0000","last_peered":"2024-07-29T00:36:29.978642+0000","last_clean":"2024-07-29T00:36:29.978642+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:36:29.978642+0000","last_undegraded":"2024-07-29T00:36:29.978642+0000","last_fullsized":"2024-07-29T00:36:29.978642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029821+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:36:30.029821+0000","last_peered":"2024-07-29T00:36:30.029821+0000","last_clean":"2024-07-29T00:36:30.029821+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:36:30.029821+0000","last_undegraded":"2024-07-29T00:36:30.029821+0000","last_fullsized":"2024-07-29T00:36:30.029821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978327+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:36:29.978327+0000","last_peered":"2024-07-29T00:36:29.978327+0000","last_clean":"2024-07-29T00:36:29.978327+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:36:29.978327+0000","last_undegraded":"2024-07-29T00:36:29.978327+0000","last_fullsized":"2024-07-29T00:36:29.978327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030159+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:36:30.030159+0000","last_peered":"2024-07-29T00:36:30.030159+0000","last_clean":"2024-07-29T00:36:30.030159+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:36:30.030159+0000","last_undegraded":"2024-07-29T00:36:30.030159+0000","last_fullsized":"2024-07-29T00:36:30.030159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270695+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:36:30.270695+0000","last_peered":"2024-07-29T00:36:30.270695+0000","last_clean":"2024-07-29T00:36:30.270695+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:36:30.270695+0000","last_undegraded":"2024-07-29T00:36:30.270695+0000","last_fullsized":"2024-07-29T00:36:30.270695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003638+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:36:30.003638+0000","last_peered":"2024-07-29T00:36:30.003638+0000","last_clean":"2024-07-29T00:36:30.003638+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:36:30.003638+0000","last_undegraded":"2024-07-29T00:36:30.003638+0000","last_fullsized":"2024-07-29T00:36:30.003638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979263+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:36:29.979263+0000","last_peered":"2024-07-29T00:36:29.979263+0000","last_clean":"2024-07-29T00:36:29.979263+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:36:29.979263+0000","last_undegraded":"2024-07-29T00:36:29.979263+0000","last_fullsized":"2024-07-29T00:36:29.979263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'67","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.952930+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:36:44.952930+0000","last_peered":"2024-07-29T00:36:44.952930+0000","last_clean":"2024-07-29T00:36:44.952930+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:36:44.952930+0000","last_undegraded":"2024-07-29T00:36:44.952930+0000","last_fullsized":"2024-07-29T00:36:44.952930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978805+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:36:29.978805+0000","last_peered":"2024-07-29T00:36:29.978805+0000","last_clean":"2024-07-29T00:36:29.978805+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:36:29.978805+0000","last_undegraded":"2024-07-29T00:36:29.978805+0000","last_fullsized":"2024-07-29T00:36:29.978805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.863870+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:36:44.863870+0000","last_peered":"2024-07-29T00:36:44.863870+0000","last_clean":"2024-07-29T00:36:44.863870+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:36:44.863870+0000","last_undegraded":"2024-07-29T00:36:44.863870+0000","last_fullsized":"2024-07-29T00:36:44.863870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979102+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:36:29.979102+0000","last_peered":"2024-07-29T00:36:29.979102+0000","last_clean":"2024-07-29T00:36:29.979102+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:36:29.979102+0000","last_undegraded":"2024-07-29T00:36:29.979102+0000","last_fullsized":"2024-07-29T00:36:29.979102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":408,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:32.081815+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:36:32.081815+0000","last_peered":"2024-07-29T00:36:32.081815+0000","last_clean":"2024-07-29T00:36:32.081815+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:36:32.081815+0000","last_undegraded":"2024-07-29T00:36:32.081815+0000","last_fullsized":"2024-07-29T00:36:32.081815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918549+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:36:29.918549+0000","last_peered":"2024-07-29T00:36:29.918549+0000","last_clean":"2024-07-29T00:36:29.918549+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:36:29.918549+0000","last_undegraded":"2024-07-29T00:36:29.918549+0000","last_fullsized":"2024-07-29T00:36:29.918549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006458+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:36:30.006458+0000","last_peered":"2024-07-29T00:36:30.006458+0000","last_clean":"2024-07-29T00:36:30.006458+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:36:30.006458+0000","last_undegraded":"2024-07-29T00:36:30.006458+0000","last_fullsized":"2024-07-29T00:36:30.006458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270459+0000","last_change":"2024-07-29T00:35:51.676982+0000","last_active":"2024-07-29T00:36:30.270459+0000","last_peered":"2024-07-29T00:36:30.270459+0000","last_clean":"2024-07-29T00:36:30.270459+0000","last_became_active":"2024-07-29T00:35:51.674179+0000","last_became_peered":"2024-07-29T00:35:51.674179+0000","last_unstale":"2024-07-29T00:36:30.270459+0000","last_undegraded":"2024-07-29T00:36:30.270459+0000","last_fullsized":"2024-07-29T00:36:30.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:24:23.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002590+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:36:30.002590+0000","last_peered":"2024-07-29T00:36:30.002590+0000","last_clean":"2024-07-29T00:36:30.002590+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:36:30.002590+0000","last_undegraded":"2024-07-29T00:36:30.002590+0000","last_fullsized":"2024-07-29T00:36:30.002590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":193,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.031746+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:36:44.031746+0000","last_peered":"2024-07-29T00:36:44.031746+0000","last_clean":"2024-07-29T00:36:44.031746+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:36:44.031746+0000","last_undegraded":"2024-07-29T00:36:44.031746+0000","last_fullsized":"2024-07-29T00:36:44.031746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978450+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:36:29.978450+0000","last_peered":"2024-07-29T00:36:29.978450+0000","last_clean":"2024-07-29T00:36:29.978450+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:36:29.978450+0000","last_undegraded":"2024-07-29T00:36:29.978450+0000","last_fullsized":"2024-07-29T00:36:29.978450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":193,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.004283+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:36:44.004283+0000","last_peered":"2024-07-29T00:36:44.004283+0000","last_clean":"2024-07-29T00:36:44.004283+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:36:44.004283+0000","last_undegraded":"2024-07-29T00:36:44.004283+0000","last_fullsized":"2024-07-29T00:36:44.004283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":895,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039767+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:36:30.039767+0000","last_peered":"2024-07-29T00:36:30.039767+0000","last_clean":"2024-07-29T00:36:30.039767+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:36:30.039767+0000","last_undegraded":"2024-07-29T00:36:30.039767+0000","last_fullsized":"2024-07-29T00:36:30.039767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921376+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:36:29.921376+0000","last_peered":"2024-07-29T00:36:29.921376+0000","last_clean":"2024-07-29T00:36:29.921376+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:36:29.921376+0000","last_undegraded":"2024-07-29T00:36:29.921376+0000","last_fullsized":"2024-07-29T00:36:29.921376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039877+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:36:30.039877+0000","last_peered":"2024-07-29T00:36:30.039877+0000","last_clean":"2024-07-29T00:36:30.039877+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:36:30.039877+0000","last_undegraded":"2024-07-29T00:36:30.039877+0000","last_fullsized":"2024-07-29T00:36:30.039877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030289+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:36:30.030289+0000","last_peered":"2024-07-29T00:36:30.030289+0000","last_clean":"2024-07-29T00:36:30.030289+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:36:30.030289+0000","last_undegraded":"2024-07-29T00:36:30.030289+0000","last_fullsized":"2024-07-29T00:36:30.030289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003485+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:36:30.003485+0000","last_peered":"2024-07-29T00:36:30.003485+0000","last_clean":"2024-07-29T00:36:30.003485+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:36:30.003485+0000","last_undegraded":"2024-07-29T00:36:30.003485+0000","last_fullsized":"2024-07-29T00:36:30.003485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'105","reported_seq":230,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:45.432574+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:36:45.432574+0000","last_peered":"2024-07-29T00:36:45.432574+0000","last_clean":"2024-07-29T00:36:45.432574+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:36:45.432574+0000","last_undegraded":"2024-07-29T00:36:45.432574+0000","last_fullsized":"2024-07-29T00:36:45.432574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006932+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:36:30.006932+0000","last_peered":"2024-07-29T00:36:30.006932+0000","last_clean":"2024-07-29T00:36:30.006932+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:36:30.006932+0000","last_undegraded":"2024-07-29T00:36:30.006932+0000","last_fullsized":"2024-07-29T00:36:30.006932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919572+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:36:29.919572+0000","last_peered":"2024-07-29T00:36:29.919572+0000","last_clean":"2024-07-29T00:36:29.919572+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:36:29.919572+0000","last_undegraded":"2024-07-29T00:36:29.919572+0000","last_fullsized":"2024-07-29T00:36:29.919572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122292+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:36:30.122292+0000","last_peered":"2024-07-29T00:36:30.122292+0000","last_clean":"2024-07-29T00:36:30.122292+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:36:30.122292+0000","last_undegraded":"2024-07-29T00:36:30.122292+0000","last_fullsized":"2024-07-29T00:36:30.122292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918120+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:36:29.918120+0000","last_peered":"2024-07-29T00:36:29.918120+0000","last_clean":"2024-07-29T00:36:29.918120+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:36:29.918120+0000","last_undegraded":"2024-07-29T00:36:29.918120+0000","last_fullsized":"2024-07-29T00:36:29.918120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029897+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:36:30.029897+0000","last_peered":"2024-07-29T00:36:30.029897+0000","last_clean":"2024-07-29T00:36:30.029897+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:36:30.029897+0000","last_undegraded":"2024-07-29T00:36:30.029897+0000","last_fullsized":"2024-07-29T00:36:30.029897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006317+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:36:30.006317+0000","last_peered":"2024-07-29T00:36:30.006317+0000","last_clean":"2024-07-29T00:36:30.006317+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:36:30.006317+0000","last_undegraded":"2024-07-29T00:36:30.006317+0000","last_fullsized":"2024-07-29T00:36:30.006317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270586+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:36:30.270586+0000","last_peered":"2024-07-29T00:36:30.270586+0000","last_clean":"2024-07-29T00:36:30.270586+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:36:30.270586+0000","last_undegraded":"2024-07-29T00:36:30.270586+0000","last_fullsized":"2024-07-29T00:36:30.270586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119490+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:36:30.119490+0000","last_peered":"2024-07-29T00:36:30.119490+0000","last_clean":"2024-07-29T00:36:30.119490+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:36:30.119490+0000","last_undegraded":"2024-07-29T00:36:30.119490+0000","last_fullsized":"2024-07-29T00:36:30.119490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006281+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:36:30.006281+0000","last_peered":"2024-07-29T00:36:30.006281+0000","last_clean":"2024-07-29T00:36:30.006281+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:36:30.006281+0000","last_undegraded":"2024-07-29T00:36:30.006281+0000","last_fullsized":"2024-07-29T00:36:30.006281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029962+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:36:30.029962+0000","last_peered":"2024-07-29T00:36:30.029962+0000","last_clean":"2024-07-29T00:36:30.029962+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:36:30.029962+0000","last_undegraded":"2024-07-29T00:36:30.029962+0000","last_fullsized":"2024-07-29T00:36:30.029962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978784+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:36:29.978784+0000","last_peered":"2024-07-29T00:36:29.978784+0000","last_clean":"2024-07-29T00:36:29.978784+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:36:29.978784+0000","last_undegraded":"2024-07-29T00:36:29.978784+0000","last_fullsized":"2024-07-29T00:36:29.978784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978667+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:36:29.978667+0000","last_peered":"2024-07-29T00:36:29.978667+0000","last_clean":"2024-07-29T00:36:29.978667+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:36:29.978667+0000","last_undegraded":"2024-07-29T00:36:29.978667+0000","last_fullsized":"2024-07-29T00:36:29.978667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030262+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:36:30.030262+0000","last_peered":"2024-07-29T00:36:30.030262+0000","last_clean":"2024-07-29T00:36:30.030262+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:36:30.030262+0000","last_undegraded":"2024-07-29T00:36:30.030262+0000","last_fullsized":"2024-07-29T00:36:30.030262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918288+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:36:29.918288+0000","last_peered":"2024-07-29T00:36:29.918288+0000","last_clean":"2024-07-29T00:36:29.918288+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:36:29.918288+0000","last_undegraded":"2024-07-29T00:36:29.918288+0000","last_fullsized":"2024-07-29T00:36:29.918288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270633+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:36:30.270633+0000","last_peered":"2024-07-29T00:36:30.270633+0000","last_clean":"2024-07-29T00:36:30.270633+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:36:30.270633+0000","last_undegraded":"2024-07-29T00:36:30.270633+0000","last_fullsized":"2024-07-29T00:36:30.270633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122784+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:36:30.122784+0000","last_peered":"2024-07-29T00:36:30.122784+0000","last_clean":"2024-07-29T00:36:30.122784+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:36:30.122784+0000","last_undegraded":"2024-07-29T00:36:30.122784+0000","last_fullsized":"2024-07-29T00:36:30.122784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006887+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:36:30.006887+0000","last_peered":"2024-07-29T00:36:30.006887+0000","last_clean":"2024-07-29T00:36:30.006887+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:36:30.006887+0000","last_undegraded":"2024-07-29T00:36:30.006887+0000","last_fullsized":"2024-07-29T00:36:30.006887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921570+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:36:29.921570+0000","last_peered":"2024-07-29T00:36:29.921570+0000","last_clean":"2024-07-29T00:36:29.921570+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:36:29.921570+0000","last_undegraded":"2024-07-29T00:36:29.921570+0000","last_fullsized":"2024-07-29T00:36:29.921570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":201,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.003394+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:36:44.003394+0000","last_peered":"2024-07-29T00:36:44.003394+0000","last_clean":"2024-07-29T00:36:44.003394+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:36:44.003394+0000","last_undegraded":"2024-07-29T00:36:44.003394+0000","last_fullsized":"2024-07-29T00:36:44.003394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797518,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1877,"num_read_kb":1175,"num_write":505,"num_write_kb":2381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2392554,"data_compressed":113050,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":384,"ondisk_log_size":384,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416432,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":501919,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678068,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716936,"statfs":{"total":95995035648,"available":95966142464,"internally_reserved":0,"allocated":1220608,"data_stored":1488181,"data_compressed":65511,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841594,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":894285,"data_compressed":33748,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103231,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":894663,"data_compressed":33748,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332164,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1095972,"data_compressed":50309,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":501093,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":1422264,"data_compressed":65511,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":570378,"data_compressed":24044,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1222059,"data_compressed":48950,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":501529,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":568476,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":502905,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34489,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":70431,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331832,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":397749,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:36:50.826 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-29T00:36:50.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph pg dump --format=json 2024-07-29T00:36:50.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:50 smithi204 ceph-mon[37368]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:51.340 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:51.732 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:51 smithi175 ceph-mon[32138]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-07-29T00:36:51.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:51 smithi181 ceph-mon[36706]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-07-29T00:36:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:51 smithi204 ceph-mon[37368]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 -- 172.21.15.175:0/3585536391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 msgr2=0x7f8758103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 --2- 172.21.15.175:0/3585536391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f8758103ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8748009930 tx=0x7f874802a420 comp rx=0 tx=0).stop 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 -- 172.21.15.175:0/3585536391 shutdown_connections 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 --2- 172.21.15.175:0/3585536391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8758104420 0x7f8758110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 --2- 172.21.15.175:0/3585536391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f8758103ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 --2- 172.21.15.175:0/3585536391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758109a80 0x7f8758109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 -- 172.21.15.175:0/3585536391 >> 172.21.15.175:0/3585536391 conn(0x7f87580ff760 msgr2=0x7f8758101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 -- 172.21.15.175:0/3585536391 shutdown_connections 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.008+0000 7f875ce00640 1 -- 172.21.15.175:0/3585536391 wait complete. 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.009+0000 7f875ce00640 1 Processor -- start 2024-07-29T00:36:52.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 -- start start 2024-07-29T00:36:52.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f875819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:52.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758104420 0x7f875819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:52.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8758109a80 0x7f875819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8758113e80 con 0x7f8758104420 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8758113d00 con 0x7f8758109a80 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f875ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8758114000 con 0x7f8758103a80 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f8755400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758104420 0x7f875819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f8755e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f875819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f8755400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758104420 0x7f875819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38448/0 (socket says 172.21.15.175:38448) 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f8755400640 1 -- 172.21.15.175:0/3022622791 learned_addr learned my addr 172.21.15.175:0/3022622791 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.010+0000 7f8756800640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8758109a80 0x7f875819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.011+0000 7f8755e00640 1 -- 172.21.15.175:0/3022622791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8758109a80 msgr2=0x7f875819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.011+0000 7f8755e00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8758109a80 0x7f875819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.011+0000 7f8755e00640 1 -- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758104420 msgr2=0x7f875819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.011+0000 7f8755e00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758104420 0x7f875819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.011+0000 7f8755e00640 1 -- 172.21.15.175:0/3022622791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875819f320 con 0x7f8758103a80 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.011+0000 7f8755e00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f875819a370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f874400e9f0 tx=0x7f874400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.012+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874400ccc0 con 0x7f8758103a80 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.012+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875819f610 con 0x7f8758103a80 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.012+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8744010910 con 0x7f8758103a80 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.012+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744018960 con 0x7f8758103a80 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.012+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a6ef0 con 0x7f8758103a80 2024-07-29T00:36:52.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.013+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8758103ee0 con 0x7f8758103a80 2024-07-29T00:36:52.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.015+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f8744010ac0 con 0x7f8758103a80 2024-07-29T00:36:52.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.016+0000 7f874ea00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f873c0a2e00 0x7f873c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:52.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.016+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f87440cfd80 con 0x7f8758103a80 2024-07-29T00:36:52.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.017+0000 7f8755400640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f873c0a2e00 0x7f873c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:52.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.017+0000 7f8755400640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f873c0a2e00 0x7f873c0a52c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8758101d10 tx=0x7f8748009560 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:52.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.018+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f874409c160 con 0x7f8758103a80 2024-07-29T00:36:52.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.204+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8758062320 con 0x7f873c0a2e00 2024-07-29T00:36:52.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.212+0000 7f874ea00640 1 -- 172.21.15.175:0/3022622791 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258136 (secure 0 0 0) 0x7f8758062320 con 0x7f873c0a2e00 2024-07-29T00:36:52.215 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:52.219 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f873c0a2e00 msgr2=0x7f873c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f873c0a2e00 0x7f873c0a52c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8758101d10 tx=0x7f8748009560 comp rx=0 tx=0).stop 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 msgr2=0x7f875819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f875819a370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f874400e9f0 tx=0x7f874400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 shutdown_connections 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8758109a80 0x7f875819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f873c0a2e00 0x7f873c0a52c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8758104420 0x7f875819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 --2- 172.21.15.175:0/3022622791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8758103a80 0x7f875819a370 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:52.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.218+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 >> 172.21.15.175:0/3022622791 conn(0x7f87580ff760 msgr2=0x7f87580fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:52.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.219+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 shutdown_connections 2024-07-29T00:36:52.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:52.219+0000 7f875ce00640 1 -- 172.21.15.175:0/3022622791 wait complete. 2024-07-29T00:36:53.312 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-07-29T00:36:51.947162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388120,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2164,"num_read_kb":1530,"num_write":708,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1241667,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1034,"ondisk_log_size":1034,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361816,"kb_used_data":12616,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580008,"statfs":{"total":1151940427776,"available":1151569928192,"internally_reserved":0,"allocated":12918784,"data_stored":10163724,"data_compressed":394126,"data_compressed_allocated":528384,"data_compressed_original":4419584,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":405,"num_read_kb":307,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005785"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270840+0000","last_change":"2024-07-29T00:33:56.728341+0000","last_active":"2024-07-29T00:36:30.270840+0000","last_peered":"2024-07-29T00:36:30.270840+0000","last_clean":"2024-07-29T00:36:30.270840+0000","last_became_active":"2024-07-29T00:33:56.727999+0000","last_became_peered":"2024-07-29T00:33:56.727999+0000","last_unstale":"2024-07-29T00:36:30.270840+0000","last_undegraded":"2024-07-29T00:36:30.270840+0000","last_fullsized":"2024-07-29T00:36:30.270840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:39:32.342175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122176+0000","last_change":"2024-07-29T00:33:56.726770+0000","last_active":"2024-07-29T00:36:30.122176+0000","last_peered":"2024-07-29T00:36:30.122176+0000","last_clean":"2024-07-29T00:36:30.122176+0000","last_became_active":"2024-07-29T00:33:56.726208+0000","last_became_peered":"2024-07-29T00:33:56.726208+0000","last_unstale":"2024-07-29T00:36:30.122176+0000","last_undegraded":"2024-07-29T00:36:30.122176+0000","last_fullsized":"2024-07-29T00:36:30.122176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:02:15.967167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121890+0000","last_change":"2024-07-29T00:33:56.729368+0000","last_active":"2024-07-29T00:36:30.121890+0000","last_peered":"2024-07-29T00:36:30.121890+0000","last_clean":"2024-07-29T00:36:30.121890+0000","last_became_active":"2024-07-29T00:33:56.726708+0000","last_became_peered":"2024-07-29T00:33:56.726708+0000","last_unstale":"2024-07-29T00:36:30.121890+0000","last_undegraded":"2024-07-29T00:36:30.121890+0000","last_fullsized":"2024-07-29T00:36:30.121890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:03:18.694178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919430+0000","last_change":"2024-07-29T00:33:56.711678+0000","last_active":"2024-07-29T00:36:29.919430+0000","last_peered":"2024-07-29T00:36:29.919430+0000","last_clean":"2024-07-29T00:36:29.919430+0000","last_became_active":"2024-07-29T00:33:56.711284+0000","last_became_peered":"2024-07-29T00:33:56.711284+0000","last_unstale":"2024-07-29T00:36:29.919430+0000","last_undegraded":"2024-07-29T00:36:29.919430+0000","last_fullsized":"2024-07-29T00:36:29.919430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:12:36.604694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030174+0000","last_change":"2024-07-29T00:33:56.724545+0000","last_active":"2024-07-29T00:36:30.030174+0000","last_peered":"2024-07-29T00:36:30.030174+0000","last_clean":"2024-07-29T00:36:30.030174+0000","last_became_active":"2024-07-29T00:33:56.724201+0000","last_became_peered":"2024-07-29T00:33:56.724201+0000","last_unstale":"2024-07-29T00:36:30.030174+0000","last_undegraded":"2024-07-29T00:36:30.030174+0000","last_fullsized":"2024-07-29T00:36:30.030174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:39:45.275284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039832+0000","last_change":"2024-07-29T00:33:56.705421+0000","last_active":"2024-07-29T00:36:30.039832+0000","last_peered":"2024-07-29T00:36:30.039832+0000","last_clean":"2024-07-29T00:36:30.039832+0000","last_became_active":"2024-07-29T00:33:56.704633+0000","last_became_peered":"2024-07-29T00:33:56.704633+0000","last_unstale":"2024-07-29T00:36:30.039832+0000","last_undegraded":"2024-07-29T00:36:30.039832+0000","last_fullsized":"2024-07-29T00:36:30.039832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:56:08.758429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978714+0000","last_change":"2024-07-29T00:34:50.713666+0000","last_active":"2024-07-29T00:36:29.978714+0000","last_peered":"2024-07-29T00:36:29.978714+0000","last_clean":"2024-07-29T00:36:29.978714+0000","last_became_active":"2024-07-29T00:34:50.713292+0000","last_became_peered":"2024-07-29T00:34:50.713292+0000","last_unstale":"2024-07-29T00:36:29.978714+0000","last_undegraded":"2024-07-29T00:36:29.978714+0000","last_fullsized":"2024-07-29T00:36:29.978714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:38:26.641753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006206+0000","last_change":"2024-07-29T00:33:56.728579+0000","last_active":"2024-07-29T00:36:30.006206+0000","last_peered":"2024-07-29T00:36:30.006206+0000","last_clean":"2024-07-29T00:36:30.006206+0000","last_became_active":"2024-07-29T00:33:56.724894+0000","last_became_peered":"2024-07-29T00:33:56.724894+0000","last_unstale":"2024-07-29T00:36:30.006206+0000","last_undegraded":"2024-07-29T00:36:30.006206+0000","last_fullsized":"2024-07-29T00:36:30.006206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:59:09.804862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918987+0000","last_change":"2024-07-29T00:33:56.715982+0000","last_active":"2024-07-29T00:36:29.918987+0000","last_peered":"2024-07-29T00:36:29.918987+0000","last_clean":"2024-07-29T00:36:29.918987+0000","last_became_active":"2024-07-29T00:33:56.715654+0000","last_became_peered":"2024-07-29T00:33:56.715654+0000","last_unstale":"2024-07-29T00:36:29.918987+0000","last_undegraded":"2024-07-29T00:36:29.918987+0000","last_fullsized":"2024-07-29T00:36:29.918987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:11.069292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.040027+0000","last_change":"2024-07-29T00:34:50.711612+0000","last_active":"2024-07-29T00:36:30.040027+0000","last_peered":"2024-07-29T00:36:30.040027+0000","last_clean":"2024-07-29T00:36:30.040027+0000","last_became_active":"2024-07-29T00:34:50.711175+0000","last_became_peered":"2024-07-29T00:34:50.711175+0000","last_unstale":"2024-07-29T00:36:30.040027+0000","last_undegraded":"2024-07-29T00:36:30.040027+0000","last_fullsized":"2024-07-29T00:36:30.040027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:09:37.841525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978875+0000","last_change":"2024-07-29T00:33:56.712960+0000","last_active":"2024-07-29T00:36:29.978875+0000","last_peered":"2024-07-29T00:36:29.978875+0000","last_clean":"2024-07-29T00:36:29.978875+0000","last_became_active":"2024-07-29T00:33:56.711975+0000","last_became_peered":"2024-07-29T00:33:56.711975+0000","last_unstale":"2024-07-29T00:36:29.978875+0000","last_undegraded":"2024-07-29T00:36:29.978875+0000","last_fullsized":"2024-07-29T00:36:29.978875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:22:34.046311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919501+0000","last_change":"2024-07-29T00:33:56.707254+0000","last_active":"2024-07-29T00:36:29.919501+0000","last_peered":"2024-07-29T00:36:29.919501+0000","last_clean":"2024-07-29T00:36:29.919501+0000","last_became_active":"2024-07-29T00:33:56.706888+0000","last_became_peered":"2024-07-29T00:33:56.706888+0000","last_unstale":"2024-07-29T00:36:29.919501+0000","last_undegraded":"2024-07-29T00:36:29.919501+0000","last_fullsized":"2024-07-29T00:36:29.919501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:46:38.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003149+0000","last_change":"2024-07-29T00:33:56.719619+0000","last_active":"2024-07-29T00:36:30.003149+0000","last_peered":"2024-07-29T00:36:30.003149+0000","last_clean":"2024-07-29T00:36:30.003149+0000","last_became_active":"2024-07-29T00:33:56.719312+0000","last_became_peered":"2024-07-29T00:33:56.719312+0000","last_unstale":"2024-07-29T00:36:30.003149+0000","last_undegraded":"2024-07-29T00:36:30.003149+0000","last_fullsized":"2024-07-29T00:36:30.003149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:46:50.516076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122517+0000","last_change":"2024-07-29T00:33:56.721607+0000","last_active":"2024-07-29T00:36:30.122517+0000","last_peered":"2024-07-29T00:36:30.122517+0000","last_clean":"2024-07-29T00:36:30.122517+0000","last_became_active":"2024-07-29T00:33:56.721230+0000","last_became_peered":"2024-07-29T00:33:56.721230+0000","last_unstale":"2024-07-29T00:36:30.122517+0000","last_undegraded":"2024-07-29T00:36:30.122517+0000","last_fullsized":"2024-07-29T00:36:30.122517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:52:05.294086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121450+0000","last_change":"2024-07-29T00:34:50.719821+0000","last_active":"2024-07-29T00:36:30.121450+0000","last_peered":"2024-07-29T00:36:30.121450+0000","last_clean":"2024-07-29T00:36:30.121450+0000","last_became_active":"2024-07-29T00:34:50.716015+0000","last_became_peered":"2024-07-29T00:34:50.716015+0000","last_unstale":"2024-07-29T00:36:30.121450+0000","last_undegraded":"2024-07-29T00:36:30.121450+0000","last_fullsized":"2024-07-29T00:36:30.121450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:36:02.382717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918029+0000","last_change":"2024-07-29T00:33:56.722086+0000","last_active":"2024-07-29T00:36:29.918029+0000","last_peered":"2024-07-29T00:36:29.918029+0000","last_clean":"2024-07-29T00:36:29.918029+0000","last_became_active":"2024-07-29T00:33:56.721700+0000","last_became_peered":"2024-07-29T00:33:56.721700+0000","last_unstale":"2024-07-29T00:36:29.918029+0000","last_undegraded":"2024-07-29T00:36:29.918029+0000","last_fullsized":"2024-07-29T00:36:29.918029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:04.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270846+0000","last_change":"2024-07-29T00:33:56.728225+0000","last_active":"2024-07-29T00:36:30.270846+0000","last_peered":"2024-07-29T00:36:30.270846+0000","last_clean":"2024-07-29T00:36:30.270846+0000","last_became_active":"2024-07-29T00:33:56.727854+0000","last_became_peered":"2024-07-29T00:33:56.727854+0000","last_unstale":"2024-07-29T00:36:30.270846+0000","last_undegraded":"2024-07-29T00:36:30.270846+0000","last_fullsized":"2024-07-29T00:36:30.270846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:35:51.715422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921670+0000","last_change":"2024-07-29T00:33:56.704401+0000","last_active":"2024-07-29T00:36:29.921670+0000","last_peered":"2024-07-29T00:36:29.921670+0000","last_clean":"2024-07-29T00:36:29.921670+0000","last_became_active":"2024-07-29T00:33:56.703853+0000","last_became_peered":"2024-07-29T00:33:56.703853+0000","last_unstale":"2024-07-29T00:36:29.921670+0000","last_undegraded":"2024-07-29T00:36:29.921670+0000","last_fullsized":"2024-07-29T00:36:29.921670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:26:25.233936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119558+0000","last_change":"2024-07-29T00:33:56.724457+0000","last_active":"2024-07-29T00:36:30.119558+0000","last_peered":"2024-07-29T00:36:30.119558+0000","last_clean":"2024-07-29T00:36:30.119558+0000","last_became_active":"2024-07-29T00:33:56.724099+0000","last_became_peered":"2024-07-29T00:33:56.724099+0000","last_unstale":"2024-07-29T00:36:30.119558+0000","last_undegraded":"2024-07-29T00:36:30.119558+0000","last_fullsized":"2024-07-29T00:36:30.119558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:01:48.812204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270504+0000","last_change":"2024-07-29T00:33:56.713672+0000","last_active":"2024-07-29T00:36:30.270504+0000","last_peered":"2024-07-29T00:36:30.270504+0000","last_clean":"2024-07-29T00:36:30.270504+0000","last_became_active":"2024-07-29T00:33:56.713305+0000","last_became_peered":"2024-07-29T00:33:56.713305+0000","last_unstale":"2024-07-29T00:36:30.270504+0000","last_undegraded":"2024-07-29T00:36:30.270504+0000","last_fullsized":"2024-07-29T00:36:30.270504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:36:07.130785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003134+0000","last_change":"2024-07-29T00:33:56.722241+0000","last_active":"2024-07-29T00:36:30.003134+0000","last_peered":"2024-07-29T00:36:30.003134+0000","last_clean":"2024-07-29T00:36:30.003134+0000","last_became_active":"2024-07-29T00:33:56.721861+0000","last_became_peered":"2024-07-29T00:33:56.721861+0000","last_unstale":"2024-07-29T00:36:30.003134+0000","last_undegraded":"2024-07-29T00:36:30.003134+0000","last_fullsized":"2024-07-29T00:36:30.003134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:09:33.634579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918840+0000","last_change":"2024-07-29T00:33:56.723437+0000","last_active":"2024-07-29T00:36:29.918840+0000","last_peered":"2024-07-29T00:36:29.918840+0000","last_clean":"2024-07-29T00:36:29.918840+0000","last_became_active":"2024-07-29T00:33:56.723018+0000","last_became_peered":"2024-07-29T00:33:56.723018+0000","last_unstale":"2024-07-29T00:36:29.918840+0000","last_undegraded":"2024-07-29T00:36:29.918840+0000","last_fullsized":"2024-07-29T00:36:29.918840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:36.718658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006207+0000","last_change":"2024-07-29T00:33:56.719656+0000","last_active":"2024-07-29T00:36:30.006207+0000","last_peered":"2024-07-29T00:36:30.006207+0000","last_clean":"2024-07-29T00:36:30.006207+0000","last_became_active":"2024-07-29T00:33:56.717866+0000","last_became_peered":"2024-07-29T00:33:56.717866+0000","last_unstale":"2024-07-29T00:36:30.006207+0000","last_undegraded":"2024-07-29T00:36:30.006207+0000","last_fullsized":"2024-07-29T00:36:30.006207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:46:18.711321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978362+0000","last_change":"2024-07-29T00:33:56.725648+0000","last_active":"2024-07-29T00:36:29.978362+0000","last_peered":"2024-07-29T00:36:29.978362+0000","last_clean":"2024-07-29T00:36:29.978362+0000","last_became_active":"2024-07-29T00:33:56.725377+0000","last_became_peered":"2024-07-29T00:33:56.725377+0000","last_unstale":"2024-07-29T00:36:29.978362+0000","last_undegraded":"2024-07-29T00:36:29.978362+0000","last_fullsized":"2024-07-29T00:36:29.978362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:13:27.443723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006816+0000","last_change":"2024-07-29T00:34:50.717173+0000","last_active":"2024-07-29T00:36:30.006816+0000","last_peered":"2024-07-29T00:36:30.006816+0000","last_clean":"2024-07-29T00:36:30.006816+0000","last_became_active":"2024-07-29T00:34:50.714795+0000","last_became_peered":"2024-07-29T00:34:50.714795+0000","last_unstale":"2024-07-29T00:36:30.006816+0000","last_undegraded":"2024-07-29T00:36:30.006816+0000","last_fullsized":"2024-07-29T00:36:30.006816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:13:08.373943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119656+0000","last_change":"2024-07-29T00:33:56.707858+0000","last_active":"2024-07-29T00:36:30.119656+0000","last_peered":"2024-07-29T00:36:30.119656+0000","last_clean":"2024-07-29T00:36:30.119656+0000","last_became_active":"2024-07-29T00:33:56.707535+0000","last_became_peered":"2024-07-29T00:33:56.707535+0000","last_unstale":"2024-07-29T00:36:30.119656+0000","last_undegraded":"2024-07-29T00:36:30.119656+0000","last_fullsized":"2024-07-29T00:36:30.119656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:57.358748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270796+0000","last_change":"2024-07-29T00:33:56.709874+0000","last_active":"2024-07-29T00:36:30.270796+0000","last_peered":"2024-07-29T00:36:30.270796+0000","last_clean":"2024-07-29T00:36:30.270796+0000","last_became_active":"2024-07-29T00:33:56.709482+0000","last_became_peered":"2024-07-29T00:33:56.709482+0000","last_unstale":"2024-07-29T00:36:30.270796+0000","last_undegraded":"2024-07-29T00:36:30.270796+0000","last_fullsized":"2024-07-29T00:36:30.270796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:03.066506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003369+0000","last_change":"2024-07-29T00:33:56.720394+0000","last_active":"2024-07-29T00:36:30.003369+0000","last_peered":"2024-07-29T00:36:30.003369+0000","last_clean":"2024-07-29T00:36:30.003369+0000","last_became_active":"2024-07-29T00:33:56.720037+0000","last_became_peered":"2024-07-29T00:33:56.720037+0000","last_unstale":"2024-07-29T00:36:30.003369+0000","last_undegraded":"2024-07-29T00:36:30.003369+0000","last_fullsized":"2024-07-29T00:36:30.003369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:31.729511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.121371+0000","last_change":"2024-07-29T00:33:56.715229+0000","last_active":"2024-07-29T00:36:30.121371+0000","last_peered":"2024-07-29T00:36:30.121371+0000","last_clean":"2024-07-29T00:36:30.121371+0000","last_became_active":"2024-07-29T00:33:56.714851+0000","last_became_peered":"2024-07-29T00:33:56.714851+0000","last_unstale":"2024-07-29T00:36:30.121371+0000","last_undegraded":"2024-07-29T00:36:30.121371+0000","last_fullsized":"2024-07-29T00:36:30.121371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:45.002591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119269+0000","last_change":"2024-07-29T00:33:56.728262+0000","last_active":"2024-07-29T00:36:30.119269+0000","last_peered":"2024-07-29T00:36:30.119269+0000","last_clean":"2024-07-29T00:36:30.119269+0000","last_became_active":"2024-07-29T00:33:56.727868+0000","last_became_peered":"2024-07-29T00:33:56.727868+0000","last_unstale":"2024-07-29T00:36:30.119269+0000","last_undegraded":"2024-07-29T00:36:30.119269+0000","last_fullsized":"2024-07-29T00:36:30.119269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:47:03.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119276+0000","last_change":"2024-07-29T00:33:56.710217+0000","last_active":"2024-07-29T00:36:30.119276+0000","last_peered":"2024-07-29T00:36:30.119276+0000","last_clean":"2024-07-29T00:36:30.119276+0000","last_became_active":"2024-07-29T00:33:56.709866+0000","last_became_peered":"2024-07-29T00:33:56.709866+0000","last_unstale":"2024-07-29T00:36:30.119276+0000","last_undegraded":"2024-07-29T00:36:30.119276+0000","last_fullsized":"2024-07-29T00:36:30.119276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:54:56.899183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270536+0000","last_change":"2024-07-29T00:33:56.710856+0000","last_active":"2024-07-29T00:36:30.270536+0000","last_peered":"2024-07-29T00:36:30.270536+0000","last_clean":"2024-07-29T00:36:30.270536+0000","last_became_active":"2024-07-29T00:33:56.710444+0000","last_became_peered":"2024-07-29T00:33:56.710444+0000","last_unstale":"2024-07-29T00:36:30.270536+0000","last_undegraded":"2024-07-29T00:36:30.270536+0000","last_fullsized":"2024-07-29T00:36:30.270536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:28:29.885759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029964+0000","last_change":"2024-07-29T00:33:56.714906+0000","last_active":"2024-07-29T00:36:30.029964+0000","last_peered":"2024-07-29T00:36:30.029964+0000","last_clean":"2024-07-29T00:36:30.029964+0000","last_became_active":"2024-07-29T00:33:56.714478+0000","last_became_peered":"2024-07-29T00:33:56.714478+0000","last_unstale":"2024-07-29T00:36:30.029964+0000","last_undegraded":"2024-07-29T00:36:30.029964+0000","last_fullsized":"2024-07-29T00:36:30.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:40:19.309661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":201,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:49.913498+0000","last_change":"2024-07-29T00:33:53.685228+0000","last_active":"2024-07-29T00:36:49.913498+0000","last_peered":"2024-07-29T00:36:49.913498+0000","last_clean":"2024-07-29T00:36:49.913498+0000","last_became_active":"2024-07-29T00:33:53.684877+0000","last_became_peered":"2024-07-29T00:33:53.684877+0000","last_unstale":"2024-07-29T00:36:49.913498+0000","last_undegraded":"2024-07-29T00:36:49.913498+0000","last_fullsized":"2024-07-29T00:36:49.913498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:08:31.353136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":161,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978592+0000","last_change":"2024-07-29T00:33:56.732448+0000","last_active":"2024-07-29T00:36:29.978592+0000","last_peered":"2024-07-29T00:36:29.978592+0000","last_clean":"2024-07-29T00:36:29.978592+0000","last_became_active":"2024-07-29T00:33:56.732107+0000","last_became_peered":"2024-07-29T00:33:56.732107+0000","last_unstale":"2024-07-29T00:36:29.978592+0000","last_undegraded":"2024-07-29T00:36:29.978592+0000","last_fullsized":"2024-07-29T00:36:29.978592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:43.445391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039762+0000","last_change":"2024-07-29T00:33:53.683924+0000","last_active":"2024-07-29T00:36:30.039762+0000","last_peered":"2024-07-29T00:36:30.039762+0000","last_clean":"2024-07-29T00:36:30.039762+0000","last_became_active":"2024-07-29T00:33:53.683478+0000","last_became_peered":"2024-07-29T00:33:53.683478+0000","last_unstale":"2024-07-29T00:36:30.039762+0000","last_undegraded":"2024-07-29T00:36:30.039762+0000","last_fullsized":"2024-07-29T00:36:30.039762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:27:48.588375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002737+0000","last_change":"2024-07-29T00:33:53.688557+0000","last_active":"2024-07-29T00:36:30.002737+0000","last_peered":"2024-07-29T00:36:30.002737+0000","last_clean":"2024-07-29T00:36:30.002737+0000","last_became_active":"2024-07-29T00:33:53.687991+0000","last_became_peered":"2024-07-29T00:33:53.687991+0000","last_unstale":"2024-07-29T00:36:30.002737+0000","last_undegraded":"2024-07-29T00:36:30.002737+0000","last_fullsized":"2024-07-29T00:36:30.002737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:56:21.136130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006358+0000","last_change":"2024-07-29T00:33:56.728845+0000","last_active":"2024-07-29T00:36:30.006358+0000","last_peered":"2024-07-29T00:36:30.006358+0000","last_clean":"2024-07-29T00:36:30.006358+0000","last_became_active":"2024-07-29T00:33:56.725550+0000","last_became_peered":"2024-07-29T00:33:56.725550+0000","last_unstale":"2024-07-29T00:36:30.006358+0000","last_undegraded":"2024-07-29T00:36:30.006358+0000","last_fullsized":"2024-07-29T00:36:30.006358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:27:36.215728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978697+0000","last_change":"2024-07-29T00:33:53.683328+0000","last_active":"2024-07-29T00:36:29.978697+0000","last_peered":"2024-07-29T00:36:29.978697+0000","last_clean":"2024-07-29T00:36:29.978697+0000","last_became_active":"2024-07-29T00:33:53.683005+0000","last_became_peered":"2024-07-29T00:33:53.683005+0000","last_unstale":"2024-07-29T00:36:29.978697+0000","last_undegraded":"2024-07-29T00:36:29.978697+0000","last_fullsized":"2024-07-29T00:36:29.978697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:40:01.836905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.920897+0000","last_change":"2024-07-29T00:33:56.724814+0000","last_active":"2024-07-29T00:36:29.920897+0000","last_peered":"2024-07-29T00:36:29.920897+0000","last_clean":"2024-07-29T00:36:29.920897+0000","last_became_active":"2024-07-29T00:33:56.724170+0000","last_became_peered":"2024-07-29T00:33:56.724170+0000","last_unstale":"2024-07-29T00:36:29.920897+0000","last_undegraded":"2024-07-29T00:36:29.920897+0000","last_fullsized":"2024-07-29T00:36:29.920897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:43:31.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":354,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:50.872171+0000","last_change":"2024-07-29T00:33:53.688134+0000","last_active":"2024-07-29T00:36:50.872171+0000","last_peered":"2024-07-29T00:36:50.872171+0000","last_clean":"2024-07-29T00:36:50.872171+0000","last_became_active":"2024-07-29T00:33:53.687738+0000","last_became_peered":"2024-07-29T00:33:53.687738+0000","last_unstale":"2024-07-29T00:36:50.872171+0000","last_undegraded":"2024-07-29T00:36:50.872171+0000","last_fullsized":"2024-07-29T00:36:50.872171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:50.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":315,"num_read_kb":315,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918684+0000","last_change":"2024-07-29T00:33:56.707493+0000","last_active":"2024-07-29T00:36:29.918684+0000","last_peered":"2024-07-29T00:36:29.918684+0000","last_clean":"2024-07-29T00:36:29.918684+0000","last_became_active":"2024-07-29T00:33:56.707181+0000","last_became_peered":"2024-07-29T00:33:56.707181+0000","last_unstale":"2024-07-29T00:36:29.918684+0000","last_undegraded":"2024-07-29T00:36:29.918684+0000","last_fullsized":"2024-07-29T00:36:29.918684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:58:05.729360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979141+0000","last_change":"2024-07-29T00:33:53.683035+0000","last_active":"2024-07-29T00:36:29.979141+0000","last_peered":"2024-07-29T00:36:29.979141+0000","last_clean":"2024-07-29T00:36:29.979141+0000","last_became_active":"2024-07-29T00:33:53.682623+0000","last_became_peered":"2024-07-29T00:33:53.682623+0000","last_unstale":"2024-07-29T00:36:29.979141+0000","last_undegraded":"2024-07-29T00:36:29.979141+0000","last_fullsized":"2024-07-29T00:36:29.979141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:06:09.827440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122906+0000","last_change":"2024-07-29T00:33:56.719005+0000","last_active":"2024-07-29T00:36:30.122906+0000","last_peered":"2024-07-29T00:36:30.122906+0000","last_clean":"2024-07-29T00:36:30.122906+0000","last_became_active":"2024-07-29T00:33:56.718615+0000","last_became_peered":"2024-07-29T00:33:56.718615+0000","last_unstale":"2024-07-29T00:36:30.122906+0000","last_undegraded":"2024-07-29T00:36:30.122906+0000","last_fullsized":"2024-07-29T00:36:30.122906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:49:15.696124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978649+0000","last_change":"2024-07-29T00:33:54.217732+0000","last_active":"2024-07-29T00:36:29.978649+0000","last_peered":"2024-07-29T00:36:29.978649+0000","last_clean":"2024-07-29T00:36:29.978649+0000","last_became_active":"2024-07-29T00:33:54.217234+0000","last_became_peered":"2024-07-29T00:33:54.217234+0000","last_unstale":"2024-07-29T00:36:29.978649+0000","last_undegraded":"2024-07-29T00:36:29.978649+0000","last_fullsized":"2024-07-29T00:36:29.978649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:45:31.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918482+0000","last_change":"2024-07-29T00:33:56.708355+0000","last_active":"2024-07-29T00:36:29.918482+0000","last_peered":"2024-07-29T00:36:29.918482+0000","last_clean":"2024-07-29T00:36:29.918482+0000","last_became_active":"2024-07-29T00:33:56.708002+0000","last_became_peered":"2024-07-29T00:33:56.708002+0000","last_unstale":"2024-07-29T00:36:29.918482+0000","last_undegraded":"2024-07-29T00:36:29.918482+0000","last_fullsized":"2024-07-29T00:36:29.918482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:57:47.823358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":418,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:50.876696+0000","last_change":"2024-07-29T00:34:50.716409+0000","last_active":"2024-07-29T00:36:50.876696+0000","last_peered":"2024-07-29T00:36:50.876696+0000","last_clean":"2024-07-29T00:36:50.876696+0000","last_became_active":"2024-07-29T00:34:50.706036+0000","last_became_peered":"2024-07-29T00:34:50.706036+0000","last_unstale":"2024-07-29T00:36:50.876696+0000","last_undegraded":"2024-07-29T00:36:50.876696+0000","last_fullsized":"2024-07-29T00:36:50.876696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:09:38.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66470,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":327,"num_read_kb":229,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59761,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030244+0000","last_change":"2024-07-29T00:33:56.710879+0000","last_active":"2024-07-29T00:36:30.030244+0000","last_peered":"2024-07-29T00:36:30.030244+0000","last_clean":"2024-07-29T00:36:30.030244+0000","last_became_active":"2024-07-29T00:33:56.710571+0000","last_became_peered":"2024-07-29T00:33:56.710571+0000","last_unstale":"2024-07-29T00:36:30.030244+0000","last_undegraded":"2024-07-29T00:36:30.030244+0000","last_fullsized":"2024-07-29T00:36:30.030244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:15:47.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119352+0000","last_change":"2024-07-29T00:33:53.687136+0000","last_active":"2024-07-29T00:36:30.119352+0000","last_peered":"2024-07-29T00:36:30.119352+0000","last_clean":"2024-07-29T00:36:30.119352+0000","last_became_active":"2024-07-29T00:33:53.686840+0000","last_became_peered":"2024-07-29T00:33:53.686840+0000","last_unstale":"2024-07-29T00:36:30.119352+0000","last_undegraded":"2024-07-29T00:36:30.119352+0000","last_fullsized":"2024-07-29T00:36:30.119352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:25:23.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002636+0000","last_change":"2024-07-29T00:33:56.723609+0000","last_active":"2024-07-29T00:36:30.002636+0000","last_peered":"2024-07-29T00:36:30.002636+0000","last_clean":"2024-07-29T00:36:30.002636+0000","last_became_active":"2024-07-29T00:33:56.723286+0000","last_became_peered":"2024-07-29T00:33:56.723286+0000","last_unstale":"2024-07-29T00:36:30.002636+0000","last_undegraded":"2024-07-29T00:36:30.002636+0000","last_fullsized":"2024-07-29T00:36:30.002636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:18:59.823410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978452+0000","last_change":"2024-07-29T00:33:53.682287+0000","last_active":"2024-07-29T00:36:29.978452+0000","last_peered":"2024-07-29T00:36:29.978452+0000","last_clean":"2024-07-29T00:36:29.978452+0000","last_became_active":"2024-07-29T00:33:53.681933+0000","last_became_peered":"2024-07-29T00:33:53.681933+0000","last_unstale":"2024-07-29T00:36:29.978452+0000","last_undegraded":"2024-07-29T00:36:29.978452+0000","last_fullsized":"2024-07-29T00:36:29.978452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:23:07.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978851+0000","last_change":"2024-07-29T00:33:56.714481+0000","last_active":"2024-07-29T00:36:29.978851+0000","last_peered":"2024-07-29T00:36:29.978851+0000","last_clean":"2024-07-29T00:36:29.978851+0000","last_became_active":"2024-07-29T00:33:56.714047+0000","last_became_peered":"2024-07-29T00:33:56.714047+0000","last_unstale":"2024-07-29T00:36:29.978851+0000","last_undegraded":"2024-07-29T00:36:29.978851+0000","last_fullsized":"2024-07-29T00:36:29.978851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:58:45.345599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":204,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:49.044982+0000","last_change":"2024-07-29T00:33:53.679770+0000","last_active":"2024-07-29T00:36:49.044982+0000","last_peered":"2024-07-29T00:36:49.044982+0000","last_clean":"2024-07-29T00:36:49.044982+0000","last_became_active":"2024-07-29T00:33:53.679376+0000","last_became_peered":"2024-07-29T00:33:53.679376+0000","last_unstale":"2024-07-29T00:36:49.044982+0000","last_undegraded":"2024-07-29T00:36:49.044982+0000","last_fullsized":"2024-07-29T00:36:49.044982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:16:57.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978642+0000","last_change":"2024-07-29T00:33:56.712331+0000","last_active":"2024-07-29T00:36:29.978642+0000","last_peered":"2024-07-29T00:36:29.978642+0000","last_clean":"2024-07-29T00:36:29.978642+0000","last_became_active":"2024-07-29T00:33:56.710050+0000","last_became_peered":"2024-07-29T00:33:56.710050+0000","last_unstale":"2024-07-29T00:36:29.978642+0000","last_undegraded":"2024-07-29T00:36:29.978642+0000","last_fullsized":"2024-07-29T00:36:29.978642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:35:25.198170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029821+0000","last_change":"2024-07-29T00:33:54.216782+0000","last_active":"2024-07-29T00:36:30.029821+0000","last_peered":"2024-07-29T00:36:30.029821+0000","last_clean":"2024-07-29T00:36:30.029821+0000","last_became_active":"2024-07-29T00:33:54.216362+0000","last_became_peered":"2024-07-29T00:33:54.216362+0000","last_unstale":"2024-07-29T00:36:30.029821+0000","last_undegraded":"2024-07-29T00:36:30.029821+0000","last_fullsized":"2024-07-29T00:36:30.029821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:06:26.357633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978327+0000","last_change":"2024-07-29T00:33:56.712865+0000","last_active":"2024-07-29T00:36:29.978327+0000","last_peered":"2024-07-29T00:36:29.978327+0000","last_clean":"2024-07-29T00:36:29.978327+0000","last_became_active":"2024-07-29T00:33:56.711384+0000","last_became_peered":"2024-07-29T00:33:56.711384+0000","last_unstale":"2024-07-29T00:36:29.978327+0000","last_undegraded":"2024-07-29T00:36:29.978327+0000","last_fullsized":"2024-07-29T00:36:29.978327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:53:08.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030159+0000","last_change":"2024-07-29T00:33:53.680118+0000","last_active":"2024-07-29T00:36:30.030159+0000","last_peered":"2024-07-29T00:36:30.030159+0000","last_clean":"2024-07-29T00:36:30.030159+0000","last_became_active":"2024-07-29T00:33:53.679774+0000","last_became_peered":"2024-07-29T00:33:53.679774+0000","last_unstale":"2024-07-29T00:36:30.030159+0000","last_undegraded":"2024-07-29T00:36:30.030159+0000","last_fullsized":"2024-07-29T00:36:30.030159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:31.281704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270695+0000","last_change":"2024-07-29T00:34:50.712384+0000","last_active":"2024-07-29T00:36:30.270695+0000","last_peered":"2024-07-29T00:36:30.270695+0000","last_clean":"2024-07-29T00:36:30.270695+0000","last_became_active":"2024-07-29T00:34:50.711418+0000","last_became_peered":"2024-07-29T00:34:50.711418+0000","last_unstale":"2024-07-29T00:36:30.270695+0000","last_undegraded":"2024-07-29T00:36:30.270695+0000","last_fullsized":"2024-07-29T00:36:30.270695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:03:22.729693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003638+0000","last_change":"2024-07-29T00:33:53.684758+0000","last_active":"2024-07-29T00:36:30.003638+0000","last_peered":"2024-07-29T00:36:30.003638+0000","last_clean":"2024-07-29T00:36:30.003638+0000","last_became_active":"2024-07-29T00:33:53.681455+0000","last_became_peered":"2024-07-29T00:33:53.681455+0000","last_unstale":"2024-07-29T00:36:30.003638+0000","last_undegraded":"2024-07-29T00:36:30.003638+0000","last_fullsized":"2024-07-29T00:36:30.003638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:10:46.067117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979263+0000","last_change":"2024-07-29T00:33:56.708157+0000","last_active":"2024-07-29T00:36:29.979263+0000","last_peered":"2024-07-29T00:36:29.979263+0000","last_clean":"2024-07-29T00:36:29.979263+0000","last_became_active":"2024-07-29T00:33:56.706454+0000","last_became_peered":"2024-07-29T00:33:56.706454+0000","last_unstale":"2024-07-29T00:36:29.979263+0000","last_undegraded":"2024-07-29T00:36:29.979263+0000","last_fullsized":"2024-07-29T00:36:29.979263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:23:19.695859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'67","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:44.952930+0000","last_change":"2024-07-29T00:33:53.681586+0000","last_active":"2024-07-29T00:36:44.952930+0000","last_peered":"2024-07-29T00:36:44.952930+0000","last_clean":"2024-07-29T00:36:44.952930+0000","last_became_active":"2024-07-29T00:33:53.681218+0000","last_became_peered":"2024-07-29T00:33:53.681218+0000","last_unstale":"2024-07-29T00:36:44.952930+0000","last_undegraded":"2024-07-29T00:36:44.952930+0000","last_fullsized":"2024-07-29T00:36:44.952930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:15:03.162786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978805+0000","last_change":"2024-07-29T00:33:56.729707+0000","last_active":"2024-07-29T00:36:29.978805+0000","last_peered":"2024-07-29T00:36:29.978805+0000","last_clean":"2024-07-29T00:36:29.978805+0000","last_became_active":"2024-07-29T00:33:56.729381+0000","last_became_peered":"2024-07-29T00:33:56.729381+0000","last_unstale":"2024-07-29T00:36:29.978805+0000","last_undegraded":"2024-07-29T00:36:29.978805+0000","last_fullsized":"2024-07-29T00:36:29.978805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:08:53.208565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:49.871146+0000","last_change":"2024-07-29T00:34:50.727930+0000","last_active":"2024-07-29T00:36:49.871146+0000","last_peered":"2024-07-29T00:36:49.871146+0000","last_clean":"2024-07-29T00:36:49.871146+0000","last_became_active":"2024-07-29T00:34:50.722598+0000","last_became_peered":"2024-07-29T00:34:50.722598+0000","last_unstale":"2024-07-29T00:36:49.871146+0000","last_undegraded":"2024-07-29T00:36:49.871146+0000","last_fullsized":"2024-07-29T00:36:49.871146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:38:09.356005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.979102+0000","last_change":"2024-07-29T00:33:56.712602+0000","last_active":"2024-07-29T00:36:29.979102+0000","last_peered":"2024-07-29T00:36:29.979102+0000","last_clean":"2024-07-29T00:36:29.979102+0000","last_became_active":"2024-07-29T00:33:56.711483+0000","last_became_peered":"2024-07-29T00:33:56.711483+0000","last_unstale":"2024-07-29T00:36:29.979102+0000","last_undegraded":"2024-07-29T00:36:29.979102+0000","last_fullsized":"2024-07-29T00:36:29.979102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:49:14.987687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":408,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:32.081815+0000","last_change":"2024-07-29T00:32:34.451436+0000","last_active":"2024-07-29T00:36:32.081815+0000","last_peered":"2024-07-29T00:36:32.081815+0000","last_clean":"2024-07-29T00:36:32.081815+0000","last_became_active":"2024-07-29T00:32:34.450890+0000","last_became_peered":"2024-07-29T00:32:34.450890+0000","last_unstale":"2024-07-29T00:36:32.081815+0000","last_undegraded":"2024-07-29T00:36:32.081815+0000","last_fullsized":"2024-07-29T00:36:32.081815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:29:04.052984+0000","last_clean_scrub_stamp":"2024-07-29T00:29:04.052984+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T05:28:17.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918549+0000","last_change":"2024-07-29T00:33:53.690474+0000","last_active":"2024-07-29T00:36:29.918549+0000","last_peered":"2024-07-29T00:36:29.918549+0000","last_clean":"2024-07-29T00:36:29.918549+0000","last_became_active":"2024-07-29T00:33:53.690111+0000","last_became_peered":"2024-07-29T00:33:53.690111+0000","last_unstale":"2024-07-29T00:36:29.918549+0000","last_undegraded":"2024-07-29T00:36:29.918549+0000","last_fullsized":"2024-07-29T00:36:29.918549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:04:36.685174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006458+0000","last_change":"2024-07-29T00:33:56.721381+0000","last_active":"2024-07-29T00:36:30.006458+0000","last_peered":"2024-07-29T00:36:30.006458+0000","last_clean":"2024-07-29T00:36:30.006458+0000","last_became_active":"2024-07-29T00:33:56.720808+0000","last_became_peered":"2024-07-29T00:33:56.720808+0000","last_unstale":"2024-07-29T00:36:30.006458+0000","last_undegraded":"2024-07-29T00:36:30.006458+0000","last_fullsized":"2024-07-29T00:36:30.006458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:37:44.963807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270459+0000","last_change":"2024-07-29T00:35:51.676982+0000","last_active":"2024-07-29T00:36:30.270459+0000","last_peered":"2024-07-29T00:36:30.270459+0000","last_clean":"2024-07-29T00:36:30.270459+0000","last_became_active":"2024-07-29T00:35:51.674179+0000","last_became_peered":"2024-07-29T00:35:51.674179+0000","last_unstale":"2024-07-29T00:36:30.270459+0000","last_undegraded":"2024-07-29T00:36:30.270459+0000","last_fullsized":"2024-07-29T00:36:30.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T04:24:23.211314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.002590+0000","last_change":"2024-07-29T00:33:56.711439+0000","last_active":"2024-07-29T00:36:30.002590+0000","last_peered":"2024-07-29T00:36:30.002590+0000","last_clean":"2024-07-29T00:36:30.002590+0000","last_became_active":"2024-07-29T00:33:56.710229+0000","last_became_peered":"2024-07-29T00:33:56.710229+0000","last_unstale":"2024-07-29T00:36:30.002590+0000","last_undegraded":"2024-07-29T00:36:30.002590+0000","last_fullsized":"2024-07-29T00:36:30.002590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:01:57.816313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":198,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:49.043991+0000","last_change":"2024-07-29T00:33:54.216267+0000","last_active":"2024-07-29T00:36:49.043991+0000","last_peered":"2024-07-29T00:36:49.043991+0000","last_clean":"2024-07-29T00:36:49.043991+0000","last_became_active":"2024-07-29T00:33:54.215184+0000","last_became_peered":"2024-07-29T00:33:54.215184+0000","last_unstale":"2024-07-29T00:36:49.043991+0000","last_undegraded":"2024-07-29T00:36:49.043991+0000","last_fullsized":"2024-07-29T00:36:49.043991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:38:12.575208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":158,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978450+0000","last_change":"2024-07-29T00:34:50.722534+0000","last_active":"2024-07-29T00:36:29.978450+0000","last_peered":"2024-07-29T00:36:29.978450+0000","last_clean":"2024-07-29T00:36:29.978450+0000","last_became_active":"2024-07-29T00:34:50.722116+0000","last_became_peered":"2024-07-29T00:34:50.722116+0000","last_unstale":"2024-07-29T00:36:29.978450+0000","last_undegraded":"2024-07-29T00:36:29.978450+0000","last_fullsized":"2024-07-29T00:36:29.978450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:05:42.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":198,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:49.015604+0000","last_change":"2024-07-29T00:33:53.680658+0000","last_active":"2024-07-29T00:36:49.015604+0000","last_peered":"2024-07-29T00:36:49.015604+0000","last_clean":"2024-07-29T00:36:49.015604+0000","last_became_active":"2024-07-29T00:33:53.679754+0000","last_became_peered":"2024-07-29T00:33:53.679754+0000","last_unstale":"2024-07-29T00:36:49.015604+0000","last_undegraded":"2024-07-29T00:36:49.015604+0000","last_fullsized":"2024-07-29T00:36:49.015604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:24:03.349203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":895,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":159,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039767+0000","last_change":"2024-07-29T00:33:56.719245+0000","last_active":"2024-07-29T00:36:30.039767+0000","last_peered":"2024-07-29T00:36:30.039767+0000","last_clean":"2024-07-29T00:36:30.039767+0000","last_became_active":"2024-07-29T00:33:56.718781+0000","last_became_peered":"2024-07-29T00:33:56.718781+0000","last_unstale":"2024-07-29T00:36:30.039767+0000","last_undegraded":"2024-07-29T00:36:30.039767+0000","last_fullsized":"2024-07-29T00:36:30.039767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:49:44.115895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921376+0000","last_change":"2024-07-29T00:33:54.219697+0000","last_active":"2024-07-29T00:36:29.921376+0000","last_peered":"2024-07-29T00:36:29.921376+0000","last_clean":"2024-07-29T00:36:29.921376+0000","last_became_active":"2024-07-29T00:33:54.218846+0000","last_became_peered":"2024-07-29T00:33:54.218846+0000","last_unstale":"2024-07-29T00:36:29.921376+0000","last_undegraded":"2024-07-29T00:36:29.921376+0000","last_fullsized":"2024-07-29T00:36:29.921376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:03:57.919245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.039877+0000","last_change":"2024-07-29T00:33:56.723991+0000","last_active":"2024-07-29T00:36:30.039877+0000","last_peered":"2024-07-29T00:36:30.039877+0000","last_clean":"2024-07-29T00:36:30.039877+0000","last_became_active":"2024-07-29T00:33:56.723593+0000","last_became_peered":"2024-07-29T00:33:56.723593+0000","last_unstale":"2024-07-29T00:36:30.039877+0000","last_undegraded":"2024-07-29T00:36:30.039877+0000","last_fullsized":"2024-07-29T00:36:30.039877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:55:16.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030289+0000","last_change":"2024-07-29T00:33:54.216727+0000","last_active":"2024-07-29T00:36:30.030289+0000","last_peered":"2024-07-29T00:36:30.030289+0000","last_clean":"2024-07-29T00:36:30.030289+0000","last_became_active":"2024-07-29T00:33:54.216285+0000","last_became_peered":"2024-07-29T00:33:54.216285+0000","last_unstale":"2024-07-29T00:36:30.030289+0000","last_undegraded":"2024-07-29T00:36:30.030289+0000","last_fullsized":"2024-07-29T00:36:30.030289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:07:51.456588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.003485+0000","last_change":"2024-07-29T00:33:56.710098+0000","last_active":"2024-07-29T00:36:30.003485+0000","last_peered":"2024-07-29T00:36:30.003485+0000","last_clean":"2024-07-29T00:36:30.003485+0000","last_became_active":"2024-07-29T00:33:56.709645+0000","last_became_peered":"2024-07-29T00:33:56.709645+0000","last_unstale":"2024-07-29T00:36:30.003485+0000","last_undegraded":"2024-07-29T00:36:30.003485+0000","last_fullsized":"2024-07-29T00:36:30.003485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:22:04.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'105","reported_seq":230,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:45.432574+0000","last_change":"2024-07-29T00:33:53.688681+0000","last_active":"2024-07-29T00:36:45.432574+0000","last_peered":"2024-07-29T00:36:45.432574+0000","last_clean":"2024-07-29T00:36:45.432574+0000","last_became_active":"2024-07-29T00:33:53.688157+0000","last_became_peered":"2024-07-29T00:33:53.688157+0000","last_unstale":"2024-07-29T00:36:45.432574+0000","last_undegraded":"2024-07-29T00:36:45.432574+0000","last_fullsized":"2024-07-29T00:36:45.432574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:58:42.733736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006932+0000","last_change":"2024-07-29T00:33:56.720702+0000","last_active":"2024-07-29T00:36:30.006932+0000","last_peered":"2024-07-29T00:36:30.006932+0000","last_clean":"2024-07-29T00:36:30.006932+0000","last_became_active":"2024-07-29T00:33:56.718389+0000","last_became_peered":"2024-07-29T00:33:56.718389+0000","last_unstale":"2024-07-29T00:36:30.006932+0000","last_undegraded":"2024-07-29T00:36:30.006932+0000","last_fullsized":"2024-07-29T00:36:30.006932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T07:46:39.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.919572+0000","last_change":"2024-07-29T00:33:53.872914+0000","last_active":"2024-07-29T00:36:29.919572+0000","last_peered":"2024-07-29T00:36:29.919572+0000","last_clean":"2024-07-29T00:36:29.919572+0000","last_became_active":"2024-07-29T00:33:53.872608+0000","last_became_peered":"2024-07-29T00:33:53.872608+0000","last_unstale":"2024-07-29T00:36:29.919572+0000","last_undegraded":"2024-07-29T00:36:29.919572+0000","last_fullsized":"2024-07-29T00:36:29.919572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T09:36:14.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122292+0000","last_change":"2024-07-29T00:33:56.733100+0000","last_active":"2024-07-29T00:36:30.122292+0000","last_peered":"2024-07-29T00:36:30.122292+0000","last_clean":"2024-07-29T00:36:30.122292+0000","last_became_active":"2024-07-29T00:33:56.732731+0000","last_became_peered":"2024-07-29T00:33:56.732731+0000","last_unstale":"2024-07-29T00:36:30.122292+0000","last_undegraded":"2024-07-29T00:36:30.122292+0000","last_fullsized":"2024-07-29T00:36:30.122292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T12:24:45.465413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918120+0000","last_change":"2024-07-29T00:33:54.214044+0000","last_active":"2024-07-29T00:36:29.918120+0000","last_peered":"2024-07-29T00:36:29.918120+0000","last_clean":"2024-07-29T00:36:29.918120+0000","last_became_active":"2024-07-29T00:33:54.213699+0000","last_became_peered":"2024-07-29T00:33:54.213699+0000","last_unstale":"2024-07-29T00:36:29.918120+0000","last_undegraded":"2024-07-29T00:36:29.918120+0000","last_fullsized":"2024-07-29T00:36:29.918120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T10:46:10.793188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029897+0000","last_change":"2024-07-29T00:33:56.707850+0000","last_active":"2024-07-29T00:36:30.029897+0000","last_peered":"2024-07-29T00:36:30.029897+0000","last_clean":"2024-07-29T00:36:30.029897+0000","last_became_active":"2024-07-29T00:33:56.707447+0000","last_became_peered":"2024-07-29T00:33:56.707447+0000","last_unstale":"2024-07-29T00:36:30.029897+0000","last_undegraded":"2024-07-29T00:36:30.029897+0000","last_fullsized":"2024-07-29T00:36:30.029897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T08:30:09.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006317+0000","last_change":"2024-07-29T00:33:53.685803+0000","last_active":"2024-07-29T00:36:30.006317+0000","last_peered":"2024-07-29T00:36:30.006317+0000","last_clean":"2024-07-29T00:36:30.006317+0000","last_became_active":"2024-07-29T00:33:53.685304+0000","last_became_peered":"2024-07-29T00:33:53.685304+0000","last_unstale":"2024-07-29T00:36:30.006317+0000","last_undegraded":"2024-07-29T00:36:30.006317+0000","last_fullsized":"2024-07-29T00:36:30.006317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:28.407406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270586+0000","last_change":"2024-07-29T00:33:56.728424+0000","last_active":"2024-07-29T00:36:30.270586+0000","last_peered":"2024-07-29T00:36:30.270586+0000","last_clean":"2024-07-29T00:36:30.270586+0000","last_became_active":"2024-07-29T00:33:56.727923+0000","last_became_peered":"2024-07-29T00:33:56.727923+0000","last_unstale":"2024-07-29T00:36:30.270586+0000","last_undegraded":"2024-07-29T00:36:30.270586+0000","last_fullsized":"2024-07-29T00:36:30.270586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:29:55.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.119490+0000","last_change":"2024-07-29T00:33:53.683274+0000","last_active":"2024-07-29T00:36:30.119490+0000","last_peered":"2024-07-29T00:36:30.119490+0000","last_clean":"2024-07-29T00:36:30.119490+0000","last_became_active":"2024-07-29T00:33:53.682919+0000","last_became_peered":"2024-07-29T00:33:53.682919+0000","last_unstale":"2024-07-29T00:36:30.119490+0000","last_undegraded":"2024-07-29T00:36:30.119490+0000","last_fullsized":"2024-07-29T00:36:30.119490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T00:47:10.295819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006281+0000","last_change":"2024-07-29T00:33:56.732829+0000","last_active":"2024-07-29T00:36:30.006281+0000","last_peered":"2024-07-29T00:36:30.006281+0000","last_clean":"2024-07-29T00:36:30.006281+0000","last_became_active":"2024-07-29T00:33:56.732460+0000","last_became_peered":"2024-07-29T00:33:56.732460+0000","last_unstale":"2024-07-29T00:36:30.006281+0000","last_undegraded":"2024-07-29T00:36:30.006281+0000","last_fullsized":"2024-07-29T00:36:30.006281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T06:57:06.941697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.029962+0000","last_change":"2024-07-29T00:33:53.683132+0000","last_active":"2024-07-29T00:36:30.029962+0000","last_peered":"2024-07-29T00:36:30.029962+0000","last_clean":"2024-07-29T00:36:30.029962+0000","last_became_active":"2024-07-29T00:33:53.682690+0000","last_became_peered":"2024-07-29T00:33:53.682690+0000","last_unstale":"2024-07-29T00:36:30.029962+0000","last_undegraded":"2024-07-29T00:36:30.029962+0000","last_fullsized":"2024-07-29T00:36:30.029962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:37:16.500002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978784+0000","last_change":"2024-07-29T00:33:56.713510+0000","last_active":"2024-07-29T00:36:29.978784+0000","last_peered":"2024-07-29T00:36:29.978784+0000","last_clean":"2024-07-29T00:36:29.978784+0000","last_became_active":"2024-07-29T00:33:56.713177+0000","last_became_peered":"2024-07-29T00:33:56.713177+0000","last_unstale":"2024-07-29T00:36:29.978784+0000","last_undegraded":"2024-07-29T00:36:29.978784+0000","last_fullsized":"2024-07-29T00:36:29.978784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:58:28.882956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.978667+0000","last_change":"2024-07-29T00:33:56.716157+0000","last_active":"2024-07-29T00:36:29.978667+0000","last_peered":"2024-07-29T00:36:29.978667+0000","last_clean":"2024-07-29T00:36:29.978667+0000","last_became_active":"2024-07-29T00:33:56.715414+0000","last_became_peered":"2024-07-29T00:33:56.715414+0000","last_unstale":"2024-07-29T00:36:29.978667+0000","last_undegraded":"2024-07-29T00:36:29.978667+0000","last_fullsized":"2024-07-29T00:36:29.978667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T03:55:49.270397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.030262+0000","last_change":"2024-07-29T00:33:53.679296+0000","last_active":"2024-07-29T00:36:30.030262+0000","last_peered":"2024-07-29T00:36:30.030262+0000","last_clean":"2024-07-29T00:36:30.030262+0000","last_became_active":"2024-07-29T00:33:53.678940+0000","last_became_peered":"2024-07-29T00:33:53.678940+0000","last_unstale":"2024-07-29T00:36:30.030262+0000","last_undegraded":"2024-07-29T00:36:30.030262+0000","last_fullsized":"2024-07-29T00:36:30.030262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T01:11:36.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.918288+0000","last_change":"2024-07-29T00:33:56.711660+0000","last_active":"2024-07-29T00:36:29.918288+0000","last_peered":"2024-07-29T00:36:29.918288+0000","last_clean":"2024-07-29T00:36:29.918288+0000","last_became_active":"2024-07-29T00:33:56.711244+0000","last_became_peered":"2024-07-29T00:33:56.711244+0000","last_unstale":"2024-07-29T00:36:29.918288+0000","last_undegraded":"2024-07-29T00:36:29.918288+0000","last_fullsized":"2024-07-29T00:36:29.918288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:14:03.810909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.270633+0000","last_change":"2024-07-29T00:34:50.720521+0000","last_active":"2024-07-29T00:36:30.270633+0000","last_peered":"2024-07-29T00:36:30.270633+0000","last_clean":"2024-07-29T00:36:30.270633+0000","last_became_active":"2024-07-29T00:34:50.707332+0000","last_became_peered":"2024-07-29T00:34:50.707332+0000","last_unstale":"2024-07-29T00:36:30.270633+0000","last_undegraded":"2024-07-29T00:36:30.270633+0000","last_fullsized":"2024-07-29T00:36:30.270633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:04:39.366461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.122784+0000","last_change":"2024-07-29T00:33:56.728755+0000","last_active":"2024-07-29T00:36:30.122784+0000","last_peered":"2024-07-29T00:36:30.122784+0000","last_clean":"2024-07-29T00:36:30.122784+0000","last_became_active":"2024-07-29T00:33:56.727203+0000","last_became_peered":"2024-07-29T00:33:56.727203+0000","last_unstale":"2024-07-29T00:36:30.122784+0000","last_undegraded":"2024-07-29T00:36:30.122784+0000","last_fullsized":"2024-07-29T00:36:30.122784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:53:40.151169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:30.006887+0000","last_change":"2024-07-29T00:33:54.217669+0000","last_active":"2024-07-29T00:36:30.006887+0000","last_peered":"2024-07-29T00:36:30.006887+0000","last_clean":"2024-07-29T00:36:30.006887+0000","last_became_active":"2024-07-29T00:33:54.216367+0000","last_became_peered":"2024-07-29T00:33:54.216367+0000","last_unstale":"2024-07-29T00:36:30.006887+0000","last_undegraded":"2024-07-29T00:36:30.006887+0000","last_fullsized":"2024-07-29T00:36:30.006887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T11:36:39.031663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:29.921570+0000","last_change":"2024-07-29T00:34:50.710057+0000","last_active":"2024-07-29T00:36:29.921570+0000","last_peered":"2024-07-29T00:36:29.921570+0000","last_clean":"2024-07-29T00:36:29.921570+0000","last_became_active":"2024-07-29T00:34:50.709079+0000","last_became_peered":"2024-07-29T00:34:50.709079+0000","last_unstale":"2024-07-29T00:36:29.921570+0000","last_undegraded":"2024-07-29T00:36:29.921570+0000","last_fullsized":"2024-07-29T00:36:29.921570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:55.675062+0000","last_clean_scrub_stamp":"2024-07-29T00:33:55.675062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:13:10.263990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":206,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-29T00:36:49.014662+0000","last_change":"2024-07-29T00:33:53.687608+0000","last_active":"2024-07-29T00:36:49.014662+0000","last_peered":"2024-07-29T00:36:49.014662+0000","last_clean":"2024-07-29T00:36:49.014662+0000","last_became_active":"2024-07-29T00:33:53.686858+0000","last_became_peered":"2024-07-29T00:33:53.686858+0000","last_unstale":"2024-07-29T00:36:49.014662+0000","last_undegraded":"2024-07-29T00:36:49.014662+0000","last_fullsized":"2024-07-29T00:36:49.014662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-29T00:33:52.638247+0000","last_clean_scrub_stamp":"2024-07-29T00:33:52.638247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-30T02:49:29.237287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":159,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797518,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1927,"num_read_kb":1200,"num_write":505,"num_write_kb":2381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":60931,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2392554,"data_compressed":113050,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":384,"ondisk_log_size":384,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107544,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416433,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":501919,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678069,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716936,"statfs":{"total":95995035648,"available":95966142464,"internally_reserved":0,"allocated":1220608,"data_stored":1488181,"data_compressed":65511,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841595,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":894285,"data_compressed":33748,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":894663,"data_compressed":33748,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1095972,"data_compressed":50309,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":501093,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":1422264,"data_compressed":65511,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":570378,"data_compressed":24044,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182488,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1222059,"data_compressed":48950,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":501529,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":568476,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":502905,"data_compressed":14461,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34489,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":70431,"data_compressed":9583,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331832,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":397749,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-29T00:36:53.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:36:53.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:53 smithi175 ceph-mon[32138]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-07-29T00:36:53.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:53 smithi175 ceph-mon[32138]: from='client.25330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:53 smithi181 ceph-mon[36706]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-07-29T00:36:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:53 smithi181 ceph-mon[36706]: from='client.25330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:53.804 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:53.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:53 smithi204 ceph-mon[37368]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-07-29T00:36:53.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:53 smithi204 ceph-mon[37368]: from='client.25330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-29T00:36:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.460+0000 7fe662a00640 1 -- 172.21.15.175:0/3650968072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c108670 msgr2=0x7fe65c114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.460+0000 7fe662a00640 1 --2- 172.21.15.175:0/3650968072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c108670 0x7fe65c114b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe654009cd0 tx=0x7fe65402a8f0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.460+0000 7fe662a00640 1 -- 172.21.15.175:0/3650968072 shutdown_connections 2024-07-29T00:36:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.460+0000 7fe662a00640 1 --2- 172.21.15.175:0/3650968072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c108670 0x7fe65c114b90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.461+0000 7fe662a00640 1 --2- 172.21.15.175:0/3650968072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c107d10 0x7fe65c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.461+0000 7fe662a00640 1 --2- 172.21.15.175:0/3650968072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c10dd50 0x7fe65c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.461+0000 7fe662a00640 1 -- 172.21.15.175:0/3650968072 >> 172.21.15.175:0/3650968072 conn(0x7fe65c077670 msgr2=0x7fe65c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.461+0000 7fe662a00640 1 -- 172.21.15.175:0/3650968072 shutdown_connections 2024-07-29T00:36:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.461+0000 7fe662a00640 1 -- 172.21.15.175:0/3650968072 wait complete. 2024-07-29T00:36:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.461+0000 7fe662a00640 1 Processor -- start 2024-07-29T00:36:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 -- start start 2024-07-29T00:36:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c107d10 0x7fe65c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c108670 0x7fe65c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c10dd50 0x7fe65c1a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1180b0 con 0x7fe65c107d10 2024-07-29T00:36:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c117f30 con 0x7fe65c108670 2024-07-29T00:36:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe662a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c118230 con 0x7fe65c10dd50 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.462+0000 7fe661600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c107d10 0x7fe65c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe661600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c107d10 0x7fe65c19e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38462/0 (socket says 172.21.15.175:38462) 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c108670 0x7fe65c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe661600640 1 -- 172.21.15.175:0/2888189372 learned_addr learned my addr 172.21.15.175:0/2888189372 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe662000640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c10dd50 0x7fe65c1a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 -- 172.21.15.175:0/2888189372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c10dd50 msgr2=0x7fe65c1a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c10dd50 0x7fe65c1a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 -- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c107d10 msgr2=0x7fe65c19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c107d10 0x7fe65c19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 -- 172.21.15.175:0/2888189372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1a34d0 con 0x7fe65c108670 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.463+0000 7fe660c00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c108670 0x7fe65c19ea60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e9f0 tx=0x7fe64c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.464+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00ccc0 con 0x7fe65c108670 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.464+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64c010910 con 0x7fe65c108670 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.464+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c1a37c0 con 0x7fe65c108670 2024-07-29T00:36:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.464+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c018960 con 0x7fe65c108670 2024-07-29T00:36:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.464+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1ab0a0 con 0x7fe65c108670 2024-07-29T00:36:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.465+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c108130 con 0x7fe65c108670 2024-07-29T00:36:54.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.467+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7fe64c010ac0 con 0x7fe65c108670 2024-07-29T00:36:54.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.468+0000 7fe65a000640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe6440a2e00 0x7fe6440a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:54.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.469+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe64c0cfe10 con 0x7fe65c108670 2024-07-29T00:36:54.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.469+0000 7fe661600640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe6440a2e00 0x7fe6440a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:54.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.469+0000 7fe661600640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe6440a2e00 0x7fe6440a52c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe650009c10 tx=0x7fe650005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:54.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.470+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe64c09c1f0 con 0x7fe65c108670 2024-07-29T00:36:54.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.680+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe65c19f880 con 0x7fe65c108670 2024-07-29T00:36:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.681+0000 7fe65a000640 1 -- 172.21.15.175:0/2888189372 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7fe65c19f880 con 0x7fe65c108670 2024-07-29T00:36:54.683 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:54.683 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:36:54.683 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:36:54.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.684+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe6440a2e00 msgr2=0x7fe6440a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:54.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.684+0000 7fe662a00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe6440a2e00 0x7fe6440a52c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe650009c10 tx=0x7fe650005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:54.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c108670 msgr2=0x7fe65c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:54.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c108670 0x7fe65c19ea60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e9f0 tx=0x7fe64c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 shutdown_connections 2024-07-29T00:36:54.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c10dd50 0x7fe65c1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c108670 0x7fe65c19ea60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe6440a2e00 0x7fe6440a52c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 --2- 172.21.15.175:0/2888189372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe65c107d10 0x7fe65c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:54.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 >> 172.21.15.175:0/2888189372 conn(0x7fe65c077670 msgr2=0x7fe65c078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:54.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 shutdown_connections 2024-07-29T00:36:54.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:54.685+0000 7fe662a00640 1 -- 172.21.15.175:0/2888189372 wait complete. 2024-07-29T00:36:55.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:36:55.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:55 smithi181 ceph-mon[36706]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-07-29T00:36:55.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:55 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2888189372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:55.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:55 smithi175 ceph-mon[32138]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-07-29T00:36:55.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:55 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2888189372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:55 smithi204 ceph-mon[37368]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-07-29T00:36:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:55 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2888189372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:56.154 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:56.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 -- 172.21.15.175:0/1778947793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010d220 msgr2=0x7f5a30114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 --2- 172.21.15.175:0/1778947793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010d220 0x7f5a30114da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009ca0 tx=0x7f5a2802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 -- 172.21.15.175:0/1778947793 shutdown_connections 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 --2- 172.21.15.175:0/1778947793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3010d220 0x7f5a30114da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 --2- 172.21.15.175:0/1778947793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a30108af0 0x7f5a3010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 --2- 172.21.15.175:0/1778947793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108140 0x7f5a30108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 -- 172.21.15.175:0/1778947793 >> 172.21.15.175:0/1778947793 conn(0x7f5a30077890 msgr2=0x7f5a30079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 -- 172.21.15.175:0/1778947793 shutdown_connections 2024-07-29T00:36:56.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.832+0000 7f5a36000640 1 -- 172.21.15.175:0/1778947793 wait complete. 2024-07-29T00:36:56.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.833+0000 7f5a36000640 1 Processor -- start 2024-07-29T00:36:56.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.833+0000 7f5a36000640 1 -- start start 2024-07-29T00:36:56.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.833+0000 7f5a36000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 0x7f5a3019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a36000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108af0 0x7f5a3019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a36000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3010d220 0x7f5a301a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a36000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30118280 con 0x7f5a3010d220 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a36000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30118100 con 0x7f5a30108140 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a36000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30118400 con 0x7f5a30108af0 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 0x7f5a3019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 0x7f5a3019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48466/0 (socket says 172.21.15.175:48466) 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108af0 0x7f5a3019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 -- 172.21.15.175:0/3788754605 learned_addr learned my addr 172.21.15.175:0/3788754605 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2fe00640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3010d220 0x7f5a301a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:56.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 -- 172.21.15.175:0/3788754605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108af0 msgr2=0x7f5a3019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108af0 0x7f5a3019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 -- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3010d220 msgr2=0x7f5a301a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3010d220 0x7f5a301a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2f400640 1 -- 172.21.15.175:0/3788754605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a301a3640 con 0x7f5a30108140 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a27e00640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108af0 0x7f5a3019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.834+0000 7f5a2fe00640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3010d220 0x7f5a301a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.835+0000 7f5a2f400640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 0x7f5a3019e690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000e9f0 tx=0x7f5a2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.835+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000cd80 con 0x7f5a30108140 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.835+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a20010910 con 0x7f5a30108140 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.835+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20018990 con 0x7f5a30108140 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.835+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a301a3930 con 0x7f5a30108140 2024-07-29T00:36:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.835+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301ab210 con 0x7f5a30108140 2024-07-29T00:36:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.836+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a30108520 con 0x7f5a30108140 2024-07-29T00:36:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.838+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f5a20010ac0 con 0x7f5a30108140 2024-07-29T00:36:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.840+0000 7f5a2cc00640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5a140a2e00 0x7f5a140a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.840+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f5a200cf990 con 0x7f5a30108140 2024-07-29T00:36:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.840+0000 7f5a27e00640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5a140a2e00 0x7f5a140a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.841+0000 7f5a27e00640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5a140a2e00 0x7f5a140a52c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a30107bd0 tx=0x7f5a18005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:56.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:56.848+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5a2009c3c0 con 0x7f5a30108140 2024-07-29T00:36:57.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.077+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a301ab480 con 0x7f5a30108140 2024-07-29T00:36:57.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.078+0000 7f5a2cc00640 1 -- 172.21.15.175:0/3788754605 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f5a2009bcc0 con 0x7f5a30108140 2024-07-29T00:36:57.077 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:57.078 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:36:57.078 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:36:57.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5a140a2e00 msgr2=0x7f5a140a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:57.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5a140a2e00 0x7f5a140a52c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a30107bd0 tx=0x7f5a18005f90 comp rx=0 tx=0).stop 2024-07-29T00:36:57.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 msgr2=0x7f5a3019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:57.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 0x7f5a3019e690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000e9f0 tx=0x7f5a2000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:36:57.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 shutdown_connections 2024-07-29T00:36:57.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5a140a2e00 0x7f5a140a52c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:57.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a3010d220 0x7f5a301a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:57.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30108af0 0x7f5a3019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:57.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 --2- 172.21.15.175:0/3788754605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a30108140 0x7f5a3019e690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:57.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 >> 172.21.15.175:0/3788754605 conn(0x7f5a30077890 msgr2=0x7f5a30079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:57.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 shutdown_connections 2024-07-29T00:36:57.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:57.081+0000 7f5a36000640 1 -- 172.21.15.175:0/3788754605 wait complete. 2024-07-29T00:36:57.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:57 smithi181 ceph-mon[36706]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-07-29T00:36:57.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:57 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3788754605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:57.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:57 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:36:57.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:57 smithi175 ceph-mon[32138]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-07-29T00:36:57.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:57 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3788754605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:57.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:57 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:36:57.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:57 smithi204 ceph-mon[37368]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-07-29T00:36:57.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:57 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3788754605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:36:57.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:57 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:36:58.019 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:36:58.500 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:36:59.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 -- 172.21.15.175:0/4055010817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31481076b0 msgr2=0x7f3148107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 --2- 172.21.15.175:0/4055010817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31481076b0 0x7f3148107a90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3138009900 tx=0x7f313802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 -- 172.21.15.175:0/4055010817 shutdown_connections 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 --2- 172.21.15.175:0/4055010817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3148102050 0x7f314810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 --2- 172.21.15.175:0/4055010817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481016b0 0x7f3148101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 --2- 172.21.15.175:0/4055010817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31481076b0 0x7f3148107a90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 -- 172.21.15.175:0/4055010817 >> 172.21.15.175:0/4055010817 conn(0x7f31480fd430 msgr2=0x7f31480ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 -- 172.21.15.175:0/4055010817 shutdown_connections 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.155+0000 7f314d000640 1 -- 172.21.15.175:0/4055010817 wait complete. 2024-07-29T00:36:59.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.156+0000 7f314d000640 1 Processor -- start 2024-07-29T00:36:59.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 -- start start 2024-07-29T00:36:59.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 0x7f3148197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:59.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3148102050 0x7f3148198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:59.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481076b0 0x7f314819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3148111ab0 con 0x7f31481016b0 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3148111930 con 0x7f3148102050 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f314d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3148111c30 con 0x7f31481076b0 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f3147400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 0x7f3148197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f3147400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 0x7f3148197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43248/0 (socket says 172.21.15.175:43248) 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f3147400640 1 -- 172.21.15.175:0/1801282107 learned_addr learned my addr 172.21.15.175:0/1801282107 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.157+0000 7f3146a00640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3148102050 0x7f3148198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:59.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481076b0 0x7f314819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 -- 172.21.15.175:0/1801282107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3148102050 msgr2=0x7f3148198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3148102050 0x7f3148198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 -- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 msgr2=0x7f3148197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 0x7f3148197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 -- 172.21.15.175:0/1801282107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314819cf90 con 0x7f31481076b0 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147400640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 0x7f3148197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.158+0000 7f3147e00640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481076b0 0x7f314819c8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f314000ec50 tx=0x7f314000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.159+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140003b70 con 0x7f31481076b0 2024-07-29T00:36:59.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.159+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314819d280 con 0x7f31481076b0 2024-07-29T00:36:59.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.159+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3140003d10 con 0x7f31481076b0 2024-07-29T00:36:59.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.159+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140010890 con 0x7f31481076b0 2024-07-29T00:36:59.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.159+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481a4b60 con 0x7f31481076b0 2024-07-29T00:36:59.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.160+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3148101b10 con 0x7f31481076b0 2024-07-29T00:36:59.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.162+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138010+0+0 (secure 0 0 0) 0x7f3140004b40 con 0x7f31481076b0 2024-07-29T00:36:59.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.163+0000 7f313fe00640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f312c0a2d80 0x7f312c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:36:59.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.163+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f31400cfc60 con 0x7f31481076b0 2024-07-29T00:36:59.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.164+0000 7f3147400640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f312c0a2d80 0x7f312c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:36:59.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.164+0000 7f3147400640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f312c0a2d80 0x7f312c0a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31380096c0 tx=0x7f3138005a20 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:36:59.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.165+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f314009c720 con 0x7f31481076b0 2024-07-29T00:36:59.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.355+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3148199340 con 0x7f31481076b0 2024-07-29T00:36:59.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.357+0000 7f313fe00640 1 -- 172.21.15.175:0/1801282107 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+23219 (secure 0 0 0) 0x7f3148199340 con 0x7f31481076b0 2024-07-29T00:36:59.357 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:36:59.357 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":92,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:36:29.909354+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1189171033":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/3246788131":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/257505941":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/2019123994":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/4008532755":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/3246788131":"2024-07-30T00:36:29.909303+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:36:59.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f312c0a2d80 msgr2=0x7f312c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:59.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f312c0a2d80 0x7f312c0a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31380096c0 tx=0x7f3138005a20 comp rx=0 tx=0).stop 2024-07-29T00:36:59.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481076b0 msgr2=0x7f314819c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:36:59.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481076b0 0x7f314819c8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f314000ec50 tx=0x7f314000c6a0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 shutdown_connections 2024-07-29T00:36:59.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31481076b0 0x7f314819c8b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3148102050 0x7f3148198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f312c0a2d80 0x7f312c0a5240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 --2- 172.21.15.175:0/1801282107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31481016b0 0x7f3148197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:36:59.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.360+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 >> 172.21.15.175:0/1801282107 conn(0x7f31480fd430 msgr2=0x7f31480fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:36:59.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.361+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 shutdown_connections 2024-07-29T00:36:59.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:36:59.361+0000 7f314d000640 1 -- 172.21.15.175:0/1801282107 wait complete. 2024-07-29T00:36:59.727 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:59 smithi175 ceph-mon[32138]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 38 op/s 2024-07-29T00:36:59.727 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:36:59 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1801282107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:59.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:59 smithi181 ceph-mon[36706]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 38 op/s 2024-07-29T00:36:59.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:36:59 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1801282107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:36:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:59 smithi204 ceph-mon[37368]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 38 op/s 2024-07-29T00:36:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:36:59 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1801282107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:37:00.190 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-07-29T00:37:00.656 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:00.784 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:37:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:37:00.511+0000 7f717c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:37:00.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:37:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:37:00.511+0000 7f717c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:37:00.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:37:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:37:00.511+0000 7f717c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:37:00.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:37:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:37:00.511+0000 7f717c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:37:00.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:37:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:37:00.511+0000 7f717c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-29T00:37:01.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:01 smithi181 ceph-mon[36706]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:01.782 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:01 smithi175 ceph-mon[32138]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:01 smithi204 ceph-mon[37368]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:02.359 INFO:teuthology.orchestra.run.smithi175.stdout:{"run_dir":"/var/run/ceph"} 2024-07-29T00:37:02.359 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-07-29T00:37:02.359 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 743 2024-07-29T00:37:02.360 DEBUG:tasks.quiescer:Yielding 2024-07-29T00:37:02.360 INFO:teuthology.run_tasks:Running task workunit... 2024-07-29T00:37:02.386 INFO:tasks.workunit:Pulling workunits from ref ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:37:02.387 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-29T00:37:02.387 INFO:tasks.workunit:timeout=3h 2024-07-29T00:37:02.387 INFO:tasks.workunit:cleanup=True 2024-07-29T00:37:02.388 DEBUG:teuthology.orchestra.run.smithi175:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-29T00:37:02.388 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:37:02.440 INFO:teuthology.orchestra.run.smithi175.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-29 00:34:25.038421217 +0000 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-29 00:34:25.038421217 +0000 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-29 00:36:29.521529436 +0000 2024-07-29T00:37:02.441 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-29 00:34:25.038421217 +0000 2024-07-29T00:37:02.441 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-29T00:37:02.441 DEBUG:teuthology.orchestra.run.smithi175:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-29T00:37:02.568 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ba82afec3628bf830c0830558bf3d3c3aea66701 2024-07-29T00:37:02.617 INFO:tasks.workunit.client.0.smithi175.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-29T00:37:02.886 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:03.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 -- 172.21.15.175:0/1631076732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0103a80 msgr2=0x7f5da0103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:03.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 --2- 172.21.15.175:0/1631076732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0103a80 0x7f5da0103ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009930 tx=0x7f5d8c02a420 comp rx=0 tx=0).stop 2024-07-29T00:37:03.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 -- 172.21.15.175:0/1631076732 shutdown_connections 2024-07-29T00:37:03.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 --2- 172.21.15.175:0/1631076732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0104420 0x7f5da0110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 --2- 172.21.15.175:0/1631076732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0103a80 0x7f5da0103ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 --2- 172.21.15.175:0/1631076732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0109a80 0x7f5da0109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 -- 172.21.15.175:0/1631076732 >> 172.21.15.175:0/1631076732 conn(0x7f5da00ff780 msgr2=0x7f5da0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:03.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 -- 172.21.15.175:0/1631076732 shutdown_connections 2024-07-29T00:37:03.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.530+0000 7f5da5800640 1 -- 172.21.15.175:0/1631076732 wait complete. 2024-07-29T00:37:03.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.531+0000 7f5da5800640 1 Processor -- start 2024-07-29T00:37:03.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 -- start start 2024-07-29T00:37:03.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0103a80 0x7f5da019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:03.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 0x7f5da019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:03.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0109a80 0x7f5da019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:03.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0113ed0 con 0x7f5da0109a80 2024-07-29T00:37:03.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0113d50 con 0x7f5da0103a80 2024-07-29T00:37:03.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.532+0000 7f5da5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0114050 con 0x7f5da0104420 2024-07-29T00:37:03.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0103a80 0x7f5da019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 0x7f5da019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 0x7f5da019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:54198/0 (socket says 172.21.15.175:54198) 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 -- 172.21.15.175:0/110871668 learned_addr learned my addr 172.21.15.175:0/110871668 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9f400640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0109a80 0x7f5da019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 -- 172.21.15.175:0/110871668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0103a80 msgr2=0x7f5da019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0103a80 0x7f5da019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 -- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0109a80 msgr2=0x7f5da019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0109a80 0x7f5da019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.533+0000 7f5d9e000640 1 -- 172.21.15.175:0/110871668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da019f320 con 0x7f5da0104420 2024-07-29T00:37:03.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.534+0000 7f5d9e000640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 0x7f5da019a8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c02a940 tx=0x7f5d8c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:03.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.534+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c03f070 con 0x7f5da0104420 2024-07-29T00:37:03.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.534+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c031c90 con 0x7f5da0104420 2024-07-29T00:37:03.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.534+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da019f5b0 con 0x7f5da0104420 2024-07-29T00:37:03.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.534+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c035030 con 0x7f5da0104420 2024-07-29T00:37:03.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.534+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01a6ef0 con 0x7f5da0104420 2024-07-29T00:37:03.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.536+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da01051c0 con 0x7f5da0104420 2024-07-29T00:37:03.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.537+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f5d8c042070 con 0x7f5da0104420 2024-07-29T00:37:03.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.539+0000 7f5d97400640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5d840a2e10 0x7f5d840a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:03.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.539+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f5d8c0efde0 con 0x7f5da0104420 2024-07-29T00:37:03.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.539+0000 7f5d9ea00640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5d840a2e10 0x7f5d840a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:03.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.540+0000 7f5d9ea00640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5d840a2e10 0x7f5d840a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005d00 tx=0x7f5d9000b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:03.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.543+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5d8c0bc1b0 con 0x7f5da0104420 2024-07-29T00:37:03.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.769+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5da010bff0 con 0x7f5da0104420 2024-07-29T00:37:03.770 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:03 smithi175 ceph-mon[32138]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-07-29T00:37:03.770 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:03 smithi175 ceph-mon[32138]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-07-29T00:37:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.770+0000 7f5d97400640 1 -- 172.21.15.175:0/110871668 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f5d8c0c04b0 con 0x7f5da0104420 2024-07-29T00:37:03.771 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:37:03.771 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:37:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:37:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5d840a2e10 msgr2=0x7f5d840a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5d840a2e10 0x7f5d840a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005d00 tx=0x7f5d9000b410 comp rx=0 tx=0).stop 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 msgr2=0x7f5da019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 0x7f5da019a8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c02a940 tx=0x7f5d8c003b90 comp rx=0 tx=0).stop 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 shutdown_connections 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5d840a2e10 0x7f5d840a52d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da0109a80 0x7f5da019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5da0104420 0x7f5da019a8b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 --2- 172.21.15.175:0/110871668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0103a80 0x7f5da019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 >> 172.21.15.175:0/110871668 conn(0x7f5da00ff780 msgr2=0x7f5da00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.773+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 shutdown_connections 2024-07-29T00:37:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:03.774+0000 7f5da5800640 1 -- 172.21.15.175:0/110871668 wait complete. 2024-07-29T00:37:03.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:03 smithi204 ceph-mon[37368]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-07-29T00:37:03.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:03 smithi204 ceph-mon[37368]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-07-29T00:37:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:03 smithi181 ceph-mon[36706]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-07-29T00:37:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:03 smithi181 ceph-mon[36706]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-07-29T00:37:04.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-29T00:33:58.041883+0000', 'modified': '2024-07-29T00:36:30.665877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24515, 'mds_2': 24503, 'mds_3': 24499, 'mds_4': 24484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/1301583270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 1301583270}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 1301583270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/4123110499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 4123110499}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 4123110499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'j', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/284852041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 284852041}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 284852041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/568693704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 568693704}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 568693704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1977765890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1977765890}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1977765890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.204:6837/2487746199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2487746199}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2487746199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6839/2455267021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2455267021}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2455267021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.181:6837/753661139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 753661139}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 753661139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.175:6839/1522099567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1522099567}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1522099567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/4150501134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 4150501134}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 4150501134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24539, 24515, 24503, 24499, 24484]} 2024-07-29T00:37:04.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-29T00:37:04.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.c status 2024-07-29T00:37:04.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:04 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/110871668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:04.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:04 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/110871668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:04.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:04 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/110871668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:05.244 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:05.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:05 smithi175 ceph-mon[32138]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-07-29T00:37:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:05 smithi204 ceph-mon[37368]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-07-29T00:37:05.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 -- 172.21.15.175:0/3591435668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174104260 msgr2=0x7f6174110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 --2- 172.21.15.175:0/3591435668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174104260 0x7f6174110790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f616c009ca0 tx=0x7f616c02a8c0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 -- 172.21.15.175:0/3591435668 shutdown_connections 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 --2- 172.21.15.175:0/3591435668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174104260 0x7f6174110790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 --2- 172.21.15.175:0/3591435668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741038c0 0x7f6174103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 --2- 172.21.15.175:0/3591435668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61741098c0 0x7f6174109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.883+0000 7f6173e00640 1 -- 172.21.15.175:0/3591435668 >> 172.21.15.175:0/3591435668 conn(0x7f61740ff600 msgr2=0x7f6174101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.884+0000 7f6173e00640 1 -- 172.21.15.175:0/3591435668 shutdown_connections 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.884+0000 7f6173e00640 1 -- 172.21.15.175:0/3591435668 wait complete. 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.885+0000 7f6173e00640 1 Processor -- start 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.885+0000 7f6173e00640 1 -- start start 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61741038c0 0x7f617419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6174104260 0x7f617419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741098c0 0x7f617419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6174113d10 con 0x7f6174104260 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6174113b90 con 0x7f61741038c0 2024-07-29T00:37:05.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6174113e90 con 0x7f61741098c0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6172000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6174104260 0x7f617419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6172a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61741038c0 0x7f617419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741098c0 0x7f617419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741098c0 0x7f617419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:54210/0 (socket says 172.21.15.175:54210) 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.886+0000 7f6173400640 1 -- 172.21.15.175:0/1611076262 learned_addr learned my addr 172.21.15.175:0/1611076262 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6172a00640 1 -- 172.21.15.175:0/1611076262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741098c0 msgr2=0x7f617419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6172a00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741098c0 0x7f617419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6172a00640 1 -- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6174104260 msgr2=0x7f617419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6172a00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6174104260 0x7f617419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6172a00640 1 -- 172.21.15.175:0/1611076262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617419f120 con 0x7f61741038c0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6172a00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61741038c0 0x7f617419a170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f616400e9f0 tx=0x7f616400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f616b400640 1 -- 172.21.15.175:0/1611076262 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400ccc0 con 0x7f61741038c0 2024-07-29T00:37:05.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f616b400640 1 -- 172.21.15.175:0/1611076262 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6164010910 con 0x7f61741038c0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f616b400640 1 -- 172.21.15.175:0/1611076262 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164018960 con 0x7f61741038c0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.887+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617419f410 con 0x7f61741038c0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.888+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741a6cf0 con 0x7f61741038c0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.892+0000 7f617c62d740 1 Processor -- start 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.892+0000 7f617c62d740 1 -- start start 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.892+0000 7f616b400640 1 -- 172.21.15.175:0/1611076262 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f6164010ac0 con 0x7f61741038c0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f617c62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ebd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f617c62d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ebdea0 0x5587c4ec8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f617c62d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ec8760 0x5587c4ecab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f617c62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587c4e23790 con 0x5587c4ebd580 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f617c62d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5587c4dbb880 con 0x5587c4ebdea0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f617c62d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5587c4e1f750 con 0x5587c4ec8760 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ebd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ebd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43302/0 (socket says 172.21.15.175:43302) 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 -- 172.21.15.175:0/690219649 learned_addr learned my addr 172.21.15.175:0/690219649 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6172000640 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ec8760 0x5587c4ecab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6172a00640 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ebdea0 0x5587c4ec8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 -- 172.21.15.175:0/690219649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ec8760 msgr2=0x5587c4ecab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ec8760 0x5587c4ecab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 -- 172.21.15.175:0/690219649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ebdea0 msgr2=0x5587c4ec8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ebdea0 0x5587c4ec8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6173400640 1 -- 172.21.15.175:0/690219649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c4b952b0 con 0x5587c4ebd580 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.893+0000 7f6172000640 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ec8760 0x5587c4ecab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.894+0000 7f6173400640 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ebd960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f616c005fb0 tx=0x7f616c009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e338c701c010fb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.894+0000 7f615fe00640 1 -- 172.21.15.175:0/690219649 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c031da0 con 0x5587c4ebd580 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.894+0000 7f615fe00640 1 -- 172.21.15.175:0/690219649 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f616c036030 con 0x5587c4ebd580 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.894+0000 7f615fe00640 1 -- 172.21.15.175:0/690219649 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c004280 con 0x5587c4ebd580 2024-07-29T00:37:05.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.894+0000 7f616b400640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f61580a2e90 0x7f61580a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f616b400640 1 -- 172.21.15.175:0/1611076262 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f61640cf9e0 con 0x7f61741038c0 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f6172a00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f61580a2e90 0x7f61580a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 -- 172.21.15.175:0/690219649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 msgr2=0x5587c4ebd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ebd960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f616c005fb0 tx=0x7f616c009a10 comp rx=0 tx=0).stop 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 -- 172.21.15.175:0/690219649 shutdown_connections 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ec8760 0x5587c4ecab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ebdea0 0x5587c4ec8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 --2- 172.21.15.175:0/690219649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ebd960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 -- 172.21.15.175:0/690219649 >> 172.21.15.175:0/690219649 conn(0x5587c4eb9b10 msgr2=0x5587c4eb9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 -- 172.21.15.175:0/690219649 shutdown_connections 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f617c62d740 1 -- 172.21.15.175:0/690219649 wait complete. 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.895+0000 7f6172a00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f61580a2e90 0x7f61580a5350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61640ccff0 tx=0x7f61640d4040 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.896+0000 7f617c62d740 1 -- start start 2024-07-29T00:37:05.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.896+0000 7f617c62d740 1 Processor -- start 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ec8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ec5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587c4ecb1b0 con 0x5587c4ebd580 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5587c4dde960 con 0x5587c4ec8760 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5587c4ecb330 con 0x5587c4ebdea0 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f617c62d740 10 client.0 _ll_register_callbacks cb 0x5587c4b327b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f6173400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ec8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f6173400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ec8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43310/0 (socket says 172.21.15.175:43310) 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f6173400640 1 -- 172.21.15.175:0/1480718402 learned_addr learned my addr 172.21.15.175:0/1480718402 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.897+0000 7f6172000640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ec5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 -- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 msgr2=0x5587c4ec5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 -- 172.21.15.175:0/1480718402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 msgr2=0x5587c4ec5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ec5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 -- 172.21.15.175:0/1480718402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c4deda90 con 0x5587c4ebd580 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6172000640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f6173400640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ec8210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f616c031840 tx=0x7f616c031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21649af422604c15 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f615cc00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.898+0000 7f615cc00640 1 -- 172.21.15.175:0/1480718402 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c058060 con 0x5587c4ebd580 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f615cc00640 1 -- 172.21.15.175:0/1480718402 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f616c004040 con 0x5587c4ebd580 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f616c036030 con 0x5587c4ebd580 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f615cc00640 1 -- 172.21.15.175:0/1480718402 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f616c003e40 con 0x5587c4ebd580 2024-07-29T00:37:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f617c62d740 10 client.15273 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587c4ecb1b0 con 0x5587c4ebd580 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f615cc00640 1 -- 172.21.15.175:0/1480718402 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f616c036030 con 0x5587c4ebd580 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f615cc00640 1 -- 172.21.15.175:0/1480718402 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f616c043070 con 0x5587c4ebd580 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.899+0000 7f617c62d740 10 client.15273 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.900+0000 7f617c62d740 10 client.15273 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.900+0000 7f617c62d740 20 client.15273 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.900+0000 7f617c62d740 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4ed8fe0 0x5587c4eda450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.900+0000 7f617c62d740 4 client.15273 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.900+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587c4e30d00 con 0x5587c4ed8fe0 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.900+0000 7f6172000640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4ed8fe0 0x5587c4eda450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.901+0000 7f6172000640 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4ed8fe0 0x5587c4eda450 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3014d936916b289 server_cookie=6cf4ec2dd3a9874e in_seq=0 out_seq=0 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.901+0000 7f615cc00640 10 client.15273 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.901+0000 7f615cc00640 1 -- 172.21.15.175:0/1480718402 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5587c4e30d00 con 0x5587c4ed8fe0 2024-07-29T00:37:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.901+0000 7f615cc00640 10 client.15273 handle_command_reply: tid=0 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 client.15273 shutdown 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 msgr2=0x5587c4ec8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ec8210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f616c031840 tx=0x7f616c031940 comp rx=0 tx=0).stop 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 shutdown_connections 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4ed8fe0 0x5587c4eda450 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ec5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 --2- 172.21.15.175:0/1480718402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4ec8210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 >> 172.21.15.175:0/1480718402 conn(0x5587c4eb9b10 msgr2=0x5587c4eb8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f615cc00640 0 client.15273 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 shutdown_connections 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 1 -- 172.21.15.175:0/1480718402 wait complete. 2024-07-29T00:37:05.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.902+0000 7f617c62d740 20 client.15273 trim_cache size 0 max 16384 2024-07-29T00:37:05.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 Processor -- start 2024-07-29T00:37:05.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 -- start start 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4eda450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4eb9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587c4ed8790 con 0x5587c4ebd580 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5587c4ed8e60 con 0x5587c4ec8760 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.922+0000 7f617c62d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5587c4ed9530 con 0x5587c4ebdea0 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6173400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4eda450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6173400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4eda450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43324/0 (socket says 172.21.15.175:43324) 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6173400640 1 -- 172.21.15.175:0/1432954659 learned_addr learned my addr 172.21.15.175:0/1432954659 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:05.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6173400640 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4eb9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 -- 172.21.15.175:0/1432954659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 msgr2=0x5587c4eb9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4eb9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 -- 172.21.15.175:0/1432954659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 msgr2=0x5587c4eda450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4eda450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 -- 172.21.15.175:0/1432954659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c4ec62d0 con 0x5587c4ebdea0 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6173400640 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4eda450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.923+0000 7f6172000640 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5587c4ec6b10 tx=0x7f6160005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e435e7ef8f1049 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f615f400640 1 -- 172.21.15.175:0/1432954659 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160004230 con 0x5587c4ebdea0 2024-07-29T00:37:05.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f615f400640 1 -- 172.21.15.175:0/1432954659 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61600043d0 con 0x5587c4ebdea0 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f615f400640 1 -- 172.21.15.175:0/1432954659 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61600208f0 con 0x5587c4ebdea0 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 -- 172.21.15.175:0/1432954659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 msgr2=0x5587c4ec4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5587c4ec6b10 tx=0x7f6160005f90 comp rx=0 tx=0).stop 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 -- 172.21.15.175:0/1432954659 shutdown_connections 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4eb9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebdea0 0x5587c4ec4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 --2- 172.21.15.175:0/1432954659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebd580 0x5587c4eda450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.924+0000 7f617c62d740 1 -- 172.21.15.175:0/1432954659 >> 172.21.15.175:0/1432954659 conn(0x5587c4eb9b10 msgr2=0x5587c4ec0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.925+0000 7f617c62d740 1 -- 172.21.15.175:0/1432954659 shutdown_connections 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.925+0000 7f617c62d740 1 -- 172.21.15.175:0/1432954659 wait complete. 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.925+0000 7f617c62d740 1 -- start start 2024-07-29T00:37:05.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.925+0000 7f617c62d740 1 Processor -- start 2024-07-29T00:37:05.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebd580 0x5587c4ec1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebdea0 0x5587c4ec24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ee16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587c4eda2d0 con 0x5587c4ebdea0 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5587c4ed9c00 con 0x5587c4ec8760 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5587c4eda9a0 con 0x5587c4ebd580 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f617c62d740 10 client.0 _ll_register_callbacks cb 0x7f61640d33c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6172000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebdea0 0x5587c4ec24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebd580 0x5587c4ec1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6172000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebdea0 0x5587c4ec24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43334/0 (socket says 172.21.15.175:43334) 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6172000640 1 -- 172.21.15.175:0/2777465856 learned_addr learned my addr 172.21.15.175:0/2777465856 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ee16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 -- 172.21.15.175:0/2777465856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 msgr2=0x5587c4ee16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ee16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 -- 172.21.15.175:0/2777465856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebdea0 msgr2=0x5587c4ec24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebdea0 0x5587c4ec24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.926+0000 7f6173400640 1 -- 172.21.15.175:0/2777465856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c4ee1be0 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f6173400640 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebd580 0x5587c4ec1f90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5587c4ec6ae0 tx=0x7f616c02b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74bd286f8a607eb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f615cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f615cc00640 1 -- 172.21.15.175:0/2777465856 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c058070 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f615cc00640 1 -- 172.21.15.175:0/2777465856 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f616c02fe70 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61600043d0 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f615cc00640 1 -- 172.21.15.175:0/2777465856 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f616c02f610 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f617c62d740 10 client.25378 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.927+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587c4ee2ee0 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f615cc00640 1 -- 172.21.15.175:0/2777465856 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61600043d0 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f615cc00640 1 -- 172.21.15.175:0/2777465856 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f616c043040 con 0x5587c4ebd580 2024-07-29T00:37:05.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f617c62d740 10 client.25378 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:05.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f617c62d740 10 client.25378 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-29T00:37:05.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f617c62d740 20 client.25378 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:05.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f617c62d740 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4f47a00 0x5587c4f46710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:05.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f617c62d740 4 client.25378 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:37:05.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "status"}) -- 0x5587c4e2b210 con 0x5587c4f47a00 2024-07-29T00:37:05.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.928+0000 7f6172000640 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4f47a00 0x5587c4f46710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:05.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f6172000640 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4f47a00 0x5587c4f46710 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83f32799918f51c3 server_cookie=434c558a83d7e52a in_seq=0 out_seq=0 2024-07-29T00:37:05.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f615cc00640 10 client.25378 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:05.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f615cc00640 1 -- 172.21.15.175:0/2777465856 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5587c4e2b210 con 0x5587c4f47a00 2024-07-29T00:37:05.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f615cc00640 10 client.25378 handle_command_reply: tid=0 2024-07-29T00:37:05.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 client.25378 shutdown 2024-07-29T00:37:05.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebd580 msgr2=0x5587c4ec1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebd580 0x5587c4ec1f90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5587c4ec6ae0 tx=0x7f616c02b040 comp rx=0 tx=0).stop 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 shutdown_connections 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5587c4f47a00 0x5587c4f46710 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587c4ec8760 0x5587c4ee16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587c4ebdea0 0x5587c4ec24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 --2- 172.21.15.175:0/2777465856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c4ebd580 0x5587c4ec1f90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f615cc00640 0 client.25378 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 >> 172.21.15.175:0/2777465856 conn(0x5587c4eb9b10 msgr2=0x5587c4ec0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.929+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 shutdown_connections 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.930+0000 7f617c62d740 1 -- 172.21.15.175:0/2777465856 wait complete. 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.930+0000 7f617c62d740 20 client.25378 trim_cache size 0 max 16384 2024-07-29T00:37:05.933 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 4, 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24484, 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 173.25555757699999, 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 29, 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 92, 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 92, 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 220.31098623599999 2024-07-29T00:37:05.934 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:05.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.930+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f61580a2e90 msgr2=0x7f61580a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.930+0000 7f6173e00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f61580a2e90 0x7f61580a5350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61640ccff0 tx=0x7f61640d4040 comp rx=0 tx=0).stop 2024-07-29T00:37:05.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.930+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61741038c0 msgr2=0x7f617419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:05.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.930+0000 7f6173e00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61741038c0 0x7f617419a170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f616400e9f0 tx=0x7f616400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 shutdown_connections 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61741098c0 0x7f617419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f61580a2e90 0x7f61580a5350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6174104260 0x7f617419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 --2- 172.21.15.175:0/1611076262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61741038c0 0x7f617419a170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 >> 172.21.15.175:0/1611076262 conn(0x7f61740ff600 msgr2=0x7f61740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 shutdown_connections 2024-07-29T00:37:05.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:05.931+0000 7f6173e00640 1 -- 172.21.15.175:0/1611076262 wait complete. 2024-07-29T00:37:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:05 smithi181 ceph-mon[36706]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-07-29T00:37:06.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.i status 2024-07-29T00:37:07.448 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:07.809 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:07 smithi175 ceph-mon[32138]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:37:07.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:07 smithi204 ceph-mon[37368]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:37:07.838 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:07 smithi181 ceph-mon[36706]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 -- 172.21.15.175:0/1580027537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41098a0 msgr2=0x7f39e4109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 --2- 172.21.15.175:0/1580027537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41098a0 0x7f39e4109c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009930 tx=0x7f39cc02a420 comp rx=0 tx=0).stop 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 -- 172.21.15.175:0/1580027537 shutdown_connections 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 --2- 172.21.15.175:0/1580027537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4104240 0x7f39e4110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 --2- 172.21.15.175:0/1580027537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e41038a0 0x7f39e4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 --2- 172.21.15.175:0/1580027537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41098a0 0x7f39e4109c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 -- 172.21.15.175:0/1580027537 >> 172.21.15.175:0/1580027537 conn(0x7f39e40ff560 msgr2=0x7f39e4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 -- 172.21.15.175:0/1580027537 shutdown_connections 2024-07-29T00:37:08.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.209+0000 7f39ea400640 1 -- 172.21.15.175:0/1580027537 wait complete. 2024-07-29T00:37:08.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.211+0000 7f39ea400640 1 Processor -- start 2024-07-29T00:37:08.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.211+0000 7f39ea400640 1 -- start start 2024-07-29T00:37:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39ea400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e41038a0 0x7f39e419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39ea400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 0x7f39e419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39ea400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e41098a0 0x7f39e419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39ea400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4113cf0 con 0x7f39e41038a0 2024-07-29T00:37:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39ea400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4113b70 con 0x7f39e41098a0 2024-07-29T00:37:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39ea400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4113e70 con 0x7f39e4104240 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 0x7f39e419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39e9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e41098a0 0x7f39e419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 0x7f39e419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:54262/0 (socket says 172.21.15.175:54262) 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39dbe00640 1 -- 172.21.15.175:0/4091157821 learned_addr learned my addr 172.21.15.175:0/4091157821 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.212+0000 7f39e9000640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e41038a0 0x7f39e419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39dbe00640 1 -- 172.21.15.175:0/4091157821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e41098a0 msgr2=0x7f39e419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39dbe00640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e41098a0 0x7f39e419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39dbe00640 1 -- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e41038a0 msgr2=0x7f39e419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39dbe00640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e41038a0 0x7f39e419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39dbe00640 1 -- 172.21.15.175:0/4091157821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e419f150 con 0x7f39e4104240 2024-07-29T00:37:08.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39dbe00640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 0x7f39e419a6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39d400e9f0 tx=0x7f39d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:08.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.213+0000 7f39e2000640 1 -- 172.21.15.175:0/4091157821 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400cd80 con 0x7f39e4104240 2024-07-29T00:37:08.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.214+0000 7f39e2000640 1 -- 172.21.15.175:0/4091157821 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d4010910 con 0x7f39e4104240 2024-07-29T00:37:08.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.214+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e419f440 con 0x7f39e4104240 2024-07-29T00:37:08.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.214+0000 7f39e2000640 1 -- 172.21.15.175:0/4091157821 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4018990 con 0x7f39e4104240 2024-07-29T00:37:08.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.214+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e41a6d20 con 0x7f39e4104240 2024-07-29T00:37:08.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.217+0000 7f39ee628740 1 Processor -- start 2024-07-29T00:37:08.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.217+0000 7f39e2000640 1 -- 172.21.15.175:0/4091157821 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f39d4010ac0 con 0x7f39e4104240 2024-07-29T00:37:08.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.217+0000 7f39ee628740 1 -- start start 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.217+0000 7f39ee628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec62960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39ee628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39ee628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39ee628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ebc8790 con 0x55ea9ec6d760 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39ee628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ea9eb60880 con 0x55ea9ec62580 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39ee628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ebc4750 con 0x55ea9ec62ea0 2024-07-29T00:37:08.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec62960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec62960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47268/0 (socket says 172.21.15.175:47268) 2024-07-29T00:37:08.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 -- 172.21.15.175:0/2423279566 learned_addr learned my addr 172.21.15.175:0/2423279566 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:08.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39dbe00640 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9000640 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 -- 172.21.15.175:0/2423279566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 msgr2=0x55ea9ec6d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 -- 172.21.15.175:0/2423279566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 msgr2=0x55ea9ec6fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.218+0000 7f39e9a00640 1 -- 172.21.15.175:0/2423279566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9e93a2b0 con 0x55ea9ec62580 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39e9a00640 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec62960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009f50 tx=0x7f39dc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23aa29146235c3a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39da000640 1 -- 172.21.15.175:0/2423279566 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc004d20 con 0x55ea9ec62580 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39e2000640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c40a2e10 0x7f39c40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39da000640 1 -- 172.21.15.175:0/2423279566 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39dc004050 con 0x55ea9ec62580 2024-07-29T00:37:08.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39da000640 1 -- 172.21.15.175:0/2423279566 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc005220 con 0x55ea9ec62580 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39e2000640 1 -- 172.21.15.175:0/4091157821 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f39d40cfb50 con 0x7f39e4104240 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.219+0000 7f39e9000640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c40a2e10 0x7f39c40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 -- 172.21.15.175:0/2423279566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 msgr2=0x55ea9ec62960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec62960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009f50 tx=0x7f39dc0118d0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 -- 172.21.15.175:0/2423279566 shutdown_connections 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 --2- 172.21.15.175:0/2423279566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec62960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 -- 172.21.15.175:0/2423279566 >> 172.21.15.175:0/2423279566 conn(0x55ea9ec5eb10 msgr2=0x55ea9ec5eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 -- 172.21.15.175:0/2423279566 shutdown_connections 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39e9000640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c40a2e10 0x7f39c40a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006010 tx=0x7f39cc009610 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.220+0000 7f39ee628740 1 -- 172.21.15.175:0/2423279566 wait complete. 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 -- start start 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 Processor -- start 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62580 0x55ea9ec6a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec70330 con 0x55ea9ec62580 2024-07-29T00:37:08.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.221+0000 7f39ee628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ea9eb83960 con 0x55ea9ec62ea0 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39ee628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec704b0 con 0x55ea9ec6d760 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39ee628740 10 client.0 _ll_register_callbacks cb 0x55ea9e8d77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39e9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62580 0x55ea9ec6a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47276/0 (socket says 172.21.15.175:47276) 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 -- 172.21.15.175:0/4228941455 learned_addr learned my addr 172.21.15.175:0/4228941455 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39e9a00640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 -- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 msgr2=0x55ea9ec6b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 -- 172.21.15.175:0/4228941455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62580 msgr2=0x55ea9ec6a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62580 0x55ea9ec6a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39dbe00640 1 -- 172.21.15.175:0/4228941455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9eb92a90 con 0x55ea9ec62ea0 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.222+0000 7f39e9a00640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:08.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39dbe00640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6aba0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f39d409baa0 tx=0x7f39d409b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f2852b7e40693f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:08.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:08.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39d2a00640 1 -- 172.21.15.175:0/4228941455 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40a0150 con 0x55ea9ec62ea0 2024-07-29T00:37:08.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39d2a00640 1 -- 172.21.15.175:0/4228941455 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f39d4018b30 con 0x55ea9ec62ea0 2024-07-29T00:37:08.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39dc004050 con 0x55ea9ec62ea0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39d2a00640 1 -- 172.21.15.175:0/4228941455 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39d40a4a10 con 0x55ea9ec62ea0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39ee628740 10 client.25391 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.223+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea9ec7bf60 con 0x55ea9ec62ea0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39d2a00640 1 -- 172.21.15.175:0/4228941455 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39dc004050 con 0x55ea9ec62ea0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39d2a00640 1 -- 172.21.15.175:0/4228941455 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39d4097af0 con 0x55ea9ec62ea0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39ee628740 10 client.25391 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39ee628740 10 client.25391 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39ee628740 20 client.25391 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39ee628740 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9ec7e5e0 0x55ea9ec7fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39ee628740 4 client.25391 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea9ebd5d00 con 0x55ea9ec7e5e0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.224+0000 7f39e9000640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9ec7e5e0 0x55ea9ec7fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.225+0000 7f39e9000640 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9ec7e5e0 0x55ea9ec7fa50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=653517e16c92fbbe server_cookie=b6502a2cb8c40035 in_seq=0 out_seq=0 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.225+0000 7f39d2a00640 10 client.25391 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39d2a00640 1 -- 172.21.15.175:0/4228941455 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea9ebd5d00 con 0x55ea9ec7e5e0 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39d2a00640 10 client.25391 handle_command_reply: tid=0 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 client.25391 shutdown 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 msgr2=0x55ea9ec6aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6aba0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f39d409baa0 tx=0x7f39d409b9d0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 shutdown_connections 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9ec7e5e0 0x55ea9ec7fa50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ec6b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec6aba0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 --2- 172.21.15.175:0/4228941455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62580 0x55ea9ec6a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39d2a00640 0 client.25391 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 >> 172.21.15.175:0/4228941455 conn(0x55ea9ec5eb10 msgr2=0x55ea9ec647f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:08.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.226+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 shutdown_connections 2024-07-29T00:37:08.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.227+0000 7f39ee628740 1 -- 172.21.15.175:0/4228941455 wait complete. 2024-07-29T00:37:08.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.227+0000 7f39ee628740 20 client.25391 trim_cache size 0 max 16384 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.246+0000 7f39ee628740 1 Processor -- start 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.246+0000 7f39ee628740 1 -- start start 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39ee628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec7fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39ee628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39ee628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39ee628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec7dd90 con 0x55ea9ec6d760 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39ee628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec7e460 con 0x55ea9ec62580 2024-07-29T00:37:08.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39ee628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec7eb30 con 0x55ea9ec62ea0 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39e9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39e9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec7fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39e9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec7fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47278/0 (socket says 172.21.15.175:47278) 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39e9000640 1 -- 172.21.15.175:0/920795197 learned_addr learned my addr 172.21.15.175:0/920795197 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.247+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39dbe00640 1 -- 172.21.15.175:0/920795197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 msgr2=0x55ea9ec7fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39dbe00640 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec7fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39dbe00640 1 -- 172.21.15.175:0/920795197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 msgr2=0x55ea9ec597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39dbe00640 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39dbe00640 1 -- 172.21.15.175:0/920795197 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9ec70440 con 0x55ea9ec62ea0 2024-07-29T00:37:08.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39e9000640 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec7fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:08.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39dbe00640 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec64fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39d4098720 tx=0x7f39d400c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc9e9b402f54df0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39d9600640 1 -- 172.21.15.175:0/920795197 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40ccfe0 con 0x55ea9ec62ea0 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39d9600640 1 -- 172.21.15.175:0/920795197 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d40d5030 con 0x55ea9ec62ea0 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.248+0000 7f39d9600640 1 -- 172.21.15.175:0/920795197 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4018d90 con 0x55ea9ec62ea0 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 -- 172.21.15.175:0/920795197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 msgr2=0x55ea9ec64fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec64fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39d4098720 tx=0x7f39d400c7d0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 -- 172.21.15.175:0/920795197 shutdown_connections 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec6d760 0x55ea9ec597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec64fe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 --2- 172.21.15.175:0/920795197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec7fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 -- 172.21.15.175:0/920795197 >> 172.21.15.175:0/920795197 conn(0x55ea9ec5eb10 msgr2=0x55ea9ec65940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:08.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 -- 172.21.15.175:0/920795197 shutdown_connections 2024-07-29T00:37:08.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.249+0000 7f39ee628740 1 -- 172.21.15.175:0/920795197 wait complete. 2024-07-29T00:37:08.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 -- start start 2024-07-29T00:37:08.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 Processor -- start 2024-07-29T00:37:08.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ece6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec7f8d0 con 0x55ea9ec62ea0 2024-07-29T00:37:08.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec7f200 con 0x55ea9ec62580 2024-07-29T00:37:08.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea9ec7ffa0 con 0x55ea9ec6d760 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.250+0000 7f39ee628740 10 client.0 _ll_register_callbacks cb 0x7f39d400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec81710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43380/0 (socket says 172.21.15.175:43380) 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39dbe00640 1 -- 172.21.15.175:0/1608935065 learned_addr learned my addr 172.21.15.175:0/1608935065 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9a00640 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ece6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9000640 1 -- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 msgr2=0x55ea9ece6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9000640 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ece6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9000640 1 -- 172.21.15.175:0/1608935065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62ea0 msgr2=0x55ea9ec81710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9000640 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec81710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.251+0000 7f39e9000640 1 -- 172.21.15.175:0/1608935065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9ece72d0 con 0x55ea9ec62580 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39e9000640 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec811d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0096c0 tx=0x7f39cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67410e3423152e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:08.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39d2a00640 1 -- 172.21.15.175:0/1608935065 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc035050 con 0x55ea9ec62580 2024-07-29T00:37:08.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39d2a00640 1 -- 172.21.15.175:0/1608935065 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f39cc004ad0 con 0x55ea9ec62580 2024-07-29T00:37:08.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d40d5030 con 0x55ea9ec62580 2024-07-29T00:37:08.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39d2a00640 1 -- 172.21.15.175:0/1608935065 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39cc004cd0 con 0x55ea9ec62580 2024-07-29T00:37:08.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39ee628740 10 client.25397 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:08.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.252+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea9ece85d0 con 0x55ea9ec62580 2024-07-29T00:37:08.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39d2a00640 1 -- 172.21.15.175:0/1608935065 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d40d5030 con 0x55ea9ec62580 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39d2a00640 1 -- 172.21.15.175:0/1608935065 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39cc059070 con 0x55ea9ec62580 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39ee628740 10 client.25397 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39ee628740 10 client.25397 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39ee628740 20 client.25397 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39ee628740 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9eced170 0x55ea9ecebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39ee628740 4 client.25397 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "status"}) -- 0x55ea9ebd0210 con 0x55ea9eced170 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.253+0000 7f39dbe00640 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9eced170 0x55ea9ecebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39dbe00640 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9eced170 0x55ea9ecebe80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93f6964f743484cc server_cookie=db60da55f9bc9060 in_seq=0 out_seq=0 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39d2a00640 10 client.25397 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39d2a00640 1 -- 172.21.15.175:0/1608935065 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ea9ebd0210 con 0x55ea9eced170 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39d2a00640 10 client.25397 handle_command_reply: tid=0 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 client.25397 shutdown 2024-07-29T00:37:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 msgr2=0x55ea9ec811d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec811d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0096c0 tx=0x7f39cc009260 comp rx=0 tx=0).stop 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 shutdown_connections 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ea9eced170 0x55ea9ecebe80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea9ec6d760 0x55ea9ece6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9ec62ea0 0x55ea9ec81710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39d2a00640 0 client.25397 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 --2- 172.21.15.175:0/1608935065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea9ec62580 0x55ea9ec811d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 >> 172.21.15.175:0/1608935065 conn(0x55ea9ec5eb10 msgr2=0x55ea9ec68ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 shutdown_connections 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.254+0000 7f39ee628740 1 -- 172.21.15.175:0/1608935065 wait complete. 2024-07-29T00:37:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ee628740 20 client.25397 trim_cache size 0 max 16384 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 3, 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24499, 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 176.59624480700001, 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 29, 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 92, 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 92, 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 217.30287812700001 2024-07-29T00:37:08.258 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:08.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c40a2e10 msgr2=0x7f39c40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c40a2e10 0x7f39c40a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006010 tx=0x7f39cc009610 comp rx=0 tx=0).stop 2024-07-29T00:37:08.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 msgr2=0x7f39e419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:08.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 0x7f39e419a6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39d400e9f0 tx=0x7f39d400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 shutdown_connections 2024-07-29T00:37:08.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e41098a0 0x7f39e419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4104240 0x7f39e419a6e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c40a2e10 0x7f39c40a52d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 --2- 172.21.15.175:0/4091157821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e41038a0 0x7f39e419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.255+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 >> 172.21.15.175:0/4091157821 conn(0x7f39e40ff560 msgr2=0x7f39e4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.256+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 shutdown_connections 2024-07-29T00:37:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:08.256+0000 7f39ea400640 1 -- 172.21.15.175:0/4091157821 wait complete. 2024-07-29T00:37:09.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.h status 2024-07-29T00:37:09.749 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:09 smithi204 ceph-mon[37368]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:37:09.898 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:09 smithi175 ceph-mon[32138]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:37:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:09 smithi181 ceph-mon[36706]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 -- 172.21.15.175:0/3281425139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181098a0 msgr2=0x7f0618109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 --2- 172.21.15.175:0/3281425139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181098a0 0x7f0618109c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0608009860 tx=0x7f060802a340 comp rx=0 tx=0).stop 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 -- 172.21.15.175:0/3281425139 shutdown_connections 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 --2- 172.21.15.175:0/3281425139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0618104240 0x7f0618110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 --2- 172.21.15.175:0/3281425139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06181038a0 0x7f0618103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 --2- 172.21.15.175:0/3281425139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181098a0 0x7f0618109c80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 -- 172.21.15.175:0/3281425139 >> 172.21.15.175:0/3281425139 conn(0x7f06180ff560 msgr2=0x7f0618101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:10.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 -- 172.21.15.175:0/3281425139 shutdown_connections 2024-07-29T00:37:10.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.413+0000 7f0617200640 1 -- 172.21.15.175:0/3281425139 wait complete. 2024-07-29T00:37:10.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.414+0000 7f0617200640 1 Processor -- start 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.414+0000 7f0617200640 1 -- start start 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0617200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 0x7f061819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0617200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0618104240 0x7f061819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0617200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06181098a0 0x7f061819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0617200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113c60 con 0x7f06181098a0 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0617200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113ae0 con 0x7f06181038a0 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0617200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113de0 con 0x7f0618104240 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0615e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 0x7f061819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0615e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 0x7f061819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57648/0 (socket says 172.21.15.175:57648) 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0615e00640 1 -- 172.21.15.175:0/3987712794 learned_addr learned my addr 172.21.15.175:0/3987712794 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f0616800640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06181098a0 0x7f061819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.415+0000 7f060fe00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0618104240 0x7f061819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0615e00640 1 -- 172.21.15.175:0/3987712794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0618104240 msgr2=0x7f061819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0615e00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0618104240 0x7f061819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0615e00640 1 -- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06181098a0 msgr2=0x7f061819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0615e00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06181098a0 0x7f061819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0615e00640 1 -- 172.21.15.175:0/3987712794 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061819f110 con 0x7f06181038a0 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0615e00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 0x7f061819a160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0608035040 tx=0x7f0608033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f060ea00640 1 -- 172.21.15.175:0/3987712794 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608037040 con 0x7f06181038a0 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f060ea00640 1 -- 172.21.15.175:0/3987712794 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0608040070 con 0x7f06181038a0 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f060ea00640 1 -- 172.21.15.175:0/3987712794 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608003a10 con 0x7f06181038a0 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061819f3a0 con 0x7f06181038a0 2024-07-29T00:37:10.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.416+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f061819f830 con 0x7f06181038a0 2024-07-29T00:37:10.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f060ea00640 1 -- 172.21.15.175:0/3987712794 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f0608004150 con 0x7f06181038a0 2024-07-29T00:37:10.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 Processor -- start 2024-07-29T00:37:10.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 -- start start 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc449960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc449ea0 0x5648fc454220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc454760 0x5648fc456b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648fc3af790 con 0x5648fc449ea0 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648fc347880 con 0x5648fc449580 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.420+0000 7f061fc43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648fc3ab750 con 0x5648fc454760 2024-07-29T00:37:10.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc449960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0615e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc449ea0 0x5648fc454220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc449960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57654/0 (socket says 172.21.15.175:57654) 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 -- 172.21.15.175:0/2273428534 learned_addr learned my addr 172.21.15.175:0/2273428534 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 -- 172.21.15.175:0/2273428534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc454760 msgr2=0x5648fc456b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc454760 0x5648fc456b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 -- 172.21.15.175:0/2273428534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc449ea0 msgr2=0x5648fc454220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc449ea0 0x5648fc454220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 -- 172.21.15.175:0/2273428534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fc1212b0 con 0x5648fc449580 2024-07-29T00:37:10.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f060ea00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f05fc0a2e90 0x7f05fc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.421+0000 7f0616800640 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc449960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0610009f50 tx=0x7f06100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9cae1d32674248 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:10.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.422+0000 7f060ea00640 1 -- 172.21.15.175:0/3987712794 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f06080ef5a0 con 0x7f06181038a0 2024-07-29T00:37:10.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.422+0000 7f0607400640 1 -- 172.21.15.175:0/2273428534 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610004d20 con 0x5648fc449580 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.422+0000 7f0607400640 1 -- 172.21.15.175:0/2273428534 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0610004050 con 0x5648fc449580 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.422+0000 7f0607400640 1 -- 172.21.15.175:0/2273428534 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610005280 con 0x5648fc449580 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.422+0000 7f0615e00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f05fc0a2e90 0x7f05fc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 -- 172.21.15.175:0/2273428534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 msgr2=0x5648fc449960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc449960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0610009f50 tx=0x7f06100118d0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f0615e00640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f05fc0a2e90 0x7f05fc0a5350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06080ecc60 tx=0x7f060802a340 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 -- 172.21.15.175:0/2273428534 shutdown_connections 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc454760 0x5648fc456b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc449ea0 0x5648fc454220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 --2- 172.21.15.175:0/2273428534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc449960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 -- 172.21.15.175:0/2273428534 >> 172.21.15.175:0/2273428534 conn(0x5648fc445b10 msgr2=0x5648fc445ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 -- 172.21.15.175:0/2273428534 shutdown_connections 2024-07-29T00:37:10.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 -- 172.21.15.175:0/2273428534 wait complete. 2024-07-29T00:37:10.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.423+0000 7f061fc43740 1 -- start start 2024-07-29T00:37:10.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 Processor -- start 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449580 0x5648fc454210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 0x5648fc451560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc451cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648fc457330 con 0x5648fc454760 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648fc36a960 con 0x5648fc449ea0 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648fc4574b0 con 0x5648fc449580 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.424+0000 7f061fc43740 10 client.0 _ll_register_callbacks cb 0x5648fc0be7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 0x5648fc451560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 0x5648fc451560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57662/0 (socket says 172.21.15.175:57662) 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 -- 172.21.15.175:0/1167757609 learned_addr learned my addr 172.21.15.175:0/1167757609 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f0616800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449580 0x5648fc454210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f0616800640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc451cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 -- 172.21.15.175:0/1167757609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449580 msgr2=0x5648fc454210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449580 0x5648fc454210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 -- 172.21.15.175:0/1167757609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 msgr2=0x5648fc451cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc451cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 -- 172.21.15.175:0/1167757609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fc379a90 con 0x5648fc449ea0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f0616800640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc451cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f060fe00640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 0x5648fc451560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06000116f0 tx=0x7f0600011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e937bea770b09447 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.425+0000 7f05fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f05fbe00640 1 -- 172.21.15.175:0/1167757609 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060001e060 con 0x5648fc449ea0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f05fbe00640 1 -- 172.21.15.175:0/1167757609 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f060000f810 con 0x5648fc449ea0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0610004050 con 0x5648fc449ea0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f05fbe00640 1 -- 172.21.15.175:0/1167757609 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f060000fa10 con 0x5648fc449ea0 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f061fc43740 10 client.25409 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:10.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648fc462ef0 con 0x5648fc449ea0 2024-07-29T00:37:10.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.426+0000 7f05fbe00640 1 -- 172.21.15.175:0/1167757609 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0610004050 con 0x5648fc449ea0 2024-07-29T00:37:10.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f05fbe00640 1 -- 172.21.15.175:0/1167757609 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f060000e3b0 con 0x5648fc449ea0 2024-07-29T00:37:10.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f061fc43740 10 client.25409 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f061fc43740 10 client.25409 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f061fc43740 20 client.25409 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f061fc43740 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc465540 0x5648fc4669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f061fc43740 4 client.25409 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648fc3bcd00 con 0x5648fc465540 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.427+0000 7f0616800640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc465540 0x5648fc4669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.428+0000 7f0616800640 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc465540 0x5648fc4669b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6d3a6868768b549 server_cookie=4a86115d25a4eeec in_seq=0 out_seq=0 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.428+0000 7f05fbe00640 10 client.25409 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f05fbe00640 1 -- 172.21.15.175:0/1167757609 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648fc3bcd00 con 0x5648fc465540 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f05fbe00640 10 client.25409 handle_command_reply: tid=0 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 client.25409 shutdown 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 msgr2=0x5648fc451560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 0x5648fc451560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06000116f0 tx=0x7f0600011af0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 shutdown_connections 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc451cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc465540 0x5648fc4669b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449ea0 0x5648fc451560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 --2- 172.21.15.175:0/1167757609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449580 0x5648fc454210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 >> 172.21.15.175:0/1167757609 conn(0x5648fc445b10 msgr2=0x5648fc444c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f05fbe00640 0 client.25409 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:10.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 shutdown_connections 2024-07-29T00:37:10.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 1 -- 172.21.15.175:0/1167757609 wait complete. 2024-07-29T00:37:10.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.429+0000 7f061fc43740 20 client.25409 trim_cache size 0 max 16384 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.448+0000 7f061fc43740 1 Processor -- start 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 -- start start 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc4669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc450390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648fc464cf0 con 0x5648fc454760 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648fc4653c0 con 0x5648fc449580 2024-07-29T00:37:10.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f061fc43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648fc465a90 con 0x5648fc449ea0 2024-07-29T00:37:10.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f060fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f0616800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc4669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f060fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58732/0 (socket says 172.21.15.175:58732) 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f060fe00640 1 -- 172.21.15.175:0/1323107674 learned_addr learned my addr 172.21.15.175:0/1323107674 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.449+0000 7f0616800640 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc450390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f060fe00640 1 -- 172.21.15.175:0/1323107674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 msgr2=0x5648fc4669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f060fe00640 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc4669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f060fe00640 1 -- 172.21.15.175:0/1323107674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 msgr2=0x5648fc450390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f060fe00640 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc450390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f060fe00640 1 -- 172.21.15.175:0/1323107674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fc449cd0 con 0x5648fc449ea0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f060fe00640 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44fe50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06000116c0 tx=0x7f0600009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281708b1e01c45e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.450+0000 7f0606a00640 1 -- 172.21.15.175:0/1323107674 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600031070 con 0x5648fc449ea0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f0606a00640 1 -- 172.21.15.175:0/1323107674 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0600022430 con 0x5648fc449ea0 2024-07-29T00:37:10.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f0606a00640 1 -- 172.21.15.175:0/1323107674 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600019670 con 0x5648fc449ea0 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 -- 172.21.15.175:0/1323107674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 msgr2=0x5648fc44fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44fe50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06000116c0 tx=0x7f0600009590 comp rx=0 tx=0).stop 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 -- 172.21.15.175:0/1323107674 shutdown_connections 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc450390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44fe50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 --2- 172.21.15.175:0/1323107674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc4669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 -- 172.21.15.175:0/1323107674 >> 172.21.15.175:0/1323107674 conn(0x5648fc445b10 msgr2=0x5648fc44c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:10.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 -- 172.21.15.175:0/1323107674 shutdown_connections 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.451+0000 7f061fc43740 1 -- 172.21.15.175:0/1323107674 wait complete. 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 -- start start 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 Processor -- start 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc44e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc44f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648fc466830 con 0x5648fc454760 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648fc466160 con 0x5648fc449580 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.452+0000 7f061fc43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648fc466f00 con 0x5648fc449ea0 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f061fc43740 10 client.0 _ll_register_callbacks cb 0x7f0600009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f0616800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc44e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f060fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f0616800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc44e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57676/0 (socket says 172.21.15.175:57676) 2024-07-29T00:37:10.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f0616800640 1 -- 172.21.15.175:0/1718588961 learned_addr learned my addr 172.21.15.175:0/1718588961 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:10.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f0616800640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc44f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f060fe00640 1 -- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 msgr2=0x5648fc44e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f060fe00640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc44e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f060fe00640 1 -- 172.21.15.175:0/1718588961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 msgr2=0x5648fc44f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f060fe00640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc44f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f060fe00640 1 -- 172.21.15.175:0/1718588961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fc46b120 con 0x5648fc449ea0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.453+0000 7f0616800640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc44e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f060fe00640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44ec50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5648fc44c450 tx=0x7f0600009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8060695571c740cb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f05fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f05fbe00640 1 -- 172.21.15.175:0/1718588961 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060001e070 con 0x5648fc449ea0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0600022430 con 0x5648fc449ea0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f05fbe00640 1 -- 172.21.15.175:0/1718588961 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0600005230 con 0x5648fc449ea0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f05fbe00640 1 -- 172.21.15.175:0/1718588961 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0600031070 con 0x5648fc449ea0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f061fc43740 10 client.25411 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648fc4d86b0 con 0x5648fc449ea0 2024-07-29T00:37:10.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.454+0000 7f05fbe00640 1 -- 172.21.15.175:0/1718588961 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0600022430 con 0x5648fc449ea0 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f05fbe00640 1 -- 172.21.15.175:0/1718588961 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f060002b030 con 0x5648fc449ea0 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f061fc43740 10 client.25411 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f061fc43740 10 client.25411 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f061fc43740 20 client.25411 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f061fc43740 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc4dd010 0x5648fc4db6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f061fc43740 4 client.25411 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "status"}) -- 0x5648fc3b7210 con 0x5648fc4dd010 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.455+0000 7f0616800640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc4dd010 0x5648fc4db6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f0616800640 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc4dd010 0x5648fc4db6d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb04482b402ef2b1 server_cookie=2890adddc2cc0c7 in_seq=0 out_seq=0 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f05fbe00640 10 client.25411 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f05fbe00640 1 -- 172.21.15.175:0/1718588961 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648fc3b7210 con 0x5648fc4dd010 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f05fbe00640 10 client.25411 handle_command_reply: tid=0 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 client.25411 shutdown 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 msgr2=0x5648fc44ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44ec50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5648fc44c450 tx=0x7f0600009f80 comp rx=0 tx=0).stop 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 shutdown_connections 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648fc454760 0x5648fc44f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648fc449ea0 0x5648fc44ec50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5648fc4dd010 0x5648fc4db6d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 --2- 172.21.15.175:0/1718588961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648fc449580 0x5648fc44e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 >> 172.21.15.175:0/1718588961 conn(0x5648fc445b10 msgr2=0x5648fc44fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f05fbe00640 0 client.25411 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 shutdown_connections 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 1 -- 172.21.15.175:0/1718588961 wait complete. 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.456+0000 7f061fc43740 20 client.25411 trim_cache size 0 max 16384 2024-07-29T00:37:10.460 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 2, 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24503, 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 179.80870613600001, 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 29, 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 92, 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 92, 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 222.08231799800001 2024-07-29T00:37:10.461 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:10.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f05fc0a2e90 msgr2=0x7f05fc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f05fc0a2e90 0x7f05fc0a5350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06080ecc60 tx=0x7f060802a340 comp rx=0 tx=0).stop 2024-07-29T00:37:10.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 msgr2=0x7f061819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:10.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 0x7f061819a160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0608035040 tx=0x7f0608033b60 comp rx=0 tx=0).stop 2024-07-29T00:37:10.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 shutdown_connections 2024-07-29T00:37:10.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f05fc0a2e90 0x7f05fc0a5350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06181098a0 0x7f061819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0618104240 0x7f061819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 --2- 172.21.15.175:0/3987712794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06181038a0 0x7f061819a160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:10.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.457+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 >> 172.21.15.175:0/3987712794 conn(0x7f06180ff560 msgr2=0x7f06180ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:10.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.458+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 shutdown_connections 2024-07-29T00:37:10.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:10.458+0000 7f0617200640 1 -- 172.21.15.175:0/3987712794 wait complete. 2024-07-29T00:37:11.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.g status 2024-07-29T00:37:11.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:11 smithi175 ceph-mon[32138]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-07-29T00:37:11.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:11 smithi204 ceph-mon[37368]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-07-29T00:37:11.922 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:11 smithi181 ceph-mon[36706]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-07-29T00:37:12.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 -- 172.21.15.175:0/1980992006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40108670 msgr2=0x7fdf40114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/1980992006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40108670 0x7fdf40114b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009930 tx=0x7fdf3802a420 comp rx=0 tx=0).stop 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 -- 172.21.15.175:0/1980992006 shutdown_connections 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/1980992006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40108670 0x7fdf40114b90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/1980992006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf40107d10 0x7fdf40108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/1980992006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4010dd50 0x7fdf4010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 -- 172.21.15.175:0/1980992006 >> 172.21.15.175:0/1980992006 conn(0x7fdf40077670 msgr2=0x7fdf40079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 -- 172.21.15.175:0/1980992006 shutdown_connections 2024-07-29T00:37:12.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.560+0000 7fdf3fe00640 1 -- 172.21.15.175:0/1980992006 wait complete. 2024-07-29T00:37:12.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.561+0000 7fdf3fe00640 1 Processor -- start 2024-07-29T00:37:12.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.561+0000 7fdf3fe00640 1 -- start start 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.561+0000 7fdf3fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 0x7fdf4019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.561+0000 7fdf3fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf40108670 0x7fdf4019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf4010dd50 0x7fdf401a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf401180d0 con 0x7fdf40108670 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40117f50 con 0x7fdf4010dd50 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40118250 con 0x7fdf40107d10 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 0x7fdf4019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 0x7fdf4019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58748/0 (socket says 172.21.15.175:58748) 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 -- 172.21.15.175:0/3109995603 learned_addr learned my addr 172.21.15.175:0/3109995603 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3f400640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf4010dd50 0x7fdf401a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3e000640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf40108670 0x7fdf4019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 -- 172.21.15.175:0/3109995603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf4010dd50 msgr2=0x7fdf401a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf4010dd50 0x7fdf401a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 -- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf40108670 msgr2=0x7fdf4019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf40108670 0x7fdf4019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3ea00640 1 -- 172.21.15.175:0/3109995603 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf401a34c0 con 0x7fdf40107d10 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3f400640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf4010dd50 0x7fdf401a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.562+0000 7fdf3e000640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf40108670 0x7fdf4019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.563+0000 7fdf3ea00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 0x7fdf4019e510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000e9f0 tx=0x7fdf3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:12.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.563+0000 7fdf37400640 1 -- 172.21.15.175:0/3109995603 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000cd80 con 0x7fdf40107d10 2024-07-29T00:37:12.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.563+0000 7fdf37400640 1 -- 172.21.15.175:0/3109995603 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf30010910 con 0x7fdf40107d10 2024-07-29T00:37:12.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.563+0000 7fdf37400640 1 -- 172.21.15.175:0/3109995603 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30018990 con 0x7fdf40107d10 2024-07-29T00:37:12.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.563+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf401a37b0 con 0x7fdf40107d10 2024-07-29T00:37:12.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.563+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf401ab090 con 0x7fdf40107d10 2024-07-29T00:37:12.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 Processor -- start 2024-07-29T00:37:12.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf37400640 1 -- 172.21.15.175:0/3109995603 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7fdf30010ac0 con 0x7fdf40107d10 2024-07-29T00:37:12.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 -- start start 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648bea8a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea95220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea95760 0x5648bea97b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648be9f0790 con 0x5648bea8a580 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648be988880 con 0x5648bea8aea0 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf48417740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648be9ec750 con 0x5648bea95760 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648bea8a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf3ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea95220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648bea8a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55310/0 (socket says 172.21.15.175:55310) 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.568+0000 7fdf3f400640 1 -- 172.21.15.175:0/1682324773 learned_addr learned my addr 172.21.15.175:0/1682324773 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3e000640 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea95760 0x5648bea97b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3f400640 1 -- 172.21.15.175:0/1682324773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea95760 msgr2=0x5648bea97b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3f400640 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea95760 0x5648bea97b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3f400640 1 -- 172.21.15.175:0/1682324773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 msgr2=0x5648bea95220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3f400640 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea95220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3f400640 1 -- 172.21.15.175:0/1682324773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648be7622b0 con 0x5648bea8a580 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3e000640 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea95760 0x5648bea97b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.569+0000 7fdf3f400640 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648bea8a960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009930 tx=0x7fdf38005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3884a349b5e6ceaa server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.570+0000 7fdf37400640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf240a2e90 0x7fdf240a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.570+0000 7fdf2be00640 1 -- 172.21.15.175:0/1682324773 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38040070 con 0x5648bea8a580 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.570+0000 7fdf37400640 1 -- 172.21.15.175:0/3109995603 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdf300cf9b0 con 0x7fdf40107d10 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.570+0000 7fdf2be00640 1 -- 172.21.15.175:0/1682324773 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf38003cd0 con 0x5648bea8a580 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.570+0000 7fdf2be00640 1 -- 172.21.15.175:0/1682324773 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38035040 con 0x5648bea8a580 2024-07-29T00:37:12.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.570+0000 7fdf3ea00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf240a2e90 0x7fdf240a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 -- 172.21.15.175:0/1682324773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 msgr2=0x5648bea8a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648bea8a960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009930 tx=0x7fdf38005d50 comp rx=0 tx=0).stop 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf3ea00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf240a2e90 0x7fdf240a5350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf300ccfc0 tx=0x7fdf300d34c0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 -- 172.21.15.175:0/1682324773 shutdown_connections 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea95760 0x5648bea97b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea95220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 --2- 172.21.15.175:0/1682324773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648bea8a960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 -- 172.21.15.175:0/1682324773 >> 172.21.15.175:0/1682324773 conn(0x5648bea86b10 msgr2=0x5648bea86ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 -- 172.21.15.175:0/1682324773 shutdown_connections 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.571+0000 7fdf48417740 1 -- 172.21.15.175:0/1682324773 wait complete. 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 -- start start 2024-07-29T00:37:12.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 Processor -- start 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8a580 0x5648bea926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea92c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea95760 0x5648bea93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648bea98330 con 0x5648bea95760 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648be9ab960 con 0x5648bea8aea0 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648bea984b0 con 0x5648bea8a580 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.572+0000 7fdf48417740 10 client.0 _ll_register_callbacks cb 0x5648be6ff7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea92c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea92c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57704/0 (socket says 172.21.15.175:57704) 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 -- 172.21.15.175:0/3400106388 learned_addr learned my addr 172.21.15.175:0/3400106388 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8a580 0x5648bea926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3f400640 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea95760 0x5648bea93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 -- 172.21.15.175:0/3400106388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8a580 msgr2=0x5648bea926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8a580 0x5648bea926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 -- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea95760 msgr2=0x5648bea93140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea95760 0x5648bea93140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 -- 172.21.15.175:0/3400106388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648be9baa90 con 0x5648bea8aea0 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf3e000640 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea92c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c011d40 tx=0x7fdf2c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fa632c992234b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.573+0000 7fdf28c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:12.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.574+0000 7fdf28c00640 1 -- 172.21.15.175:0/3400106388 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c004b20 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.574+0000 7fdf28c00640 1 -- 172.21.15.175:0/3400106388 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdf2c004cc0 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.574+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf38003cd0 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.574+0000 7fdf28c00640 1 -- 172.21.15.175:0/3400106388 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf2c005020 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.574+0000 7fdf48417740 10 client.25427 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.574+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648beaa3fb0 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf28c00640 1 -- 172.21.15.175:0/3400106388 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf38003cd0 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf28c00640 1 -- 172.21.15.175:0/3400106388 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf2c01a670 con 0x5648bea8aea0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf48417740 10 client.25427 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf48417740 10 client.25427 resolve_mds: resolved name 'g' to daemon mds.g 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf48417740 20 client.25427 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf48417740 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beaa65e0 0x5648beaa7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf48417740 4 client.25427 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648be9fdd00 con 0x5648beaa65e0 2024-07-29T00:37:12.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.575+0000 7fdf3f400640 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beaa65e0 0x5648beaa7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.576+0000 7fdf3f400640 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beaa65e0 0x5648beaa7a50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25ff0b3a7e78d8af server_cookie=94a03f71f6cc10f3 in_seq=0 out_seq=0 2024-07-29T00:37:12.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.576+0000 7fdf28c00640 10 client.25427 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf28c00640 1 -- 172.21.15.175:0/3400106388 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648be9fdd00 con 0x5648beaa65e0 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf28c00640 10 client.25427 handle_command_reply: tid=0 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 client.25427 shutdown 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 msgr2=0x5648bea92c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea92c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c011d40 tx=0x7fdf2c0107a0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 shutdown_connections 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beaa65e0 0x5648beaa7a50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea95760 0x5648bea93140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea8aea0 0x5648bea92c00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 --2- 172.21.15.175:0/3400106388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8a580 0x5648bea926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf28c00640 0 client.25427 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:12.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 >> 172.21.15.175:0/3400106388 conn(0x5648bea86b10 msgr2=0x5648bea8c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:12.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 shutdown_connections 2024-07-29T00:37:12.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.577+0000 7fdf48417740 1 -- 172.21.15.175:0/3400106388 wait complete. 2024-07-29T00:37:12.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.578+0000 7fdf48417740 20 client.25427 trim_cache size 0 max 16384 2024-07-29T00:37:12.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 Processor -- start 2024-07-29T00:37:12.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 -- start start 2024-07-29T00:37:12.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beaa7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea92fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648beaa5d90 con 0x5648bea8a580 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648beaa6460 con 0x5648bea95760 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.597+0000 7fdf48417740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648beaa6b30 con 0x5648bea8aea0 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beaa7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beaa7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55320/0 (socket says 172.21.15.175:55320) 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3f400640 1 -- 172.21.15.175:0/528447274 learned_addr learned my addr 172.21.15.175:0/528447274 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3f400640 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea92fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 -- 172.21.15.175:0/528447274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 msgr2=0x5648bea92fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea92fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 -- 172.21.15.175:0/528447274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 msgr2=0x5648beaa7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beaa7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 -- 172.21.15.175:0/528447274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bea98440 con 0x5648bea8aea0 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3f400640 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beaa7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:12.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.598+0000 7fdf3e000640 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea86ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c01aa40 tx=0x7fdf2c010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8daac2e9d51c7280 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.599+0000 7fdf2b400640 1 -- 172.21.15.175:0/528447274 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c005bf0 con 0x5648bea8aea0 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.599+0000 7fdf2b400640 1 -- 172.21.15.175:0/528447274 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c0049c0 con 0x5648bea8aea0 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.599+0000 7fdf2b400640 1 -- 172.21.15.175:0/528447274 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0057e0 con 0x5648bea8aea0 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.599+0000 7fdf48417740 1 -- 172.21.15.175:0/528447274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 msgr2=0x5648bea86ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.599+0000 7fdf48417740 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea86ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c01aa40 tx=0x7fdf2c010ca0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 -- 172.21.15.175:0/528447274 shutdown_connections 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea92fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea86ef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 --2- 172.21.15.175:0/528447274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beaa7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 -- 172.21.15.175:0/528447274 >> 172.21.15.175:0/528447274 conn(0x5648bea86b10 msgr2=0x5648bea90ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 -- 172.21.15.175:0/528447274 shutdown_connections 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 -- 172.21.15.175:0/528447274 wait complete. 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 -- start start 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.600+0000 7fdf48417740 1 Processor -- start 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beab0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea8f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648beaa78d0 con 0x5648bea8a580 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648beaa7200 con 0x5648bea95760 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648beaa7fa0 con 0x5648bea8aea0 2024-07-29T00:37:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf48417740 10 client.0 _ll_register_callbacks cb 0x7fdf2c009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beab0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beab0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55328/0 (socket says 172.21.15.175:55328) 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf3f400640 1 -- 172.21.15.175:0/2052630144 learned_addr learned my addr 172.21.15.175:0/2052630144 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.601+0000 7fdf3e000640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea8f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 -- 172.21.15.175:0/2052630144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 msgr2=0x5648bea8f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea8f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 -- 172.21.15.175:0/2052630144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 msgr2=0x5648bea8f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:12.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea8f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 -- 172.21.15.175:0/2052630144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bea90090 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3e000640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea8f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf3f400640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beab0360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006010 tx=0x7fdf3802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519e8d33ba29378 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf28c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf28c00640 1 -- 172.21.15.175:0/2052630144 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38035070 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2c0049c0 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.602+0000 7fdf28c00640 1 -- 172.21.15.175:0/2052630144 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdf38054e80 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf28c00640 1 -- 172.21.15.175:0/2052630144 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf38064050 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 10 client.15333 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648beb10cf0 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf28c00640 1 -- 172.21.15.175:0/2052630144 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c0049c0 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf28c00640 1 -- 172.21.15.175:0/2052630144 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf38003710 con 0x5648bea8a580 2024-07-29T00:37:12.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 10 client.15333 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:12.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 10 client.15333 resolve_mds: resolved name 'g' to daemon mds.g 2024-07-29T00:37:12.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 20 client.15333 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:12.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beb152e0 0x5648beb13ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:12.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 4 client.15333 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:37:12.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.603+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "status"}) -- 0x5648be9f8210 con 0x5648beb152e0 2024-07-29T00:37:12.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.604+0000 7fdf3e000640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beb152e0 0x5648beb13ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:12.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.604+0000 7fdf3e000640 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beb152e0 0x5648beb13ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1fc794f4511d54e server_cookie=c3c4b1fa3df1b990 in_seq=0 out_seq=0 2024-07-29T00:37:12.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.604+0000 7fdf28c00640 10 client.15333 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:12.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf28c00640 1 -- 172.21.15.175:0/2052630144 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648be9f8210 con 0x5648beb152e0 2024-07-29T00:37:12.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf28c00640 10 client.15333 handle_command_reply: tid=0 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 client.15333 shutdown 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 msgr2=0x5648beab0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beab0360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006010 tx=0x7fdf3802a420 comp rx=0 tx=0).stop 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 shutdown_connections 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5648beb152e0 0x5648beb13ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648bea95760 0x5648bea8f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bea8aea0 0x5648bea8f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 --2- 172.21.15.175:0/2052630144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648bea8a580 0x5648beab0360 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf28c00640 0 client.15333 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 >> 172.21.15.175:0/2052630144 conn(0x5648bea86b10 msgr2=0x5648beaa0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.605+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 shutdown_connections 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf48417740 1 -- 172.21.15.175:0/2052630144 wait complete. 2024-07-29T00:37:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf48417740 20 client.15333 trim_cache size 0 max 16384 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 1, 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24515, 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 182.96617860800001, 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 29, 2024-07-29T00:37:12.609 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 92, 2024-07-29T00:37:12.610 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 92, 2024-07-29T00:37:12.610 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 216.95974347000001 2024-07-29T00:37:12.610 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:12.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf240a2e90 msgr2=0x7fdf240a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf240a2e90 0x7fdf240a5350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf300ccfc0 tx=0x7fdf300d34c0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 msgr2=0x7fdf4019e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 0x7fdf4019e510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000e9f0 tx=0x7fdf3000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.606+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 shutdown_connections 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf4010dd50 0x7fdf401a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdf240a2e90 0x7fdf240a5350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf40108670 0x7fdf4019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 --2- 172.21.15.175:0/3109995603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf40107d10 0x7fdf4019e510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 >> 172.21.15.175:0/3109995603 conn(0x7fdf40077670 msgr2=0x7fdf40078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 shutdown_connections 2024-07-29T00:37:12.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:12.607+0000 7fdf3fe00640 1 -- 172.21.15.175:0/3109995603 wait complete. 2024-07-29T00:37:13.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.b status 2024-07-29T00:37:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:13 smithi175 ceph-mon[32138]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:37:13.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:13 smithi204 ceph-mon[37368]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:37:13.915 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:13 smithi181 ceph-mon[36706]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 -- 172.21.15.175:0/1475888256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 msgr2=0x7f8d14103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 --2- 172.21.15.175:0/1475888256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d14103d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009930 tx=0x7f8cfc02a420 comp rx=0 tx=0).stop 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 -- 172.21.15.175:0/1475888256 shutdown_connections 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 --2- 172.21.15.175:0/1475888256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d14104260 0x7f8d14110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 --2- 172.21.15.175:0/1475888256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d14103d20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 --2- 172.21.15.175:0/1475888256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d141098c0 0x7f8d14109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 -- 172.21.15.175:0/1475888256 >> 172.21.15.175:0/1475888256 conn(0x7f8d140ff5c0 msgr2=0x7f8d141019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 -- 172.21.15.175:0/1475888256 shutdown_connections 2024-07-29T00:37:14.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.553+0000 7f8d19000640 1 -- 172.21.15.175:0/1475888256 wait complete. 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.554+0000 7f8d19000640 1 Processor -- start 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 -- start start 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d1419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d14104260 0x7f8d1419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d141098c0 0x7f8d1419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14113d10 con 0x7f8d141098c0 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14113b90 con 0x7f8d14104260 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d19000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14113e90 con 0x7f8d141038c0 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d1419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.555+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d14104260 0x7f8d1419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d1419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58814/0 (socket says 172.21.15.175:58814) 2024-07-29T00:37:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 -- 172.21.15.175:0/1428320904 learned_addr learned my addr 172.21.15.175:0/1428320904 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13e00640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d141098c0 0x7f8d1419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 -- 172.21.15.175:0/1428320904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d14104260 msgr2=0x7f8d1419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d14104260 0x7f8d1419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 -- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d141098c0 msgr2=0x7f8d1419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d141098c0 0x7f8d1419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 -- 172.21.15.175:0/1428320904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1419f1a0 con 0x7f8d141038c0 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13e00640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d141098c0 0x7f8d1419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.556+0000 7f8d13400640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d1419a1f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ea60 tx=0x7f8d0400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.557+0000 7f8d10c00640 1 -- 172.21.15.175:0/1428320904 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400cde0 con 0x7f8d141038c0 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.557+0000 7f8d10c00640 1 -- 172.21.15.175:0/1428320904 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d040035f0 con 0x7f8d141038c0 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.557+0000 7f8d10c00640 1 -- 172.21.15.175:0/1428320904 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04010670 con 0x7f8d141038c0 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.557+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1419f490 con 0x7f8d141038c0 2024-07-29T00:37:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.557+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a6d70 con 0x7f8d141038c0 2024-07-29T00:37:14.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.561+0000 7f8d10c00640 1 -- 172.21.15.175:0/1428320904 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f8d040037a0 con 0x7f8d141038c0 2024-07-29T00:37:14.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.561+0000 7f8d1cdfd740 1 Processor -- start 2024-07-29T00:37:14.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.561+0000 7f8d1cdfd740 1 -- start start 2024-07-29T00:37:14.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5df580 0x55e21d5df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5ea760 0x55e21d5ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e21d545790 con 0x55e21d5df580 2024-07-29T00:37:14.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e21d4dd880 con 0x55e21d5dfea0 2024-07-29T00:37:14.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e21d541750 con 0x55e21d5ea760 2024-07-29T00:37:14.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d13e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5df580 0x55e21d5df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d13400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d13e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5df580 0x55e21d5df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55352/0 (socket says 172.21.15.175:55352) 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d13400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5ea220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57750/0 (socket says 172.21.15.175:57750) 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d13e00640 1 -- 172.21.15.175:0/3645960831 learned_addr learned my addr 172.21.15.175:0/3645960831 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.562+0000 7f8d0be00640 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5ea760 0x55e21d5ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d13400640 1 -- 172.21.15.175:0/3645960831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5ea760 msgr2=0x55e21d5ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d13400640 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5ea760 0x55e21d5ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d13400640 1 -- 172.21.15.175:0/3645960831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5df580 msgr2=0x55e21d5df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d13400640 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5df580 0x55e21d5df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d13400640 1 -- 172.21.15.175:0/3645960831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21d2b72b0 con 0x55e21d5dfea0 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d13400640 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5ea220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d040ccf60 tx=0x7f8d040d5ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b4165c218243ce server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d10c00640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8cf80a2e90 0x7f8cf80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d08c00640 1 -- 172.21.15.175:0/3645960831 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040de070 con 0x55e21d5dfea0 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d08c00640 1 -- 172.21.15.175:0/3645960831 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0409c360 con 0x55e21d5dfea0 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d08c00640 1 -- 172.21.15.175:0/3645960831 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04003b30 con 0x55e21d5dfea0 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.563+0000 7f8d10c00640 1 -- 172.21.15.175:0/1428320904 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8d040cfa10 con 0x7f8d141038c0 2024-07-29T00:37:14.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d13e00640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8cf80a2e90 0x7f8cf80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/3645960831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 msgr2=0x55e21d5ea220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5ea220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d040ccf60 tx=0x7f8d040d5ef0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d13e00640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8cf80a2e90 0x7f8cf80a5350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0099a0 tx=0x7f8d0c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/3645960831 shutdown_connections 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5ea760 0x55e21d5ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5ea220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/3645960831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5df580 0x55e21d5df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.564+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/3645960831 >> 172.21.15.175:0/3645960831 conn(0x55e21d5dbb10 msgr2=0x55e21d5dbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.565+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/3645960831 shutdown_connections 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.565+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/3645960831 wait complete. 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.565+0000 7f8d1cdfd740 1 -- start start 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.565+0000 7f8d1cdfd740 1 Processor -- start 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5ed330 con 0x55e21d5ea760 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e21d500960 con 0x55e21d5dfea0 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5ed4b0 con 0x55e21d5df580 2024-07-29T00:37:14.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d1cdfd740 10 client.0 _ll_register_callbacks cb 0x55e21d2547b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5e7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57760/0 (socket says 172.21.15.175:57760) 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d0be00640 1 -- 172.21.15.175:0/2472634416 learned_addr learned my addr 172.21.15.175:0/2472634416 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d13e00640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.566+0000 7f8d13400640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.567+0000 7f8d13e00640 1 -- 172.21.15.175:0/2472634416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 msgr2=0x55e21d5e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.567+0000 7f8d13e00640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.567+0000 7f8d13e00640 1 -- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 msgr2=0x55e21d5e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.567+0000 7f8d13e00640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.567+0000 7f8d13e00640 1 -- 172.21.15.175:0/2472634416 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21d50fa90 con 0x55e21d5ea760 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d13e00640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e8050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005f40 tx=0x7f8d0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c383bbc2eea4dd server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d01600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d01600640 1 -- 172.21.15.175:0/2472634416 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c02c060 con 0x55e21d5ea760 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d01600640 1 -- 172.21.15.175:0/2472634416 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8d0c017030 con 0x55e21d5ea760 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0409c360 con 0x55e21d5ea760 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d01600640 1 -- 172.21.15.175:0/2472634416 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d0c01fc00 con 0x55e21d5ea760 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d1cdfd740 10 client.15345 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:14.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.568+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e21d5f8ec0 con 0x55e21d5ea760 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d01600640 1 -- 172.21.15.175:0/2472634416 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0409c360 con 0x55e21d5ea760 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d01600640 1 -- 172.21.15.175:0/2472634416 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8d0c0045a0 con 0x55e21d5ea760 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d1cdfd740 10 client.15345 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d1cdfd740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d1cdfd740 20 client.15345 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d5fb560 0x55e21d5fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d1cdfd740 4 client.15345 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.569+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21d552d00 con 0x55e21d5fb560 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.570+0000 7f8d13400640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d5fb560 0x55e21d5fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.570+0000 7f8d13400640 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d5fb560 0x55e21d5fc9d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af30a78b89cf91b7 server_cookie=a83c1581e0627756 in_seq=0 out_seq=0 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.570+0000 7f8d01600640 10 client.15345 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d01600640 1 -- 172.21.15.175:0/2472634416 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e21d552d00 con 0x55e21d5fb560 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d01600640 10 client.15345 handle_command_reply: tid=0 2024-07-29T00:37:14.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 client.15345 shutdown 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 msgr2=0x55e21d5e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e8050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005f40 tx=0x7f8d0c004ab0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 shutdown_connections 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d5fb560 0x55e21d5fc9d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e8050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2472634416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 >> 172.21.15.175:0/2472634416 conn(0x55e21d5dbb10 msgr2=0x55e21d5e1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d01600640 0 client.15345 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 shutdown_connections 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.571+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2472634416 wait complete. 2024-07-29T00:37:14.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.572+0000 7f8d1cdfd740 20 client.15345 trim_cache size 0 max 16384 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.590+0000 7f8d1cdfd740 1 Processor -- start 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 -- start start 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5fad10 con 0x55e21d5ea760 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5fb3e0 con 0x55e21d5dfea0 2024-07-29T00:37:14.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5fbab0 con 0x55e21d5df580 2024-07-29T00:37:14.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58842/0 (socket says 172.21.15.175:58842) 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d13400640 1 -- 172.21.15.175:0/2755158326 learned_addr learned my addr 172.21.15.175:0/2755158326 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.591+0000 7f8d0be00640 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13e00640 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13400640 1 -- 172.21.15.175:0/2755158326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 msgr2=0x55e21d5d6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13400640 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5d6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13400640 1 -- 172.21.15.175:0/2755158326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 msgr2=0x55e21d5e4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13400640 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13400640 1 -- 172.21.15.175:0/2755158326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21d5db940 con 0x55e21d5df580 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d13400640 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5fbc30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c760 tx=0x7f8d0409b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126c217ba86434bf server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d03e00640 1 -- 172.21.15.175:0/2755158326 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401ae30 con 0x55e21d5df580 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d03e00640 1 -- 172.21.15.175:0/2755158326 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d040ccea0 con 0x55e21d5df580 2024-07-29T00:37:14.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.592+0000 7f8d03e00640 1 -- 172.21.15.175:0/2755158326 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04008220 con 0x55e21d5df580 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2755158326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 msgr2=0x55e21d5fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5fbc30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c760 tx=0x7f8d0409b5c0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2755158326 shutdown_connections 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5e4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5d6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2755158326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5fbc30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2755158326 >> 172.21.15.175:0/2755158326 conn(0x55e21d5dbb10 msgr2=0x55e21d5e2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2755158326 shutdown_connections 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.593+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2755158326 wait complete. 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.594+0000 7f8d1cdfd740 1 -- start start 2024-07-29T00:37:14.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.594+0000 7f8d1cdfd740 1 Processor -- start 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.594+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d1cdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5daf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5fc850 con 0x55e21d5ea760 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5fc180 con 0x55e21d5dfea0 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d1cdfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e21d5fcf20 con 0x55e21d5df580 2024-07-29T00:37:14.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d1cdfd740 10 client.0 _ll_register_callbacks cb 0x7f8d0c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:14.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5da490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58844/0 (socket says 172.21.15.175:58844) 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 -- 172.21.15.175:0/2751954259 learned_addr learned my addr 172.21.15.175:0/2751954259 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d0be00640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13e00640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 -- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 msgr2=0x55e21d5da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 -- 172.21.15.175:0/2751954259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 msgr2=0x55e21d5daf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5daf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.595+0000 7f8d13400640 1 -- 172.21.15.175:0/2751954259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21d663eb0 con 0x55e21d5df580 2024-07-29T00:37:14.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d13400640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5da490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d0401f000 tx=0x7f8d040d5ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ee64c0d4783af0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d01600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d01600640 1 -- 172.21.15.175:0/2751954259 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040b4e60 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d01600640 1 -- 172.21.15.175:0/2751954259 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8d04099d00 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d040ccea0 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d01600640 1 -- 172.21.15.175:0/2751954259 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d040de070 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d1cdfd740 10 client.25447 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e21d6651b0 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d01600640 1 -- 172.21.15.175:0/2751954259 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d040ccea0 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d01600640 1 -- 172.21.15.175:0/2751954259 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8d040d9040 con 0x55e21d5df580 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d1cdfd740 10 client.25447 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.596+0000 7f8d1cdfd740 10 client.25447 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d1cdfd740 20 client.25447 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:14.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d6693a0 0x55e21d6680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d1cdfd740 4 client.25447 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "status"}) -- 0x55e21d54d210 con 0x55e21d6693a0 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d0be00640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d6693a0 0x55e21d6680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d0be00640 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d6693a0 0x55e21d6680b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61162ef3ea71d8b7 server_cookie=db4cbfc92a56fe2f in_seq=0 out_seq=0 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.597+0000 7f8d01600640 10 client.25447 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d01600640 1 -- 172.21.15.175:0/2751954259 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e21d54d210 con 0x55e21d6693a0 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d01600640 10 client.25447 handle_command_reply: tid=0 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 client.25447 shutdown 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 msgr2=0x55e21d5da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5da490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d0401f000 tx=0x7f8d040d5ef0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 shutdown_connections 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e21d6693a0 0x55e21d6680b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d5ea760 0x55e21d5daf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e21d5dfea0 0x55e21d5da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 --2- 172.21.15.175:0/2751954259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e21d5df580 0x55e21d5da490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d01600640 0 client.25447 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 >> 172.21.15.175:0/2751954259 conn(0x55e21d5dbb10 msgr2=0x55e21d5fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 shutdown_connections 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 1 -- 172.21.15.175:0/2751954259 wait complete. 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.598+0000 7f8d1cdfd740 20 client.25447 trim_cache size 0 max 16384 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "cluster_fsid": "f484ff76-4d40-11ef-bcc8-c7b262605968", 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "whoami": 0, 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 24539, 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "want_state": "up:active", 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "state": "up:active", 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs", 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "rank_uptime": 195.86039441400001, 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "mdsmap_epoch": 29, 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch": 92, 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_epoch_barrier": 92, 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "uptime": 212.17638534899999 2024-07-29T00:37:14.604 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8cf80a2e90 msgr2=0x7f8cf80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8cf80a2e90 0x7f8cf80a5350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0099a0 tx=0x7f8d0c009500 comp rx=0 tx=0).stop 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 msgr2=0x7f8d1419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d1419a1f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ea60 tx=0x7f8d0400ef20 comp rx=0 tx=0).stop 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 shutdown_connections 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8cf80a2e90 0x7f8cf80a5350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d141098c0 0x7f8d1419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d14104260 0x7f8d1419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 --2- 172.21.15.175:0/1428320904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d141038c0 0x7f8d1419a1f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:14.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 >> 172.21.15.175:0/1428320904 conn(0x7f8d140ff5c0 msgr2=0x7f8d140ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:14.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 shutdown_connections 2024-07-29T00:37:14.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:14.599+0000 7f8d19000640 1 -- 172.21.15.175:0/1428320904 wait complete. 2024-07-29T00:37:15.420 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-07-29T00:37:15.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:15 smithi175 ceph-mon[32138]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-07-29T00:37:15.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:15 smithi204 ceph-mon[37368]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-07-29T00:37:15.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:15 smithi181 ceph-mon[36706]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-07-29T00:37:16.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:37:16.751 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 -- 172.21.15.175:0/1308130503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85981076e0 msgr2=0x7f8598107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 --2- 172.21.15.175:0/1308130503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85981076e0 0x7f8598107ac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8588009930 tx=0x7f858802a420 comp rx=0 tx=0).stop 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 -- 172.21.15.175:0/1308130503 shutdown_connections 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 --2- 172.21.15.175:0/1308130503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8598102080 0x7f859810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 --2- 172.21.15.175:0/1308130503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f8598101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 --2- 172.21.15.175:0/1308130503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85981076e0 0x7f8598107ac0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 -- 172.21.15.175:0/1308130503 >> 172.21.15.175:0/1308130503 conn(0x7f85980fd420 msgr2=0x7f85980ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 -- 172.21.15.175:0/1308130503 shutdown_connections 2024-07-29T00:37:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.431+0000 7f859d600640 1 -- 172.21.15.175:0/1308130503 wait complete. 2024-07-29T00:37:17.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 Processor -- start 2024-07-29T00:37:17.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 -- start start 2024-07-29T00:37:17.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f85980741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8598102080 0x7f8598074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85981076e0 0x7f8598074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8598111b30 con 0x7f85981076e0 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85981119b0 con 0x7f85981016e0 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.433+0000 7f859d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8598111cb0 con 0x7f8598102080 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8597400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85981076e0 0x7f8598074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f85980741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8597400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85981076e0 0x7f8598074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55392/0 (socket says 172.21.15.175:55392) 2024-07-29T00:37:17.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8597400640 1 -- 172.21.15.175:0/3514169433 learned_addr learned my addr 172.21.15.175:0/3514169433 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:17.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f85980741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57794/0 (socket says 172.21.15.175:57794) 2024-07-29T00:37:17.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596000640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8598102080 0x7f8598074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 -- 172.21.15.175:0/3514169433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8598102080 msgr2=0x7f8598074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8598102080 0x7f8598074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 -- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85981076e0 msgr2=0x7f8598074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85981076e0 0x7f8598074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 -- 172.21.15.175:0/3514169433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a6470 con 0x7f85981016e0 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596000640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8598102080 0x7f8598074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.434+0000 7f8596a00640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f85980741e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f858802a940 tx=0x7f8588003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.435+0000 7f858f400640 1 -- 172.21.15.175:0/3514169433 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858803f070 con 0x7f85981016e0 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.435+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a6700 con 0x7f85981016e0 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.435+0000 7f858f400640 1 -- 172.21.15.175:0/3514169433 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8588031c90 con 0x7f85981016e0 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.435+0000 7f858f400640 1 -- 172.21.15.175:0/3514169433 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588035030 con 0x7f85981016e0 2024-07-29T00:37:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.435+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a6be0 con 0x7f85981016e0 2024-07-29T00:37:17.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.438+0000 7f85a0058740 1 Processor -- start 2024-07-29T00:37:17.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.438+0000 7f858f400640 1 -- 172.21.15.175:0/3514169433 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f8588042070 con 0x7f85981016e0 2024-07-29T00:37:17.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.438+0000 7f85a0058740 1 -- start start 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.438+0000 7f85a0058740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 0x5641b9a88ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.438+0000 7f85a0058740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a89430 0x5641b9a937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f85a0058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f85a0058740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641b99edad0 con 0x5641b9a88b10 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f85a0058740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5641b9985af0 con 0x5641b9a89430 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f85a0058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5641b99e9a90 con 0x5641b9a93cf0 2024-07-29T00:37:17.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 0x5641b9a88ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 0x5641b9a88ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55396/0 (socket says 172.21.15.175:55396) 2024-07-29T00:37:17.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 -- 172.21.15.175:0/3655455872 learned_addr learned my addr 172.21.15.175:0/3655455872 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:17.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8596000640 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8596a00640 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a89430 0x5641b9a937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 -- 172.21.15.175:0/3655455872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 msgr2=0x5641b9a960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 -- 172.21.15.175:0/3655455872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a89430 msgr2=0x5641b9a937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a89430 0x5641b9a937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.439+0000 7f8597400640 1 -- 172.21.15.175:0/3655455872 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b975f3c0 con 0x5641b9a88b10 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f8596a00640 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a89430 0x5641b9a937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f8597400640 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 0x5641b9a88ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8590011470 tx=0x7f8590011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54735ccb720efee server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f858f400640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f857c0a2e10 0x7f857c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f858f400640 1 -- 172.21.15.175:0/3514169433 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f85880eff10 con 0x7f85981016e0 2024-07-29T00:37:17.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f8583400640 1 -- 172.21.15.175:0/3655455872 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590004b20 con 0x5641b9a88b10 2024-07-29T00:37:17.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f8583400640 1 -- 172.21.15.175:0/3655455872 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8590004cc0 con 0x5641b9a88b10 2024-07-29T00:37:17.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f8596a00640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f857c0a2e10 0x7f857c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.440+0000 7f8583400640 1 -- 172.21.15.175:0/3655455872 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590005450 con 0x5641b9a88b10 2024-07-29T00:37:17.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 -- 172.21.15.175:0/3655455872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 msgr2=0x5641b9a88ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f8596a00640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f857c0a2e10 0x7f857c0a52d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f85880ed4c0 tx=0x7f858802a420 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:17.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 0x5641b9a88ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8590011470 tx=0x7f8590011930 comp rx=0 tx=0).stop 2024-07-29T00:37:17.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 -- 172.21.15.175:0/3655455872 shutdown_connections 2024-07-29T00:37:17.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a89430 0x5641b9a937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 --2- 172.21.15.175:0/3655455872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a88b10 0x5641b9a88ef0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.441+0000 7f85a0058740 1 -- 172.21.15.175:0/3655455872 >> 172.21.15.175:0/3655455872 conn(0x5641b9a850a0 msgr2=0x5641b9a85480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.442+0000 7f85a0058740 1 -- 172.21.15.175:0/3655455872 shutdown_connections 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.442+0000 7f85a0058740 1 -- 172.21.15.175:0/3655455872 wait complete. 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.442+0000 7f85a0058740 1 -- start start 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.442+0000 7f85a0058740 1 Processor -- start 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a91060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641b9a968c0 con 0x5641b9a89430 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5641b99a8bc0 con 0x5641b9a88b10 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5641b9a96a40 con 0x5641b9a93cf0 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f85a0058740 10 client.0 _ll_register_callbacks cb 0x5641b9994d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:17.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f8597400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f8596000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a91060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f8597400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a90b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57808/0 (socket says 172.21.15.175:57808) 2024-07-29T00:37:17.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f8597400640 1 -- 172.21.15.175:0/3528420618 learned_addr learned my addr 172.21.15.175:0/3528420618 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:17.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.443+0000 7f8597400640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8596000640 1 -- 172.21.15.175:0/3528420618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 msgr2=0x5641b9a915a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8596000640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a915a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8596000640 1 -- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 msgr2=0x5641b9a90b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8596000640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a90b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8596000640 1 -- 172.21.15.175:0/3528420618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b9a9fd30 con 0x5641b9a89430 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8597400640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8596000640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a91060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8584012cf0 tx=0x7f858400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5614a2b971669e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8577e00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8577e00640 1 -- 172.21.15.175:0/3528420618 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584012110 con 0x5641b9a89430 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f8577e00640 1 -- 172.21.15.175:0/3528420618 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8584004050 con 0x5641b9a89430 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.444+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8590004cc0 con 0x5641b9a89430 2024-07-29T00:37:17.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f8577e00640 1 -- 172.21.15.175:0/3528420618 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85840122b0 con 0x5641b9a89430 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f85a0058740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641b9aa26f0 con 0x5641b9a89430 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f8577e00640 1 -- 172.21.15.175:0/3528420618 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8590004cc0 con 0x5641b9a89430 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f8577e00640 1 -- 172.21.15.175:0/3528420618 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8584010550 con 0x5641b9a89430 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f85a0058740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f85a0058740 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.445+0000 7f85a0058740 20 client.15363 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.446+0000 7f85a0058740 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9aa3bb0 0x5641b9aa5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.446+0000 7f85a0058740 4 client.15363 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.446+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b99fb040 con 0x5641b9aa3bb0 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.446+0000 7f8597400640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9aa3bb0 0x5641b9aa5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.447+0000 7f8597400640 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9aa3bb0 0x5641b9aa5fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74fc7af383e7e521 server_cookie=e73f13bc9f58e953 in_seq=0 out_seq=0 2024-07-29T00:37:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.447+0000 7f8577e00640 10 client.15363 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:17.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.447+0000 7f8577e00640 1 -- 172.21.15.175:0/3528420618 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5641b99fb040 con 0x5641b9aa3bb0 2024-07-29T00:37:17.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.447+0000 7f8577e00640 10 client.15363 handle_command_reply: tid=0 2024-07-29T00:37:17.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.447+0000 7f85a0058740 1 client.15363 shutdown 2024-07-29T00:37:17.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 msgr2=0x5641b9a91060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a91060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8584012cf0 tx=0x7f858400f790 comp rx=0 tx=0).stop 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 shutdown_connections 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9aa3bb0 0x5641b9aa5fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a915a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a91060 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 --2- 172.21.15.175:0/3528420618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f8577e00640 0 client.15363 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 >> 172.21.15.175:0/3528420618 conn(0x5641b9a850a0 msgr2=0x5641b9a84200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 shutdown_connections 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 1 -- 172.21.15.175:0/3528420618 wait complete. 2024-07-29T00:37:17.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.448+0000 7f85a0058740 20 client.15363 trim_cache size 0 max 16384 2024-07-29T00:37:17.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 Processor -- start 2024-07-29T00:37:17.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 -- start start 2024-07-29T00:37:17.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a92a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85981af6e0 con 0x5641b9a89430 2024-07-29T00:37:17.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85981b1220 con 0x5641b9a88b10 2024-07-29T00:37:17.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.467+0000 7f85a0058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85981afdb0 con 0x5641b9a93cf0 2024-07-29T00:37:17.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a8c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57810/0 (socket says 172.21.15.175:57810) 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 -- 172.21.15.175:0/1843927564 learned_addr learned my addr 172.21.15.175:0/1843927564 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8596000640 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a92a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 -- 172.21.15.175:0/1843927564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 msgr2=0x5641b9a92a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a92a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 -- 172.21.15.175:0/1843927564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 msgr2=0x5641b9a913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a913f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 -- 172.21.15.175:0/1843927564 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b9a85d30 con 0x5641b9a88b10 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8596000640 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.468+0000 7f8597400640 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a8c290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8590009480 tx=0x7f859000ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a13d7e4f15f574 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:17.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.469+0000 7f8582a00640 1 -- 172.21.15.175:0/1843927564 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590005bd0 con 0x5641b9a88b10 2024-07-29T00:37:17.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.469+0000 7f8582a00640 1 -- 172.21.15.175:0/1843927564 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8590028450 con 0x5641b9a88b10 2024-07-29T00:37:17.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.469+0000 7f8582a00640 1 -- 172.21.15.175:0/1843927564 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590004fc0 con 0x5641b9a88b10 2024-07-29T00:37:17.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.469+0000 7f85a0058740 1 -- 172.21.15.175:0/1843927564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 msgr2=0x5641b9a8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.469+0000 7f85a0058740 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a8c290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8590009480 tx=0x7f859000ffd0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 -- 172.21.15.175:0/1843927564 shutdown_connections 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a93cf0 0x5641b9a92a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a89430 0x5641b9a913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 --2- 172.21.15.175:0/1843927564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a8c290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 -- 172.21.15.175:0/1843927564 >> 172.21.15.175:0/1843927564 conn(0x5641b9a850a0 msgr2=0x5641b9a92380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 -- 172.21.15.175:0/1843927564 shutdown_connections 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 -- 172.21.15.175:0/1843927564 wait complete. 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 -- start start 2024-07-29T00:37:17.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.470+0000 7f85a0058740 1 Processor -- start 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a89430 0x5641b9a975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a93cf0 0x5641b9a97af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85981b0b50 con 0x5641b9a93cf0 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85981b0480 con 0x5641b9a88b10 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85981ad6c0 con 0x5641b9a89430 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f85a0058740 10 client.0 _ll_register_callbacks cb 0x7f85840127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f8597400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f8597400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a9c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57820/0 (socket says 172.21.15.175:57820) 2024-07-29T00:37:17.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f8597400640 1 -- 172.21.15.175:0/1694072996 learned_addr learned my addr 172.21.15.175:0/1694072996 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f8596000640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a89430 0x5641b9a975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.471+0000 7f8597400640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a93cf0 0x5641b9a97af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8596000640 1 -- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 msgr2=0x5641b9a9c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8596000640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a9c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8596000640 1 -- 172.21.15.175:0/1694072996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a93cf0 msgr2=0x5641b9a97af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8596000640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a93cf0 0x5641b9a97af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8596000640 1 -- 172.21.15.175:0/1694072996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b9a98190 con 0x5641b9a89430 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8597400640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a9c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8596000640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a89430 0x5641b9a975b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8584010970 tx=0x7f8584010d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=358334891bd004ca server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.472+0000 7f8577e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:17.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f8577e00640 1 -- 172.21.15.175:0/1694072996 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584032060 con 0x5641b9a89430 2024-07-29T00:37:17.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8590028450 con 0x5641b9a89430 2024-07-29T00:37:17.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f8577e00640 1 -- 172.21.15.175:0/1694072996 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8584010430 con 0x5641b9a89430 2024-07-29T00:37:17.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f8577e00640 1 -- 172.21.15.175:0/1694072996 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8584029040 con 0x5641b9a89430 2024-07-29T00:37:17.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 10 client.25465 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:17.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641b9b0e760 con 0x5641b9a89430 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f8577e00640 1 -- 172.21.15.175:0/1694072996 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8590028450 con 0x5641b9a89430 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f8577e00640 1 -- 172.21.15.175:0/1694072996 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f858402c070 con 0x5641b9a89430 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 10 client.25465 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 20 client.25465 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9b0ed60 0x5641b9b11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 4 client.25465 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.473+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641b99f5550 con 0x5641b9b0ed60 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.474+0000 7f8597400640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9b0ed60 0x5641b9b11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.474+0000 7f8597400640 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9b0ed60 0x5641b9b11150 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2e517dcd1a6c67 server_cookie=13d947236c3610c6 in_seq=0 out_seq=0 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.474+0000 7f8577e00640 10 client.25465 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f8577e00640 1 -- 172.21.15.175:0/1694072996 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5641b99f5550 con 0x5641b9b0ed60 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f8577e00640 10 client.25465 handle_command_reply: tid=0 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 client.25465 shutdown 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a89430 msgr2=0x5641b9a975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a89430 0x5641b9a975b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8584010970 tx=0x7f8584010d90 comp rx=0 tx=0).stop 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 shutdown_connections 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5641b9b0ed60 0x5641b9b11150 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641b9a93cf0 0x5641b9a97af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f8577e00640 0 client.25465 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641b9a89430 0x5641b9a975b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 --2- 172.21.15.175:0/1694072996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641b9a88b10 0x5641b9a9c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 >> 172.21.15.175:0/1694072996 conn(0x5641b9a850a0 msgr2=0x5641b9a84200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 shutdown_connections 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 1 -- 172.21.15.175:0/1694072996 wait complete. 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.475+0000 7f85a0058740 20 client.25465 trim_cache size 0 max 16384 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:37:17.479 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f857c0a2e10 msgr2=0x7f857c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f857c0a2e10 0x7f857c0a52d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f85880ed4c0 tx=0x7f858802a420 comp rx=0 tx=0).stop 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 msgr2=0x7f85980741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f85980741e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f858802a940 tx=0x7f8588003b20 comp rx=0 tx=0).stop 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 shutdown_connections 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f857c0a2e10 0x7f857c0a52d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85981076e0 0x7f8598074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8598102080 0x7f8598074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 --2- 172.21.15.175:0/3514169433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85981016e0 0x7f85980741e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:17.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.476+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 >> 172.21.15.175:0/3514169433 conn(0x7f85980fd420 msgr2=0x7f85980fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:17.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.477+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 shutdown_connections 2024-07-29T00:37:17.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:17.477+0000 7f859d600640 1 -- 172.21.15.175:0/3514169433 wait complete. 2024-07-29T00:37:17.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:17 smithi204 ceph-mon[37368]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-29T00:37:17.975 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:17 smithi175 ceph-mon[32138]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-29T00:37:17.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:17 smithi181 ceph-mon[36706]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-29T00:37:18.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:37:18.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:37:19.105 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 -- 172.21.15.175:0/2979440887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 msgr2=0x7f1f3c101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2979440887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c101b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009860 tx=0x7f1f2802a340 comp rx=0 tx=0).stop 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 -- 172.21.15.175:0/2979440887 shutdown_connections 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2979440887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 0x7f1f3c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2979440887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c101b20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2979440887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f3c1076c0 0x7f1f3c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.738+0000 7f1f3b200640 1 -- 172.21.15.175:0/2979440887 >> 172.21.15.175:0/2979440887 conn(0x7f1f3c0fd400 msgr2=0x7f1f3c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:19.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.739+0000 7f1f3b200640 1 -- 172.21.15.175:0/2979440887 shutdown_connections 2024-07-29T00:37:19.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.739+0000 7f1f3b200640 1 -- 172.21.15.175:0/2979440887 wait complete. 2024-07-29T00:37:19.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 Processor -- start 2024-07-29T00:37:19.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 -- start start 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c1085f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 0x7f1f3c108b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f3c1076c0 0x7f1f3c109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111ac0 con 0x7f1f3c102060 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111940 con 0x7f1f3c1016c0 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.740+0000 7f1f3b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111c40 con 0x7f1f3c1076c0 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c1085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 0x7f1f3c108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c1085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57554/0 (socket says 172.21.15.175:57554) 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39e00640 1 -- 172.21.15.175:0/2251874445 learned_addr learned my addr 172.21.15.175:0/2251874445 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:19.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f3a800640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f3c1076c0 0x7f1f3c109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:19.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 -- 172.21.15.175:0/2251874445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f3c1076c0 msgr2=0x7f1f3c109070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:19.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f3c1076c0 0x7f1f3c109070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 -- 172.21.15.175:0/2251874445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 msgr2=0x7f1f3c1085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:19.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c1085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 -- 172.21.15.175:0/2251874445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c1aa8b0 con 0x7f1f3c102060 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.741+0000 7f1f39400640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 0x7f1f3c108b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009780 tx=0x7f1f28033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.742+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28037040 con 0x7f1f3c102060 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.742+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c1aab40 con 0x7f1f3c102060 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.742+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f28040070 con 0x7f1f3c102060 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.742+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f280039f0 con 0x7f1f3c102060 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.742+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1ab020 con 0x7f1f3c102060 2024-07-29T00:37:19.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.743+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c005290 con 0x7f1f3c102060 2024-07-29T00:37:19.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.745+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f1f28003b90 con 0x7f1f3c102060 2024-07-29T00:37:19.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.746+0000 7f1f32a00640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1f200a2e10 0x7f1f200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:19.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.747+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1f280ef3f0 con 0x7f1f3c102060 2024-07-29T00:37:19.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.747+0000 7f1f39e00640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1f200a2e10 0x7f1f200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:19.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.747+0000 7f1f39e00640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1f200a2e10 0x7f1f200a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009c50 tx=0x7f1f2c005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:19.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.751+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1f280bbe50 con 0x7f1f3c102060 2024-07-29T00:37:19.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:19 smithi204 ceph-mon[37368]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:19.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.978+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f1c005850 con 0x7f1f3c102060 2024-07-29T00:37:19.979 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:19 smithi175 ceph-mon[32138]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:19.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.980+0000 7f1f32a00640 1 -- 172.21.15.175:0/2251874445 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f1f280bb750 con 0x7f1f3c102060 2024-07-29T00:37:19.980 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:37:19.980 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:37:19.981 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:37:19.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.982+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1f200a2e10 msgr2=0x7f1f200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:19.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.982+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1f200a2e10 0x7f1f200a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009c50 tx=0x7f1f2c005f90 comp rx=0 tx=0).stop 2024-07-29T00:37:19.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.982+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 msgr2=0x7f1f3c108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:19.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.982+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 0x7f1f3c108b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009780 tx=0x7f1f28033b60 comp rx=0 tx=0).stop 2024-07-29T00:37:19.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 shutdown_connections 2024-07-29T00:37:19.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f3c1076c0 0x7f1f3c109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f1f200a2e10 0x7f1f200a52d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c102060 0x7f1f3c108b30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 --2- 172.21.15.175:0/2251874445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c1016c0 0x7f1f3c1085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:19.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 >> 172.21.15.175:0/2251874445 conn(0x7f1f3c0fd400 msgr2=0x7f1f3c10cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:19.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 shutdown_connections 2024-07-29T00:37:19.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:19.983+0000 7f1f3b200640 1 -- 172.21.15.175:0/2251874445 wait complete. 2024-07-29T00:37:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:19 smithi181 ceph-mon[36706]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:20.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:20 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2251874445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:20.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:20 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2251874445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:20.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:37:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:20 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2251874445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:21.351 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:21.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:21 smithi175 ceph-mon[32138]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:21 smithi204 ceph-mon[37368]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 -- 172.21.15.175:0/286624244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85401005c0 msgr2=0x7f85401009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 --2- 172.21.15.175:0/286624244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85401005c0 0x7f85401009a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f853002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 -- 172.21.15.175:0/286624244 shutdown_connections 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 --2- 172.21.15.175:0/286624244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854010e170 0x7f8540110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 --2- 172.21.15.175:0/286624244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8540100f70 0x7f854010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 --2- 172.21.15.175:0/286624244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85401005c0 0x7f85401009a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.986+0000 7f8545a00640 1 -- 172.21.15.175:0/286624244 >> 172.21.15.175:0/286624244 conn(0x7f85400fc490 msgr2=0x7f85400fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.987+0000 7f8545a00640 1 -- 172.21.15.175:0/286624244 shutdown_connections 2024-07-29T00:37:21.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.987+0000 7f8545a00640 1 -- 172.21.15.175:0/286624244 wait complete. 2024-07-29T00:37:21.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.987+0000 7f8545a00640 1 Processor -- start 2024-07-29T00:37:21.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 -- start start 2024-07-29T00:37:21.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85401005c0 0x7f854019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 0x7f854019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f854010e170 0x7f854019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8540113ec0 con 0x7f854010e170 2024-07-29T00:37:21.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8540113d40 con 0x7f8540100f70 2024-07-29T00:37:21.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8545a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8540114040 con 0x7f85401005c0 2024-07-29T00:37:21.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.988+0000 7f8536000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 0x7f854019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 0x7f854019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57558/0 (socket says 172.21.15.175:57558) 2024-07-29T00:37:21.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 -- 172.21.15.175:0/3083958153 learned_addr learned my addr 172.21.15.175:0/3083958153 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f853ea00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85401005c0 0x7f854019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f853f400640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f854010e170 0x7f854019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 -- 172.21.15.175:0/3083958153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85401005c0 msgr2=0x7f854019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85401005c0 0x7f854019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 -- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f854010e170 msgr2=0x7f854019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f854010e170 0x7f854019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 -- 172.21.15.175:0/3083958153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854019f300 con 0x7f8540100f70 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f853f400640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f854010e170 0x7f854019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.989+0000 7f8536000640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 0x7f854019a890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f852800d950 tx=0x7f852800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.990+0000 7f8537e00640 1 -- 172.21.15.175:0/3083958153 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528017030 con 0x7f8540100f70 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.990+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854019f5f0 con 0x7f8540100f70 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.990+0000 7f8537e00640 1 -- 172.21.15.175:0/3083958153 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8528004020 con 0x7f8540100f70 2024-07-29T00:37:21.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.990+0000 7f8537e00640 1 -- 172.21.15.175:0/3083958153 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280047a0 con 0x7f8540100f70 2024-07-29T00:37:21.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.990+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a6ed0 con 0x7f8540100f70 2024-07-29T00:37:21.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.993+0000 7f8537e00640 1 -- 172.21.15.175:0/3083958153 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f85280041d0 con 0x7f8540100f70 2024-07-29T00:37:21.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 Processor -- start 2024-07-29T00:37:21.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 -- start start 2024-07-29T00:37:21.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c81dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c8eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555773be7ab0 con 0x555773c82ef0 2024-07-29T00:37:21.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555773b7fa80 con 0x555773c8c720 2024-07-29T00:37:21.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.994+0000 7f854847d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555773be3a70 con 0x555773c81980 2024-07-29T00:37:21.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c81440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41348/0 (socket says 172.21.15.175:41348) 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 -- 172.21.15.175:0/4120152512 learned_addr learned my addr 172.21.15.175:0/4120152512 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853ea00640 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c81dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f8537e00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f85240a2e10 0x7f85240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f8537e00640 1 -- 172.21.15.175:0/3083958153 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f85280cf060 con 0x7f8540100f70 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f8536000640 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c8eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 -- 172.21.15.175:0/4120152512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 msgr2=0x555773c81dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c81dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 -- 172.21.15.175:0/4120152512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 msgr2=0x555773c8eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.995+0000 7f853f400640 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c8eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f853f400640 1 -- 172.21.15.175:0/4120152512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557739593c0 con 0x555773c82ef0 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f853f400640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f85240a2e10 0x7f85240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f853f400640 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c81440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f85380104d0 tx=0x7f8538010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72b716b0d9b4298 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:21.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f8536000640 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c8eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:37:21.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f852fe00640 1 -- 172.21.15.175:0/4120152512 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538019070 con 0x555773c82ef0 2024-07-29T00:37:21.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f852fe00640 1 -- 172.21.15.175:0/4120152512 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538022070 con 0x555773c82ef0 2024-07-29T00:37:21.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.996+0000 7f852fe00640 1 -- 172.21.15.175:0/4120152512 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538005130 con 0x555773c82ef0 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 -- 172.21.15.175:0/4120152512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 msgr2=0x555773c81440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c81440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f85380104d0 tx=0x7f8538010990 comp rx=0 tx=0).stop 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f853f400640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f85240a2e10 0x7f85240a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8538021000 tx=0x7f853801f000 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 -- 172.21.15.175:0/4120152512 shutdown_connections 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c8eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c81dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 --2- 172.21.15.175:0/4120152512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c81440 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 -- 172.21.15.175:0/4120152512 >> 172.21.15.175:0/4120152512 conn(0x555773c4a950 msgr2=0x555773c7e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.997+0000 7f854847d740 1 -- 172.21.15.175:0/4120152512 shutdown_connections 2024-07-29T00:37:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.998+0000 7f854847d740 1 -- 172.21.15.175:0/4120152512 wait complete. 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.998+0000 7f854847d740 1 -- start start 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.998+0000 7f854847d740 1 Processor -- start 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773c89be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555773c8f4c0 con 0x555773c8c720 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555773ba2c10 con 0x555773c82ef0 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555773c8f640 con 0x555773c81980 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f854847d740 10 client.0 _ll_register_callbacks cb 0x555773b6ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f853ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f853ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c896a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44112/0 (socket says 172.21.15.175:44112) 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f853ea00640 1 -- 172.21.15.175:0/119442054 learned_addr learned my addr 172.21.15.175:0/119442054 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:21.999+0000 7f8536000640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773c89be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853f400640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853ea00640 1 -- 172.21.15.175:0/119442054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 msgr2=0x555773c89be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853ea00640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773c89be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853ea00640 1 -- 172.21.15.175:0/119442054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 msgr2=0x555773c8a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853ea00640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853ea00640 1 -- 172.21.15.175:0/119442054 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555773ba61e0 con 0x555773c81980 2024-07-29T00:37:22.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853f400640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:37:22.003 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:21 smithi181 ceph-mon[36706]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f853ea00640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c896a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f8530004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4c589f7d6c948b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f852cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f852cc00640 1 -- 172.21.15.175:0/119442054 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530035040 con 0x555773c81980 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.000+0000 7f852cc00640 1 -- 172.21.15.175:0/119442054 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8530005450 con 0x555773c81980 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.001+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8538022070 con 0x555773c81980 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.001+0000 7f852cc00640 1 -- 172.21.15.175:0/119442054 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8530005650 con 0x555773c81980 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.001+0000 7f854847d740 10 client.25483 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.001+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555773c9b1d0 con 0x555773c81980 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.001+0000 7f852cc00640 1 -- 172.21.15.175:0/119442054 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538022070 con 0x555773c81980 2024-07-29T00:37:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f852cc00640 1 -- 172.21.15.175:0/119442054 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8530003bd0 con 0x555773c81980 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f854847d740 10 client.25483 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f854847d740 10 client.25483 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f854847d740 20 client.25483 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f854847d740 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773c9c690 0x555773c9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f854847d740 4 client.25483 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555773bf5020 con 0x555773c9c690 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.002+0000 7f8536000640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773c9c690 0x555773c9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.003+0000 7f8536000640 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773c9c690 0x555773c9eb10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ab4dcd7e54b0d8b server_cookie=53aa11fd365f4832 in_seq=0 out_seq=0 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.003+0000 7f852cc00640 10 client.25483 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.003+0000 7f852cc00640 1 -- 172.21.15.175:0/119442054 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555773bf5020 con 0x555773c9c690 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.003+0000 7f852cc00640 10 client.25483 handle_command_reply: tid=0 2024-07-29T00:37:22.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.003+0000 7f854847d740 1 client.25483 shutdown 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 msgr2=0x555773c896a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c896a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f8530004440 comp rx=0 tx=0).stop 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 shutdown_connections 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773c9c690 0x555773c9eb10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773c89be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 --2- 172.21.15.175:0/119442054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c896a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 >> 172.21.15.175:0/119442054 conn(0x555773c4a950 msgr2=0x555773c83ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f852cc00640 0 client.25483 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 shutdown_connections 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 1 -- 172.21.15.175:0/119442054 wait complete. 2024-07-29T00:37:22.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.004+0000 7f854847d740 20 client.25483 trim_cache size 0 max 16384 2024-07-29T00:37:22.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 Processor -- start 2024-07-29T00:37:22.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 -- start start 2024-07-29T00:37:22.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773ca43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773ca4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00e850 con 0x555773c8c720 2024-07-29T00:37:22.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f851c010390 con 0x555773c82ef0 2024-07-29T00:37:22.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.020+0000 7f854847d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00ef20 con 0x555773c81980 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773ca43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773ca43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44128/0 (socket says 172.21.15.175:44128) 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 -- 172.21.15.175:0/4119474830 learned_addr learned my addr 172.21.15.175:0/4119474830 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853f400640 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f8536000640 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773ca4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 -- 172.21.15.175:0/4119474830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 msgr2=0x555773ca4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773ca4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 -- 172.21.15.175:0/4119474830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 msgr2=0x555773c8af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:22.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 -- 172.21.15.175:0/4119474830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555773c844d0 con 0x555773c81980 2024-07-29T00:37:22.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.021+0000 7f853ea00640 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773ca43e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8530009850 tx=0x7f853002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8644b7a8f9dae71f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:22.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.022+0000 7f852f400640 1 -- 172.21.15.175:0/4119474830 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530035030 con 0x555773c81980 2024-07-29T00:37:22.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.022+0000 7f852f400640 1 -- 172.21.15.175:0/4119474830 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8530041070 con 0x555773c81980 2024-07-29T00:37:22.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.022+0000 7f852f400640 1 -- 172.21.15.175:0/4119474830 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530053070 con 0x555773c81980 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.022+0000 7f854847d740 1 -- 172.21.15.175:0/4119474830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 msgr2=0x555773ca43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.022+0000 7f854847d740 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773ca43e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8530009850 tx=0x7f853002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 -- 172.21.15.175:0/4119474830 shutdown_connections 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c8c720 0x555773c8af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c82ef0 0x555773ca4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 --2- 172.21.15.175:0/4119474830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773ca43e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 -- 172.21.15.175:0/4119474830 >> 172.21.15.175:0/4119474830 conn(0x555773c4a950 msgr2=0x555773c84a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 -- 172.21.15.175:0/4119474830 shutdown_connections 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 -- 172.21.15.175:0/4119474830 wait complete. 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.023+0000 7f854847d740 1 -- start start 2024-07-29T00:37:22.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 Processor -- start 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c95070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00fcc0 con 0x555773c82ef0 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00f5f0 con 0x555773c8c720 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00c830 con 0x555773c81980 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f854847d740 10 client.0 _ll_register_callbacks cb 0x7f8530009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.024+0000 7f8536000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c95070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c95070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44130/0 (socket says 172.21.15.175:44130) 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 -- 172.21.15.175:0/3999473332 learned_addr learned my addr 172.21.15.175:0/3999473332 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:22.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853f400640 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 -- 172.21.15.175:0/3999473332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 msgr2=0x555773c908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:22.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 -- 172.21.15.175:0/3999473332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 msgr2=0x555773c903a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:22.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c903a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 -- 172.21.15.175:0/3999473332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555773c91130 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f853ea00640 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c95070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f8530003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73899ca5d0f1f8be server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.025+0000 7f852cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f852cc00640 1 -- 172.21.15.175:0/3999473332 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530035030 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f852cc00640 1 -- 172.21.15.175:0/3999473332 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f85300037f0 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8530041070 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f852cc00640 1 -- 172.21.15.175:0/3999473332 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8530067050 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f854847d740 10 client.25489 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555773d073e0 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.026+0000 7f852cc00640 1 -- 172.21.15.175:0/3999473332 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8530041070 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f852cc00640 1 -- 172.21.15.175:0/3999473332 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8530004c20 con 0x555773c81980 2024-07-29T00:37:22.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f854847d740 10 client.25489 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f854847d740 10 client.25489 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f854847d740 20 client.25489 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f854847d740 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773d079e0 0x555773d09dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f854847d740 4 client.25489 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555773bef530 con 0x555773d079e0 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.027+0000 7f8536000640 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773d079e0 0x555773d09dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f8536000640 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773d079e0 0x555773d09dd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d28abf5bdc0acd0c server_cookie=9c1d072efb138c43 in_seq=0 out_seq=0 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f852cc00640 10 client.25489 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f852cc00640 1 -- 172.21.15.175:0/3999473332 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555773bef530 con 0x555773d079e0 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f852cc00640 10 client.25489 handle_command_reply: tid=0 2024-07-29T00:37:22.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 client.25489 shutdown 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 msgr2=0x555773c95070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c95070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f8530003fa0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 shutdown_connections 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555773d079e0 0x555773d09dd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555773c8c720 0x555773c908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555773c82ef0 0x555773c903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 --2- 172.21.15.175:0/3999473332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555773c81980 0x555773c95070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 >> 172.21.15.175:0/3999473332 conn(0x555773c4a950 msgr2=0x555773c8af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:22.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f852cc00640 0 client.25489 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:37:22.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.028+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 shutdown_connections 2024-07-29T00:37:22.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.029+0000 7f854847d740 1 -- 172.21.15.175:0/3999473332 wait complete. 2024-07-29T00:37:22.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.029+0000 7f854847d740 20 client.25489 trim_cache size 0 max 16384 2024-07-29T00:37:22.032 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.029+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f85240a2e10 msgr2=0x7f85240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.029+0000 7f8545a00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f85240a2e10 0x7f85240a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8538021000 tx=0x7f853801f000 comp rx=0 tx=0).stop 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.029+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 msgr2=0x7f854019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.029+0000 7f8545a00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 0x7f854019a890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f852800d950 tx=0x7f852800de10 comp rx=0 tx=0).stop 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 shutdown_connections 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f85240a2e10 0x7f85240a52d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f854010e170 0x7f854019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8540100f70 0x7f854019a890 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 --2- 172.21.15.175:0/3083958153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85401005c0 0x7f854019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 >> 172.21.15.175:0/3083958153 conn(0x7f85400fc490 msgr2=0x7f85400fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:22.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 shutdown_connections 2024-07-29T00:37:22.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:22.030+0000 7f8545a00640 1 -- 172.21.15.175:0/3083958153 wait complete. 2024-07-29T00:37:22.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:37:23.387 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:23.806 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:23 smithi175 ceph-mon[32138]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:23.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:23 smithi204 ceph-mon[37368]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:23 smithi181 ceph-mon[36706]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 -- 172.21.15.175:0/1345475416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc107740 msgr2=0x7f02fc107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 --2- 172.21.15.175:0/1345475416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc107740 0x7f02fc107b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009930 tx=0x7f02ec02a440 comp rx=0 tx=0).stop 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 -- 172.21.15.175:0/1345475416 shutdown_connections 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 --2- 172.21.15.175:0/1345475416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1005d0 0x7f02fc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 --2- 172.21.15.175:0/1345475416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 --2- 172.21.15.175:0/1345475416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc107740 0x7f02fc107b20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 -- 172.21.15.175:0/1345475416 >> 172.21.15.175:0/1345475416 conn(0x7f02fc076d60 msgr2=0x7f02fc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:24.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 -- 172.21.15.175:0/1345475416 shutdown_connections 2024-07-29T00:37:24.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.066+0000 7f0301000640 1 -- 172.21.15.175:0/1345475416 wait complete. 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.067+0000 7f0301000640 1 Processor -- start 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 -- start start 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc1005d0 0x7f02fc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc107740 0x7f02fc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc111b90 con 0x7f02fc1005d0 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc111a10 con 0x7f02fc0ffc30 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.068+0000 7f0301000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc111d10 con 0x7f02fc107740 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57594/0 (socket says 172.21.15.175:57594) 2024-07-29T00:37:24.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 -- 172.21.15.175:0/4129480298 learned_addr learned my addr 172.21.15.175:0/4129480298 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:24.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fbe00640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc107740 0x7f02fc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02f3e00640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc1005d0 0x7f02fc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 -- 172.21.15.175:0/4129480298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc107740 msgr2=0x7f02fc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc107740 0x7f02fc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 -- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc1005d0 msgr2=0x7f02fc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc1005d0 0x7f02fc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 -- 172.21.15.175:0/4129480298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc19f1d0 con 0x7f02fc0ffc30 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.069+0000 7f02fb400640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc19a220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02ec02a960 tx=0x7f02ec003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.070+0000 7f02f8c00640 1 -- 172.21.15.175:0/4129480298 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec03f070 con 0x7f02fc0ffc30 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.070+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc19f460 con 0x7f02fc0ffc30 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.070+0000 7f02f8c00640 1 -- 172.21.15.175:0/4129480298 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ec031c90 con 0x7f02fc0ffc30 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.070+0000 7f02f8c00640 1 -- 172.21.15.175:0/4129480298 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec035030 con 0x7f02fc0ffc30 2024-07-29T00:37:24.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.070+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a6da0 con 0x7f02fc0ffc30 2024-07-29T00:37:24.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.073+0000 7f02f8c00640 1 -- 172.21.15.175:0/4129480298 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f02ec042070 con 0x7f02fc0ffc30 2024-07-29T00:37:24.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.073+0000 7f030523f740 1 Processor -- start 2024-07-29T00:37:24.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 -- start start 2024-07-29T00:37:24.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984917e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984925510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e984875ab0 con 0x55e9849198f0 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e98480da80 con 0x55e984918380 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f030523f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e984871a70 con 0x55e984923120 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f02fbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984917e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f02f3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984925510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f02fbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984917e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41400/0 (socket says 172.21.15.175:41400) 2024-07-29T00:37:24.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f02fbe00640 1 -- 172.21.15.175:0/1137206209 learned_addr learned my addr 172.21.15.175:0/1137206209 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.074+0000 7f02fb400640 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f3e00640 1 -- 172.21.15.175:0/1137206209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 msgr2=0x55e9849187c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f3e00640 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849187c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f3e00640 1 -- 172.21.15.175:0/1137206209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 msgr2=0x55e984917e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f3e00640 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984917e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f3e00640 1 -- 172.21.15.175:0/1137206209 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9845e73c0 con 0x55e984923120 2024-07-29T00:37:24.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02fbe00640 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984917e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:24.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f8c00640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02e00a2e10 0x7f02e00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f8c00640 1 -- 172.21.15.175:0/4129480298 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f02ec0efe30 con 0x7f02fc0ffc30 2024-07-29T00:37:24.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02f3e00640 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984925510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02e4005fd0 tx=0x7f02e400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b85e2237775220 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:24.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.075+0000 7f02fbe00640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02e00a2e10 0x7f02e00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.076+0000 7f02f0c00640 1 -- 172.21.15.175:0/1137206209 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400dea0 con 0x55e984923120 2024-07-29T00:37:24.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.076+0000 7f02f0c00640 1 -- 172.21.15.175:0/1137206209 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4005500 con 0x55e984923120 2024-07-29T00:37:24.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.076+0000 7f02f0c00640 1 -- 172.21.15.175:0/1137206209 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4011680 con 0x55e984923120 2024-07-29T00:37:24.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 -- 172.21.15.175:0/1137206209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 msgr2=0x55e984925510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984925510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02e4005fd0 tx=0x7f02e400f730 comp rx=0 tx=0).stop 2024-07-29T00:37:24.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f02fbe00640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02e00a2e10 0x7f02e00a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005f40 tx=0x7f02f4009600 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:24.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 -- 172.21.15.175:0/1137206209 shutdown_connections 2024-07-29T00:37:24.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984925510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 --2- 172.21.15.175:0/1137206209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984917e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 -- 172.21.15.175:0/1137206209 >> 172.21.15.175:0/1137206209 conn(0x55e9848d8950 msgr2=0x55e984914a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:24.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 -- 172.21.15.175:0/1137206209 shutdown_connections 2024-07-29T00:37:24.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.077+0000 7f030523f740 1 -- 172.21.15.175:0/1137206209 wait complete. 2024-07-29T00:37:24.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.078+0000 7f030523f740 1 -- start start 2024-07-29T00:37:24.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.078+0000 7f030523f740 1 Processor -- start 2024-07-29T00:37:24.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e9849205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984923120 0x55e984920b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e984925ec0 con 0x55e984923120 2024-07-29T00:37:24.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e984830c10 con 0x55e984918380 2024-07-29T00:37:24.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e984926040 con 0x55e9849198f0 2024-07-29T00:37:24.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f030523f740 10 client.0 _ll_register_callbacks cb 0x55e9847f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:24.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f02fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f02fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57618/0 (socket says 172.21.15.175:57618) 2024-07-29T00:37:24.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f02fb400640 1 -- 172.21.15.175:0/882447945 learned_addr learned my addr 172.21.15.175:0/882447945 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:24.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f02f3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e9849205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.079+0000 7f02fbe00640 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984923120 0x55e984920b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02fb400640 1 -- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 msgr2=0x55e9849205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02fb400640 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e9849205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02fb400640 1 -- 172.21.15.175:0/882447945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984923120 msgr2=0x55e984920b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02fb400640 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984923120 0x55e984920b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02fb400640 1 -- 172.21.15.175:0/882447945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9848341e0 con 0x55e984918380 2024-07-29T00:37:24.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02fb400640 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849200a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f02ec000c00 tx=0x7f02ec03b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6947cdc1ad0d32ab server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:24.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02e9600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:24.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02e9600640 1 -- 172.21.15.175:0/882447945 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec049070 con 0x55e984918380 2024-07-29T00:37:24.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e4005500 con 0x55e984918380 2024-07-29T00:37:24.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02e9600640 1 -- 172.21.15.175:0/882447945 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f02ec0c01e0 con 0x55e984918380 2024-07-29T00:37:24.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.080+0000 7f02e9600640 1 -- 172.21.15.175:0/882447945 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02ec0b8450 con 0x55e984918380 2024-07-29T00:37:24.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 10 client.25502 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:24.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e984931c30 con 0x55e984918380 2024-07-29T00:37:24.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f02e9600640 1 -- 172.21.15.175:0/882447945 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4005500 con 0x55e984918380 2024-07-29T00:37:24.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f02e9600640 1 -- 172.21.15.175:0/882447945 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02ec0dc0e0 con 0x55e984918380 2024-07-29T00:37:24.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 10 client.25502 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:24.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 10 client.25502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:37:24.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 20 client.25502 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:24.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e9849330c0 0x55e984935510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 4 client.25502 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:24.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.081+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e984883020 con 0x55e9849330c0 2024-07-29T00:37:24.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.082+0000 7f02f3e00640 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e9849330c0 0x55e984935510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.082+0000 7f02f3e00640 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e9849330c0 0x55e984935510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a94c78ae91dee8ca server_cookie=249eab9dc7afdce3 in_seq=0 out_seq=0 2024-07-29T00:37:24.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.082+0000 7f02e9600640 10 client.25502 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:24.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f02e9600640 1 -- 172.21.15.175:0/882447945 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e984883020 con 0x55e9849330c0 2024-07-29T00:37:24.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f02e9600640 10 client.25502 handle_command_reply: tid=0 2024-07-29T00:37:24.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 client.25502 shutdown 2024-07-29T00:37:24.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 msgr2=0x55e9849200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849200a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f02ec000c00 tx=0x7f02ec03b990 comp rx=0 tx=0).stop 2024-07-29T00:37:24.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 shutdown_connections 2024-07-29T00:37:24.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e9849330c0 0x55e984935510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984923120 0x55e984920b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e9849205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 --2- 172.21.15.175:0/882447945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849200a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 >> 172.21.15.175:0/882447945 conn(0x55e9848d8950 msgr2=0x55e98491a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.083+0000 7f02e9600640 0 client.25502 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.084+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 shutdown_connections 2024-07-29T00:37:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.084+0000 7f030523f740 1 -- 172.21.15.175:0/882447945 wait complete. 2024-07-29T00:37:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.084+0000 7f030523f740 20 client.25502 trim_cache size 0 max 16384 2024-07-29T00:37:24.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 Processor -- start 2024-07-29T00:37:24.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 -- start start 2024-07-29T00:37:24.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984924510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984924a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1ab2e0 con 0x55e9849198f0 2024-07-29T00:37:24.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1ace20 con 0x55e984918380 2024-07-29T00:37:24.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f030523f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1ab9b0 con 0x55e984923120 2024-07-29T00:37:24.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.098+0000 7f02f3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984924510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984924510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41412/0 (socket says 172.21.15.175:41412) 2024-07-29T00:37:24.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 -- 172.21.15.175:0/2723949931 learned_addr learned my addr 172.21.15.175:0/2723949931 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:24.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02fb400640 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02fbe00640 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984924a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 -- 172.21.15.175:0/2723949931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 msgr2=0x55e984924a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984924a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 -- 172.21.15.175:0/2723949931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 msgr2=0x55e9849350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849350d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 -- 172.21.15.175:0/2723949931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98491e5f0 con 0x55e9849198f0 2024-07-29T00:37:24.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02fb400640 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:24.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.099+0000 7f02f3e00640 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984924510 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02e4005fd0 tx=0x7f02e400de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c717cea4bc3323 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:24.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.100+0000 7f02ebe00640 1 -- 172.21.15.175:0/2723949931 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4004880 con 0x55e9849198f0 2024-07-29T00:37:24.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.100+0000 7f02ebe00640 1 -- 172.21.15.175:0/2723949931 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4005290 con 0x55e9849198f0 2024-07-29T00:37:24.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.100+0000 7f02ebe00640 1 -- 172.21.15.175:0/2723949931 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e401b6a0 con 0x55e9849198f0 2024-07-29T00:37:24.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 -- 172.21.15.175:0/2723949931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 msgr2=0x55e984924510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984924510 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02e4005fd0 tx=0x7f02e400de80 comp rx=0 tx=0).stop 2024-07-29T00:37:24.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 -- 172.21.15.175:0/2723949931 shutdown_connections 2024-07-29T00:37:24.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e984923120 0x55e984924a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9849198f0 0x55e984924510 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 --2- 172.21.15.175:0/2723949931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984918380 0x55e9849350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 -- 172.21.15.175:0/2723949931 >> 172.21.15.175:0/2723949931 conn(0x55e9848d8950 msgr2=0x55e984907070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:24.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 -- 172.21.15.175:0/2723949931 shutdown_connections 2024-07-29T00:37:24.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 -- 172.21.15.175:0/2723949931 wait complete. 2024-07-29T00:37:24.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.101+0000 7f030523f740 1 -- start start 2024-07-29T00:37:24.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 Processor -- start 2024-07-29T00:37:24.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 0x55e98493a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e98493a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984923120 0x55e98492a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1ac750 con 0x55e984918380 2024-07-29T00:37:24.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1ac080 con 0x55e984923120 2024-07-29T00:37:24.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1a9300 con 0x55e9849198f0 2024-07-29T00:37:24.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f030523f740 10 client.0 _ll_register_callbacks cb 0x7f02ec009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:24.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f02fb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 0x55e98493a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f02fb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 0x55e98493a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41416/0 (socket says 172.21.15.175:41416) 2024-07-29T00:37:24.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.102+0000 7f02fb400640 1 -- 172.21.15.175:0/3651453005 learned_addr learned my addr 172.21.15.175:0/3651453005 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:24.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02f3e00640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e98493a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fbe00640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984923120 0x55e98492a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fb400640 1 -- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 msgr2=0x55e98493a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fb400640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e98493a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fb400640 1 -- 172.21.15.175:0/3651453005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984923120 msgr2=0x55e98492a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:24.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fb400640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984923120 0x55e98492a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fb400640 1 -- 172.21.15.175:0/3651453005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98492a810 con 0x55e984918380 2024-07-29T00:37:24.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02fb400640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 0x55e98493a370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f02ec02aa00 tx=0x7f02ec0b8a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80069efae54672c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:24.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02e9600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.103+0000 7f02e9600640 1 -- 172.21.15.175:0/3651453005 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec049040 con 0x55e984918380 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f02e9600640 1 -- 172.21.15.175:0/3651453005 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f02ec0bc410 con 0x55e984918380 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e4005290 con 0x55e984918380 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f02e9600640 1 -- 172.21.15.175:0/3651453005 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02ec0bbb10 con 0x55e984918380 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 10 client.15408 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e98492a810 con 0x55e984918380 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f02e9600640 1 -- 172.21.15.175:0/3651453005 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4005290 con 0x55e984918380 2024-07-29T00:37:24.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f02e9600640 1 -- 172.21.15.175:0/3651453005 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02ec0c00c0 con 0x55e984918380 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 10 client.15408 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 10 client.15408 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 20 client.15408 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e98499ddb0 0x55e9849a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 4 client.15408 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.104+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9849358a0 con 0x55e98499ddb0 2024-07-29T00:37:24.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.105+0000 7f02f3e00640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e98499ddb0 0x55e9849a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.105+0000 7f02f3e00640 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e98499ddb0 0x55e9849a01a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1beba033251707ae server_cookie=5fc7045b06c37dcb in_seq=0 out_seq=0 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.105+0000 7f02e9600640 10 client.15408 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f02e9600640 1 -- 172.21.15.175:0/3651453005 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9849358a0 con 0x55e98499ddb0 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f02e9600640 10 client.15408 handle_command_reply: tid=0 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 client.15408 shutdown 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 msgr2=0x55e98493a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 0x55e98493a370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f02ec02aa00 tx=0x7f02ec0b8a50 comp rx=0 tx=0).stop 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 shutdown_connections 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55e98499ddb0 0x55e9849a01a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e984923120 0x55e98492a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9849198f0 0x55e98493a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 --2- 172.21.15.175:0/3651453005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e984918380 0x55e98493a370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 >> 172.21.15.175:0/3651453005 conn(0x55e9848d8950 msgr2=0x55e98491e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f02e9600640 0 client.15408 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 shutdown_connections 2024-07-29T00:37:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.106+0000 7f030523f740 1 -- 172.21.15.175:0/3651453005 wait complete. 2024-07-29T00:37:24.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f030523f740 20 client.15408 trim_cache size 0 max 16384 2024-07-29T00:37:24.121 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:37:24.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02e00a2e10 msgr2=0x7f02e00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02e00a2e10 0x7f02e00a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005f40 tx=0x7f02f4009600 comp rx=0 tx=0).stop 2024-07-29T00:37:24.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 msgr2=0x7f02fc19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc19a220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02ec02a960 tx=0x7f02ec003b90 comp rx=0 tx=0).stop 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 shutdown_connections 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc107740 0x7f02fc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02e00a2e10 0x7f02e00a52d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02fc1005d0 0x7f02fc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 --2- 172.21.15.175:0/4129480298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02fc0ffc30 0x7f02fc19a220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.107+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 >> 172.21.15.175:0/4129480298 conn(0x7f02fc076d60 msgr2=0x7f02fc0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.108+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 shutdown_connections 2024-07-29T00:37:24.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:24.108+0000 7f0301000640 1 -- 172.21.15.175:0/4129480298 wait complete. 2024-07-29T00:37:25.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:37:25.506 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:25.890 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:25 smithi175 ceph-mon[32138]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:25 smithi181 ceph-mon[36706]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:25 smithi204 ceph-mon[37368]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:26.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2045336572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074200 msgr2=0x7f4bcc0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2045336572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074200 0x7f4bcc0745e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009900 tx=0x7f4bc002a480 comp rx=0 tx=0).stop 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2045336572 shutdown_connections 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2045336572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc10df80 0x7f4bcc110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2045336572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc074b20 0x7f4bcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2045336572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074200 0x7f4bcc0745e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2045336572 >> 172.21.15.175:0/2045336572 conn(0x7f4bcc0fd400 msgr2=0x7f4bcc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2045336572 shutdown_connections 2024-07-29T00:37:26.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.172+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2045336572 wait complete. 2024-07-29T00:37:26.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 Processor -- start 2024-07-29T00:37:26.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 -- start start 2024-07-29T00:37:26.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc074200 0x7f4bcc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 0x7f4bcc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc10df80 0x7f4bcc19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113d00 con 0x7f4bcc074b20 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113b80 con 0x7f4bcc10df80 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.173+0000 7f4bd2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113e80 con 0x7f4bcc074200 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 0x7f4bcc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc074200 0x7f4bcc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 0x7f4bcc19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41428/0 (socket says 172.21.15.175:41428) 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd0e00640 1 -- 172.21.15.175:0/2468027674 learned_addr learned my addr 172.21.15.175:0/2468027674 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:26.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd2200640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc10df80 0x7f4bcc19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd1800640 1 -- 172.21.15.175:0/2468027674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc10df80 msgr2=0x7f4bcc19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd1800640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc10df80 0x7f4bcc19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd1800640 1 -- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 msgr2=0x7f4bcc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd1800640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 0x7f4bcc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd1800640 1 -- 172.21.15.175:0/2468027674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc19f1f0 con 0x7f4bcc074200 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd2200640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc10df80 0x7f4bcc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:26.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.174+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 0x7f4bcc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.175+0000 7f4bd1800640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc074200 0x7f4bcc19a240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bc002a9a0 tx=0x7f4bc0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.175+0000 7f4bca000640 1 -- 172.21.15.175:0/2468027674 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc003f070 con 0x7f4bcc074200 2024-07-29T00:37:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.175+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc19f480 con 0x7f4bcc074200 2024-07-29T00:37:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.175+0000 7f4bca000640 1 -- 172.21.15.175:0/2468027674 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0031c90 con 0x7f4bcc074200 2024-07-29T00:37:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.175+0000 7f4bca000640 1 -- 172.21.15.175:0/2468027674 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0035030 con 0x7f4bcc074200 2024-07-29T00:37:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.175+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a6dc0 con 0x7f4bcc074200 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.178+0000 7f4bd6e1e740 1 Processor -- start 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bca000640 1 -- 172.21.15.175:0/2468027674 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f4bc0042070 con 0x7f4bcc074200 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 -- start start 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdcf9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdcf9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5556cdc5fab0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5556cdbf7a80 con 0x5556cdd0d0f0 2024-07-29T00:37:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.179+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5556cdc5ba70 con 0x5556cdcf9b40 2024-07-29T00:37:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdcf9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57658/0 (socket says 172.21.15.175:57658) 2024-07-29T00:37:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3006030500 learned_addr learned my addr 172.21.15.175:0/3006030500 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd1800640 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdcf9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3006030500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 msgr2=0x5556cdcf9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdcf9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3006030500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 msgr2=0x5556cdcf9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdcf9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3006030500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556cd9d13c0 con 0x5556cdd0d0f0 2024-07-29T00:37:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.180+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0f4e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010580 tx=0x7f4bb8010a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6125ff22068bd1b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:26.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.181+0000 7f4bbea00640 1 -- 172.21.15.175:0/3006030500 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800e7d0 con 0x5556cdd0d0f0 2024-07-29T00:37:26.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.181+0000 7f4bca000640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f4bb40a2e90 0x7f4bb40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.181+0000 7f4bbea00640 1 -- 172.21.15.175:0/3006030500 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb800ee50 con 0x5556cdd0d0f0 2024-07-29T00:37:26.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.181+0000 7f4bbea00640 1 -- 172.21.15.175:0/3006030500 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80193d0 con 0x5556cdd0d0f0 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.181+0000 7f4bca000640 1 -- 172.21.15.175:0/2468027674 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f4bc00efc70 con 0x7f4bcc074200 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.181+0000 7f4bd2200640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f4bb40a2e90 0x7f4bb40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3006030500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 msgr2=0x5556cdd0f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0f4e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010580 tx=0x7f4bb8010a40 comp rx=0 tx=0).stop 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3006030500 shutdown_connections 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0f4e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdcf9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3006030500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdcf9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3006030500 >> 172.21.15.175:0/3006030500 conn(0x5556cdcc2950 msgr2=0x5556cdcf61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3006030500 shutdown_connections 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd2200640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f4bb40a2e90 0x7f4bb40a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0751e0 tx=0x7f4bc4009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:26.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.182+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3006030500 wait complete. 2024-07-29T00:37:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 -- start start 2024-07-29T00:37:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 Processor -- start 2024-07-29T00:37:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 0x5556cdd0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd0a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5556cdd0fe90 con 0x5556cdcf9b40 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.183+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5556cdc1ac10 con 0x5556cdcfb0b0 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5556cdd10010 con 0x5556cdd0d0f0 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd6e1e740 10 client.0 _ll_register_callbacks cb 0x5556cdbe3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 0x5556cdd0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 0x5556cdd0a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41440/0 (socket says 172.21.15.175:41440) 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd1800640 1 -- 172.21.15.175:0/3723842681 learned_addr learned my addr 172.21.15.175:0/3723842681 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd0a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd2200640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3723842681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 msgr2=0x5556cdd0aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3723842681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 msgr2=0x5556cdd0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 0x5556cdd0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3723842681 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556cdc1e1e0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.184+0000 7f4bd2200640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bd1800640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 0x5556cdd0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd0a590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010580 tx=0x7f4bb800be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa454bac4dc4f837 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bb3400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bb3400640 1 -- 172.21.15.175:0/3723842681 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8029060 con 0x5556cdcfb0b0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb800ee50 con 0x5556cdcfb0b0 2024-07-29T00:37:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bb3400640 1 -- 172.21.15.175:0/3723842681 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f4bb8011440 con 0x5556cdcfb0b0 2024-07-29T00:37:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bb3400640 1 -- 172.21.15.175:0/3723842681 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bb8011640 con 0x5556cdcfb0b0 2024-07-29T00:37:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bd6e1e740 10 client.25520 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.185+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556cdd1bc00 con 0x5556cdcfb0b0 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bb3400640 1 -- 172.21.15.175:0/3723842681 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb800ee50 con 0x5556cdcfb0b0 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bb3400640 1 -- 172.21.15.175:0/3723842681 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4bb80054d0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd6e1e740 10 client.25520 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd6e1e740 10 client.25520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd6e1e740 20 client.25520 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd1cfb0 0x5556cdd1f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd6e1e740 4 client.25520 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556cdc6d020 con 0x5556cdd1cfb0 2024-07-29T00:37:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.186+0000 7f4bd1800640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd1cfb0 0x5556cdd1f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.187+0000 7f4bd1800640 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd1cfb0 0x5556cdd1f3a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c606057dd2902a63 server_cookie=8013c9aa25ec980f in_seq=0 out_seq=0 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.187+0000 7f4bb3400640 10 client.25520 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bb3400640 1 -- 172.21.15.175:0/3723842681 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556cdc6d020 con 0x5556cdd1cfb0 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bb3400640 10 client.25520 handle_command_reply: tid=0 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 client.25520 shutdown 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 msgr2=0x5556cdd0a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd0a590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010580 tx=0x7f4bb800be90 comp rx=0 tx=0).stop 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 shutdown_connections 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd1cfb0 0x5556cdd1f3a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd0a590 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3723842681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcf9b40 0x5556cdd0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 >> 172.21.15.175:0/3723842681 conn(0x5556cdcc2950 msgr2=0x5556cdcfbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bb3400640 0 client.25520 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 shutdown_connections 2024-07-29T00:37:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.188+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3723842681 wait complete. 2024-07-29T00:37:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.189+0000 7f4bd6e1e740 20 client.25520 trim_cache size 0 max 16384 2024-07-29T00:37:26.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.204+0000 7f4bd6e1e740 1 Processor -- start 2024-07-29T00:37:26.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.204+0000 7f4bd6e1e740 1 -- start start 2024-07-29T00:37:26.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdd072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdd0d0f0 0x5556cdcf49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1ab390 con 0x5556cdd0d0f0 2024-07-29T00:37:26.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1aced0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1aba60 con 0x5556cdcf9b40 2024-07-29T00:37:26.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdd0d0f0 0x5556cdcf49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdd0d0f0 0x5556cdcf49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41446/0 (socket says 172.21.15.175:41446) 2024-07-29T00:37:26.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd2200640 1 -- 172.21.15.175:0/3202722932 learned_addr learned my addr 172.21.15.175:0/3202722932 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.205+0000 7f4bd1800640 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdd072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3202722932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 msgr2=0x5556cdd072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdd072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3202722932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdd0d0f0 msgr2=0x5556cdcf49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdd0d0f0 0x5556cdcf49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bd0e00640 1 -- 172.21.15.175:0/3202722932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556cdd085a0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd07810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800e6d0 tx=0x7f4bb800b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd91469c232454f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bbe000640 1 -- 172.21.15.175:0/3202722932 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8029070 con 0x5556cdcfb0b0 2024-07-29T00:37:26.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bbe000640 1 -- 172.21.15.175:0/3202722932 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb8011440 con 0x5556cdcfb0b0 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.206+0000 7f4bbe000640 1 -- 172.21.15.175:0/3202722932 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800eb20 con 0x5556cdcfb0b0 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3202722932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 msgr2=0x5556cdd07810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd07810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800e6d0 tx=0x7f4bb800b4a0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3202722932 shutdown_connections 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdd0d0f0 0x5556cdcf49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcfb0b0 0x5556cdd07810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/3202722932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdcf9b40 0x5556cdd072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3202722932 >> 172.21.15.175:0/3202722932 conn(0x5556cdcc2950 msgr2=0x5556cdd053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3202722932 shutdown_connections 2024-07-29T00:37:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.207+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/3202722932 wait complete. 2024-07-29T00:37:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 -- start start 2024-07-29T00:37:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 Processor -- start 2024-07-29T00:37:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcf9b40 0x5556cdd1f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdd1fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1ac800 con 0x5556cdcfb0b0 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1ac130 con 0x5556cdcf9b40 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.208+0000 7f4bd6e1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1a9370 con 0x5556cdd0d0f0 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd6e1e740 10 client.0 _ll_register_callbacks cb 0x7f4bc00f35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdd1fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdd1fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41462/0 (socket says 172.21.15.175:41462) 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 -- 172.21.15.175:0/4107836023 learned_addr learned my addr 172.21.15.175:0/4107836023 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd1800640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcf9b40 0x5556cdd1f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd2200640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 -- 172.21.15.175:0/4107836023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 msgr2=0x5556cdd858e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd858e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 -- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcf9b40 msgr2=0x5556cdd1f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcf9b40 0x5556cdd1f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd0e00640 1 -- 172.21.15.175:0/4107836023 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556cdd85ed0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.209+0000 7f4bd1800640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcf9b40 0x5556cdd1f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bd0e00640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdd1fa80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8011610 tx=0x7f4bb8012da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f67e3365d8946f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bb3400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bb3400640 1 -- 172.21.15.175:0/4107836023 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8029070 con 0x5556cdcfb0b0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb8011440 con 0x5556cdcfb0b0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bb3400640 1 -- 172.21.15.175:0/4107836023 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f4bb80117e0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bb3400640 1 -- 172.21.15.175:0/4107836023 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bb8044050 con 0x5556cdcfb0b0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.210+0000 7f4bd6e1e740 10 client.15426 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556cdd871b0 con 0x5556cdcfb0b0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bb3400640 1 -- 172.21.15.175:0/4107836023 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb8011440 con 0x5556cdcfb0b0 2024-07-29T00:37:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bb3400640 1 -- 172.21.15.175:0/4107836023 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4bb8003680 con 0x5556cdcfb0b0 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 10 client.15426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 20 client.15426 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd87880 0x5556cdd89ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 4 client.15426 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.211+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556cdc67530 con 0x5556cdd87880 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.212+0000 7f4bd1800640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd87880 0x5556cdd89ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.212+0000 7f4bd1800640 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd87880 0x5556cdd89ca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=668104b0a6b35de8 server_cookie=7813a68ec3eb15c0 in_seq=0 out_seq=0 2024-07-29T00:37:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.212+0000 7f4bb3400640 10 client.15426 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.212+0000 7f4bb3400640 1 -- 172.21.15.175:0/4107836023 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556cdc67530 con 0x5556cdd87880 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.212+0000 7f4bb3400640 10 client.15426 handle_command_reply: tid=0 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.212+0000 7f4bd6e1e740 1 client.15426 shutdown 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 msgr2=0x5556cdd1fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdd1fa80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8011610 tx=0x7f4bb8012da0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 shutdown_connections 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556cdd0d0f0 0x5556cdd858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556cdcfb0b0 0x5556cdd1fa80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5556cdd87880 0x5556cdd89ca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 --2- 172.21.15.175:0/4107836023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556cdcf9b40 0x5556cdd1f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 >> 172.21.15.175:0/4107836023 conn(0x5556cdcc2950 msgr2=0x5556cdcf49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bb3400640 0 client.15426 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:37:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 shutdown_connections 2024-07-29T00:37:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 1 -- 172.21.15.175:0/4107836023 wait complete. 2024-07-29T00:37:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.213+0000 7f4bd6e1e740 20 client.15426 trim_cache size 0 max 16384 2024-07-29T00:37:26.215 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f4bb40a2e90 msgr2=0x7f4bb40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f4bb40a2e90 0x7f4bb40a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0751e0 tx=0x7f4bc4009500 comp rx=0 tx=0).stop 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc074200 msgr2=0x7f4bcc19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc074200 0x7f4bcc19a240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bc002a9a0 tx=0x7f4bc0003b90 comp rx=0 tx=0).stop 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 shutdown_connections 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bcc10df80 0x7f4bcc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f4bb40a2e90 0x7f4bb40a5350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bcc074b20 0x7f4bcc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 --2- 172.21.15.175:0/2468027674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc074200 0x7f4bcc19a240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 >> 172.21.15.175:0/2468027674 conn(0x7f4bcc0fd400 msgr2=0x7f4bcc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 shutdown_connections 2024-07-29T00:37:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:26.214+0000 7f4bd2c00640 1 -- 172.21.15.175:0/2468027674 wait complete. 2024-07-29T00:37:27.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:37:27.652 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:27 smithi181 ceph-mon[36706]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:28.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:27 smithi175 ceph-mon[32138]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:28.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:27 smithi204 ceph-mon[37368]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:28.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 -- 172.21.15.175:0/1077824038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100e40 msgr2=0x7f65cc10db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 --2- 172.21.15.175:0/1077824038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100e40 0x7f65cc10db00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009900 tx=0x7f65b802a410 comp rx=0 tx=0).stop 2024-07-29T00:37:28.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 -- 172.21.15.175:0/1077824038 shutdown_connections 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 --2- 172.21.15.175:0/1077824038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65cc10e040 0x7f65cc110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 --2- 172.21.15.175:0/1077824038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100e40 0x7f65cc10db00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 --2- 172.21.15.175:0/1077824038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100490 0x7f65cc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 -- 172.21.15.175:0/1077824038 >> 172.21.15.175:0/1077824038 conn(0x7f65cc0fc300 msgr2=0x7f65cc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 -- 172.21.15.175:0/1077824038 shutdown_connections 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.344+0000 7f65d0a00640 1 -- 172.21.15.175:0/1077824038 wait complete. 2024-07-29T00:37:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 Processor -- start 2024-07-29T00:37:28.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 -- start start 2024-07-29T00:37:28.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100490 0x7f65cc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 0x7f65cc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65cc10e040 0x7f65cc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc113d30 con 0x7f65cc100490 2024-07-29T00:37:28.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc113bb0 con 0x7f65cc100e40 2024-07-29T00:37:28.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.346+0000 7f65d0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc113eb0 con 0x7f65cc10e040 2024-07-29T00:37:28.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65ca800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100490 0x7f65cc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 0x7f65cc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65ca800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100490 0x7f65cc19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41486/0 (socket says 172.21.15.175:41486) 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65ca800640 1 -- 172.21.15.175:0/4131902914 learned_addr learned my addr 172.21.15.175:0/4131902914 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 0x7f65cc19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57680/0 (socket says 172.21.15.175:57680) 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65cb200640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65cc10e040 0x7f65cc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 -- 172.21.15.175:0/4131902914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65cc10e040 msgr2=0x7f65cc19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65cc10e040 0x7f65cc19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 -- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100490 msgr2=0x7f65cc19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100490 0x7f65cc19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 -- 172.21.15.175:0/4131902914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc19f180 con 0x7f65cc100e40 2024-07-29T00:37:28.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.347+0000 7f65c9e00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 0x7f65cc19a710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65b802a930 tx=0x7f65b8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:28.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.348+0000 7f65c3400640 1 -- 172.21.15.175:0/4131902914 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b803f070 con 0x7f65cc100e40 2024-07-29T00:37:28.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.348+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc19f410 con 0x7f65cc100e40 2024-07-29T00:37:28.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.348+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1a6d50 con 0x7f65cc100e40 2024-07-29T00:37:28.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.348+0000 7f65c3400640 1 -- 172.21.15.175:0/4131902914 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b803b940 con 0x7f65cc100e40 2024-07-29T00:37:28.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.348+0000 7f65c3400640 1 -- 172.21.15.175:0/4131902914 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8049060 con 0x7f65cc100e40 2024-07-29T00:37:28.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.352+0000 7f65d4039740 1 Processor -- start 2024-07-29T00:37:28.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 -- start start 2024-07-29T00:37:28.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fbbe439ab0 con 0x55fbbe4de910 2024-07-29T00:37:28.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fbbe3d1a80 con 0x55fbbe4d50e0 2024-07-29T00:37:28.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65d4039740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fbbe435a70 con 0x55fbbe4d3b70 2024-07-29T00:37:28.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65cb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65c3400640 1 -- 172.21.15.175:0/4131902914 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f65b8042070 con 0x7f65cc100e40 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65ca800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65cb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57684/0 (socket says 172.21.15.175:57684) 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.353+0000 7f65cb200640 1 -- 172.21.15.175:0/664868414 learned_addr learned my addr 172.21.15.175:0/664868414 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.354+0000 7f65cb200640 1 -- 172.21.15.175:0/664868414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 msgr2=0x55fbbe4d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.354+0000 7f65cb200640 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.354+0000 7f65cb200640 1 -- 172.21.15.175:0/664868414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 msgr2=0x55fbbe4e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.354+0000 7f65cb200640 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.354+0000 7f65cb200640 1 -- 172.21.15.175:0/664868414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbe1ab3c0 con 0x55fbbe4d50e0 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.354+0000 7f65cb200640 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4d3630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009f30 tx=0x7f65c40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56895fef4ff9ffb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.355+0000 7f65b7400640 1 -- 172.21.15.175:0/664868414 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4004d20 con 0x55fbbe4d50e0 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.355+0000 7f65b7400640 1 -- 172.21.15.175:0/664868414 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c4004050 con 0x55fbbe4d50e0 2024-07-29T00:37:28.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.355+0000 7f65b7400640 1 -- 172.21.15.175:0/664868414 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4005260 con 0x55fbbe4d50e0 2024-07-29T00:37:28.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.355+0000 7f65c3400640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f65b00a2e90 0x7f65b00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.355+0000 7f65d4039740 1 -- 172.21.15.175:0/664868414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 msgr2=0x55fbbe4d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.355+0000 7f65d4039740 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4d3630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009f30 tx=0x7f65c40118d0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65c3400640 1 -- 172.21.15.175:0/4131902914 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f65b80ef6f0 con 0x7f65cc100e40 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65ca800640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f65b00a2e90 0x7f65b00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 -- 172.21.15.175:0/664868414 shutdown_connections 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 --2- 172.21.15.175:0/664868414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4d3630 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 -- 172.21.15.175:0/664868414 >> 172.21.15.175:0/664868414 conn(0x55fbbe49c950 msgr2=0x55fbbe4d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 -- 172.21.15.175:0/664868414 shutdown_connections 2024-07-29T00:37:28.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.356+0000 7f65d4039740 1 -- 172.21.15.175:0/664868414 wait complete. 2024-07-29T00:37:28.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.357+0000 7f65ca800640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f65b00a2e90 0x7f65b00a5350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55fbbe4d1060 tx=0x7f65bc005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:28.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.357+0000 7f65d4039740 1 -- start start 2024-07-29T00:37:28.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.357+0000 7f65d4039740 1 Processor -- start 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.357+0000 7f65d4039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65d4039740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65d4039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65d4039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fbbe4e16b0 con 0x55fbbe4de910 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65d4039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fbbe3f4c10 con 0x55fbbe4d3b70 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65d4039740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fbbe4e1830 con 0x55fbbe4d50e0 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65d4039740 10 client.0 _ll_register_callbacks cb 0x55fbbe3bda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65c9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4db880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57686/0 (socket says 172.21.15.175:57686) 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4dc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41504/0 (socket says 172.21.15.175:41504) 2024-07-29T00:37:28.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 -- 172.21.15.175:0/1335128214 learned_addr learned my addr 172.21.15.175:0/1335128214 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:28.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 -- 172.21.15.175:0/1335128214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 msgr2=0x55fbbe4dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 -- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 msgr2=0x55fbbe4dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.358+0000 7f65ca800640 1 -- 172.21.15.175:0/1335128214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbe3f81e0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.359+0000 7f65ca800640 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4db880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65bc004620 tx=0x7f65bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5b7f2338825bf7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:28.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.359+0000 7f65abe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:28.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.359+0000 7f65abe00640 1 -- 172.21.15.175:0/1335128214 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc005020 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.359+0000 7f65abe00640 1 -- 172.21.15.175:0/1335128214 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f65bc0051c0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.359+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c4004050 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65abe00640 1 -- 172.21.15.175:0/1335128214 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65bc0053c0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 10 client.25538 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fbbe4f5c90 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65abe00640 1 -- 172.21.15.175:0/1335128214 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c4004050 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65abe00640 1 -- 172.21.15.175:0/1335128214 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65bc027030 con 0x55fbbe4d3b70 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 10 client.25538 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 10 client.25538 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 20 client.25538 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe4f7150 0x55fbbe4f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 4 client.25538 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:28.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.360+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbbe447020 con 0x55fbbe4f7150 2024-07-29T00:37:28.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.361+0000 7f65c9e00640 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe4f7150 0x55fbbe4f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.361+0000 7f65c9e00640 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe4f7150 0x55fbbe4f9580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42c89ee0774c197f server_cookie=ca87a6337545c59f in_seq=0 out_seq=0 2024-07-29T00:37:28.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.361+0000 7f65abe00640 10 client.25538 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:28.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65abe00640 1 -- 172.21.15.175:0/1335128214 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fbbe447020 con 0x55fbbe4f7150 2024-07-29T00:37:28.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65abe00640 10 client.25538 handle_command_reply: tid=0 2024-07-29T00:37:28.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 client.25538 shutdown 2024-07-29T00:37:28.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 msgr2=0x55fbbe4db880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4db880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65bc004620 tx=0x7f65bc004730 comp rx=0 tx=0).stop 2024-07-29T00:37:28.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 shutdown_connections 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe4f7150 0x55fbbe4f9580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 --2- 172.21.15.175:0/1335128214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4db880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 >> 172.21.15.175:0/1335128214 conn(0x55fbbe49c950 msgr2=0x55fbbe4d5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.362+0000 7f65abe00640 0 client.25538 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.363+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 shutdown_connections 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.363+0000 7f65d4039740 1 -- 172.21.15.175:0/1335128214 wait complete. 2024-07-29T00:37:28.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.363+0000 7f65d4039740 20 client.25538 trim_cache size 0 max 16384 2024-07-29T00:37:28.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 Processor -- start 2024-07-29T00:37:28.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 -- start start 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4fee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac00e850 con 0x55fbbe4de910 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac010390 con 0x55fbbe4d3b70 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.377+0000 7f65d4039740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac00ef20 con 0x55fbbe4d50e0 2024-07-29T00:37:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57698/0 (socket says 172.21.15.175:57698) 2024-07-29T00:37:28.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 -- 172.21.15.175:0/1621556431 learned_addr learned my addr 172.21.15.175:0/1621556431 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:28.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65cb200640 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65c9e00640 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 -- 172.21.15.175:0/1621556431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 msgr2=0x55fbbe4fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 -- 172.21.15.175:0/1621556431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 msgr2=0x55fbbe4ff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4ff3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.378+0000 7f65ca800640 1 -- 172.21.15.175:0/1621556431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbe4d9dd0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.379+0000 7f65ca800640 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d03b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65bc01fab0 tx=0x7f65bc012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6be7b6f8c3d297e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.379+0000 7f65b6a00640 1 -- 172.21.15.175:0/1621556431 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc035030 con 0x55fbbe4d3b70 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.379+0000 7f65b6a00640 1 -- 172.21.15.175:0/1621556431 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc005560 con 0x55fbbe4d3b70 2024-07-29T00:37:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.379+0000 7f65b6a00640 1 -- 172.21.15.175:0/1621556431 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc031450 con 0x55fbbe4d3b70 2024-07-29T00:37:28.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 -- 172.21.15.175:0/1621556431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 msgr2=0x55fbbe4d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d03b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65bc01fab0 tx=0x7f65bc012d00 comp rx=0 tx=0).stop 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 -- 172.21.15.175:0/1621556431 shutdown_connections 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe4ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 --2- 172.21.15.175:0/1621556431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4d03b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 -- 172.21.15.175:0/1621556431 >> 172.21.15.175:0/1621556431 conn(0x55fbbe49c950 msgr2=0x55fbbe4cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 -- 172.21.15.175:0/1621556431 shutdown_connections 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.380+0000 7f65d4039740 1 -- 172.21.15.175:0/1621556431 wait complete. 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 -- start start 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 Processor -- start 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe55fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac00fcc0 con 0x55fbbe4de910 2024-07-29T00:37:28.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac00f5f0 con 0x55fbbe4d50e0 2024-07-29T00:37:28.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac00c830 con 0x55fbbe4d3b70 2024-07-29T00:37:28.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.381+0000 7f65d4039740 10 client.0 _ll_register_callbacks cb 0x7f65bc009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:28.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe55fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4ea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44292/0 (socket says 172.21.15.175:44292) 2024-07-29T00:37:28.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe55fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41518/0 (socket says 172.21.15.175:41518) 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65cb200640 1 -- 172.21.15.175:0/2558779644 learned_addr learned my addr 172.21.15.175:0/2558779644 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4eaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57700/0 (socket says 172.21.15.175:57700) 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 -- 172.21.15.175:0/2558779644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 msgr2=0x55fbbe4eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 -- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 msgr2=0x55fbbe55fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe55fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65ca800640 1 -- 172.21.15.175:0/2558779644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbe5603c0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65cb200640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe55fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.382+0000 7f65c9e00640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65ca800640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4ea960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65bc019040 tx=0x7f65bc01f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2754529584a30b5b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65abe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:28.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65abe00640 1 -- 172.21.15.175:0/2558779644 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc035070 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65bc005560 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65abe00640 1 -- 172.21.15.175:0/2558779644 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f65bc01d8a0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65abe00640 1 -- 172.21.15.175:0/2558779644 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65bc047050 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65d4039740 10 client.25543 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.383+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fbbe5616a0 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65abe00640 1 -- 172.21.15.175:0/2558779644 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc005560 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65abe00640 1 -- 172.21.15.175:0/2558779644 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65bc027070 con 0x55fbbe4d3b70 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65d4039740 10 client.25543 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65d4039740 10 client.25543 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65d4039740 20 client.25543 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65d4039740 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe561d00 0x55fbbe5640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65d4039740 4 client.25543 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:37:28.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.384+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbbe441530 con 0x55fbbe561d00 2024-07-29T00:37:28.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.385+0000 7f65c9e00640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe561d00 0x55fbbe5640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:28.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.385+0000 7f65c9e00640 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe561d00 0x55fbbe5640f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11f7784254131ece server_cookie=f6c06c8bcb2e0ed in_seq=0 out_seq=0 2024-07-29T00:37:28.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.385+0000 7f65abe00640 10 client.25543 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:28.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.385+0000 7f65abe00640 1 -- 172.21.15.175:0/2558779644 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbbe441530 con 0x55fbbe561d00 2024-07-29T00:37:28.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.385+0000 7f65abe00640 10 client.25543 handle_command_reply: tid=0 2024-07-29T00:37:28.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 client.25543 shutdown 2024-07-29T00:37:28.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 msgr2=0x55fbbe4ea960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4ea960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65bc019040 tx=0x7f65bc01f8f0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 shutdown_connections 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55fbbe561d00 0x55fbbe5640f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbbe4de910 0x55fbbe55fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fbbe4d50e0 0x55fbbe4eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 --2- 172.21.15.175:0/2558779644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbbe4d3b70 0x55fbbe4ea960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 >> 172.21.15.175:0/2558779644 conn(0x55fbbe49c950 msgr2=0x55fbbe4dd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65abe00640 0 client.25543 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 shutdown_connections 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 1 -- 172.21.15.175:0/2558779644 wait complete. 2024-07-29T00:37:28.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.386+0000 7f65d4039740 20 client.25543 trim_cache size 0 max 16384 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f65b00a2e90 msgr2=0x7f65b00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f65b00a2e90 0x7f65b00a5350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55fbbe4d1060 tx=0x7f65bc005f90 comp rx=0 tx=0).stop 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 msgr2=0x7f65cc19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 0x7f65cc19a710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65b802a930 tx=0x7f65b8003b20 comp rx=0 tx=0).stop 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 shutdown_connections 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65cc10e040 0x7f65cc19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc100e40 0x7f65cc19a710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f65b00a2e90 0x7f65b00a5350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 --2- 172.21.15.175:0/4131902914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65cc100490 0x7f65cc19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 >> 172.21.15.175:0/4131902914 conn(0x7f65cc0fc300 msgr2=0x7f65cc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:28.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 shutdown_connections 2024-07-29T00:37:28.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:28.387+0000 7f65d0a00640 1 -- 172.21.15.175:0/4131902914 wait complete. 2024-07-29T00:37:29.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:37:29.739 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:29.889 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:29 smithi175 ceph-mon[32138]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:29 smithi181 ceph-mon[36706]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:29 smithi204 ceph-mon[37368]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:30.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 -- 172.21.15.175:0/3269390149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa36410df00 msgr2=0x7fa364110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 --2- 172.21.15.175:0/3269390149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa36410df00 0x7fa364110350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009ca0 tx=0x7fa35c02a950 comp rx=0 tx=0).stop 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 -- 172.21.15.175:0/3269390149 shutdown_connections 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 --2- 172.21.15.175:0/3269390149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa36410df00 0x7fa364110350 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 --2- 172.21.15.175:0/3269390149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 0x7fa36410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 --2- 172.21.15.175:0/3269390149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa364074310 0x7fa3640746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 -- 172.21.15.175:0/3269390149 >> 172.21.15.175:0/3269390149 conn(0x7fa3640fd400 msgr2=0x7fa3640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 -- 172.21.15.175:0/3269390149 shutdown_connections 2024-07-29T00:37:30.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.394+0000 7fa363e00640 1 -- 172.21.15.175:0/3269390149 wait complete. 2024-07-29T00:37:30.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.395+0000 7fa363e00640 1 Processor -- start 2024-07-29T00:37:30.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.395+0000 7fa363e00640 1 -- start start 2024-07-29T00:37:30.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 0x7fa36419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 0x7fa36419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa36410df00 0x7fa36419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa364113d00 con 0x7fa364074310 2024-07-29T00:37:30.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa364113b80 con 0x7fa364074c30 2024-07-29T00:37:30.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa364113e80 con 0x7fa36410df00 2024-07-29T00:37:30.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa362a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 0x7fa36419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa362000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 0x7fa36419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa363400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa36410df00 0x7fa36419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa362a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 0x7fa36419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54440/0 (socket says 172.21.15.175:54440) 2024-07-29T00:37:30.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.396+0000 7fa362a00640 1 -- 172.21.15.175:0/3446183295 learned_addr learned my addr 172.21.15.175:0/3446183295 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:30.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362000640 1 -- 172.21.15.175:0/3446183295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa36410df00 msgr2=0x7fa36419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362000640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa36410df00 0x7fa36419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362000640 1 -- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 msgr2=0x7fa36419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362000640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 0x7fa36419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362000640 1 -- 172.21.15.175:0/3446183295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419f110 con 0x7fa364074c30 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa363400640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa36410df00 0x7fa36419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362a00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 0x7fa36419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.397+0000 7fa362000640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 0x7fa36419a6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa35000e9f0 tx=0x7fa35000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.398+0000 7fa368a00640 1 -- 172.21.15.175:0/3446183295 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35000cd80 con 0x7fa364074c30 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.398+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419f400 con 0x7fa364074c30 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.398+0000 7fa368a00640 1 -- 172.21.15.175:0/3446183295 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3500035f0 con 0x7fa364074c30 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.398+0000 7fa368a00640 1 -- 172.21.15.175:0/3446183295 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350010670 con 0x7fa364074c30 2024-07-29T00:37:30.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.398+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a6ce0 con 0x7fa364074c30 2024-07-29T00:37:30.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 Processor -- start 2024-07-29T00:37:30.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 -- start start 2024-07-29T00:37:30.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d24a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d24afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d257d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d1b0ab0 con 0x55fc1d24c0e0 2024-07-29T00:37:30.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d148a80 con 0x55fc1d255910 2024-07-29T00:37:30.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.401+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d1aca70 con 0x55fc1d24ab70 2024-07-29T00:37:30.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa368a00640 1 -- 172.21.15.175:0/3446183295 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7fa3500037a0 con 0x7fa364074c30 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa363400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d24a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa362000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d257d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa362a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d24afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa363400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d24a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54450/0 (socket says 172.21.15.175:54450) 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa363400640 1 -- 172.21.15.175:0/3671497986 learned_addr learned my addr 172.21.15.175:0/3671497986 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa363400640 1 -- 172.21.15.175:0/3671497986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 msgr2=0x55fc1d24afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.402+0000 7fa363400640 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d24afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.403+0000 7fa363400640 1 -- 172.21.15.175:0/3671497986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 msgr2=0x55fc1d257d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.403+0000 7fa363400640 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d257d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.403+0000 7fa363400640 1 -- 172.21.15.175:0/3671497986 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1cf223c0 con 0x55fc1d24c0e0 2024-07-29T00:37:30.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.403+0000 7fa362000640 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d257d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:30.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.403+0000 7fa363400640 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d24a630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009ca0 tx=0x7fa35c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbb86688e000949 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:30.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.403+0000 7fa34fe00640 1 -- 172.21.15.175:0/3671497986 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c048070 con 0x55fc1d24c0e0 2024-07-29T00:37:30.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa34fe00640 1 -- 172.21.15.175:0/3671497986 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35c003e50 con 0x55fc1d24c0e0 2024-07-29T00:37:30.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa34fe00640 1 -- 172.21.15.175:0/3671497986 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c034040 con 0x55fc1d24c0e0 2024-07-29T00:37:30.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa368a00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa3480a2f10 0x7fa3480a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa368a00640 1 -- 172.21.15.175:0/3446183295 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa3500cf990 con 0x7fa364074c30 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3671497986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 msgr2=0x55fc1d24a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d24a630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009ca0 tx=0x7fa35c005f80 comp rx=0 tx=0).stop 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.404+0000 7fa362a00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa3480a2f10 0x7fa3480a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3671497986 shutdown_connections 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d257d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d24afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3671497986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d24a630 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3671497986 >> 172.21.15.175:0/3671497986 conn(0x55fc1d213950 msgr2=0x55fc1d247200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3671497986 shutdown_connections 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3671497986 wait complete. 2024-07-29T00:37:30.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa36c9bc740 1 -- start start 2024-07-29T00:37:30.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.405+0000 7fa362a00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa3480a2f10 0x7fa3480a53d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55fc1d248060 tx=0x7fa354005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:30.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 Processor -- start 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d252850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d252d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d2532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d2586a0 con 0x55fc1d255910 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d16bc10 con 0x55fc1d24c0e0 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d258820 con 0x55fc1d24ab70 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.406+0000 7fa36c9bc740 10 client.0 _ll_register_callbacks cb 0x55fc1d134a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d252d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d252d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42842/0 (socket says 172.21.15.175:42842) 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 -- 172.21.15.175:0/3350633787 learned_addr learned my addr 172.21.15.175:0/3350633787 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:30.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa363400640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d2532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 -- 172.21.15.175:0/3350633787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 msgr2=0x55fc1d252850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362a00640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d252850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d252850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 -- 172.21.15.175:0/3350633787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 msgr2=0x55fc1d2532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d2532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.407+0000 7fa362000640 1 -- 172.21.15.175:0/3350633787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1d16f1e0 con 0x55fc1d24c0e0 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.408+0000 7fa363400640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d2532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.408+0000 7fa362000640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d252d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c6a0 tx=0x7fa35009b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdeff373b0f38b8d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.408+0000 7fa34cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.408+0000 7fa34cc00640 1 -- 172.21.15.175:0/3350633787 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35009ff90 con 0x55fc1d24c0e0 2024-07-29T00:37:30.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.408+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35c003e50 con 0x55fc1d24c0e0 2024-07-29T00:37:30.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.408+0000 7fa34cc00640 1 -- 172.21.15.175:0/3350633787 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa350098250 con 0x55fc1d24c0e0 2024-07-29T00:37:30.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa34cc00640 1 -- 172.21.15.175:0/3350633787 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3500a4850 con 0x55fc1d24c0e0 2024-07-29T00:37:30.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 10 client.25556 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:30.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc1d264430 con 0x55fc1d24c0e0 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa34cc00640 1 -- 172.21.15.175:0/3350633787 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35c003e50 con 0x55fc1d24c0e0 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa34cc00640 1 -- 172.21.15.175:0/3350633787 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa35009aaa0 con 0x55fc1d24c0e0 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 10 client.25556 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 20 client.25556 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d265840 0x55fc1d267c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 4 client.25556 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.409+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1d1be020 con 0x55fc1d265840 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.410+0000 7fa362a00640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d265840 0x55fc1d267c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.410+0000 7fa362a00640 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d265840 0x55fc1d267c70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c75cb4f29167c295 server_cookie=b8b4ee4afa400d55 in_seq=0 out_seq=0 2024-07-29T00:37:30.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.410+0000 7fa34cc00640 10 client.25556 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa34cc00640 1 -- 172.21.15.175:0/3350633787 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc1d1be020 con 0x55fc1d265840 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa34cc00640 10 client.25556 handle_command_reply: tid=0 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 client.25556 shutdown 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 msgr2=0x55fc1d252d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d252d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c6a0 tx=0x7fa35009b540 comp rx=0 tx=0).stop 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 shutdown_connections 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d265840 0x55fc1d267c70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d2532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d252d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/3350633787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d252850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.411+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 >> 172.21.15.175:0/3350633787 conn(0x55fc1d213950 msgr2=0x55fc1d24cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:30.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.412+0000 7fa34cc00640 0 client.25556 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:30.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.412+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 shutdown_connections 2024-07-29T00:37:30.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.412+0000 7fa36c9bc740 1 -- 172.21.15.175:0/3350633787 wait complete. 2024-07-29T00:37:30.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.412+0000 7fa36c9bc740 20 client.25556 trim_cache size 0 max 16384 2024-07-29T00:37:30.421 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.7 2024-07-29T00:37:30.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:37:30.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 Processor -- start 2024-07-29T00:37:30.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 -- start start 2024-07-29T00:37:30.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d26d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d26da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d245b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641af620 con 0x55fc1d255910 2024-07-29T00:37:30.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641b1160 con 0x55fc1d24c0e0 2024-07-29T00:37:30.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.430+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641afcf0 con 0x55fc1d24ab70 2024-07-29T00:37:30.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d26d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d26d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35252/0 (socket says 172.21.15.175:35252) 2024-07-29T00:37:30.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 -- 172.21.15.175:0/250043269 learned_addr learned my addr 172.21.15.175:0/250043269 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:30.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362000640 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d26da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa363400640 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d245b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 -- 172.21.15.175:0/250043269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 msgr2=0x55fc1d26da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d26da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 -- 172.21.15.175:0/250043269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 msgr2=0x55fc1d245b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d245b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 -- 172.21.15.175:0/250043269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1d24d6c0 con 0x55fc1d24ab70 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.431+0000 7fa362a00640 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d26d540 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa35400f920 tx=0x7fa354017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dfc1908cbb090aa server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.432+0000 7fa34f400640 1 -- 172.21.15.175:0/250043269 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35402f070 con 0x55fc1d24ab70 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.432+0000 7fa34f400640 1 -- 172.21.15.175:0/250043269 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35401dd60 con 0x55fc1d24ab70 2024-07-29T00:37:30.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.432+0000 7fa34f400640 1 -- 172.21.15.175:0/250043269 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354029dc0 con 0x55fc1d24ab70 2024-07-29T00:37:30.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 -- 172.21.15.175:0/250043269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 msgr2=0x55fc1d26d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d26d540 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa35400f920 tx=0x7fa354017000 comp rx=0 tx=0).stop 2024-07-29T00:37:30.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 -- 172.21.15.175:0/250043269 shutdown_connections 2024-07-29T00:37:30.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d255910 0x55fc1d245b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d24c0e0 0x55fc1d26da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/250043269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d26d540 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 -- 172.21.15.175:0/250043269 >> 172.21.15.175:0/250043269 conn(0x55fc1d213950 msgr2=0x55fc1d251940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 -- 172.21.15.175:0/250043269 shutdown_connections 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 -- 172.21.15.175:0/250043269 wait complete. 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.433+0000 7fa36c9bc740 1 -- start start 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 Processor -- start 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d25dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d25e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d24f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641b0a90 con 0x55fc1d24c0e0 2024-07-29T00:37:30.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641b03c0 con 0x55fc1d255910 2024-07-29T00:37:30.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641ad600 con 0x55fc1d24ab70 2024-07-29T00:37:30.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa36c9bc740 10 client.0 _ll_register_callbacks cb 0x7fa3500ccf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:30.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa362000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d25e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa362000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d25e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54482/0 (socket says 172.21.15.175:54482) 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.434+0000 7fa362000640 1 -- 172.21.15.175:0/2270787334 learned_addr learned my addr 172.21.15.175:0/2270787334 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa362a00640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d25dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d24f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 -- 172.21.15.175:0/2270787334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 msgr2=0x55fc1d25dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d25dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 -- 172.21.15.175:0/2270787334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 msgr2=0x55fc1d25e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:30.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d25e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 -- 172.21.15.175:0/2270787334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1d24f770 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa362000640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d25e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.435+0000 7fa363400640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d24f060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005e30 tx=0x7fa35c02fd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9acba2f87344afa8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa34cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa34cc00640 1 -- 172.21.15.175:0/2270787334 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c048040 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa34cc00640 1 -- 172.21.15.175:0/2270787334 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa35c040040 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35401dd60 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa34cc00640 1 -- 172.21.15.175:0/2270787334 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa35c05b050 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa36c9bc740 10 client.25562 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.436+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc1d2d86c0 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa34cc00640 1 -- 172.21.15.175:0/2270787334 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35401dd60 con 0x55fc1d255910 2024-07-29T00:37:30.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa34cc00640 1 -- 172.21.15.175:0/2270787334 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa35c04b030 con 0x55fc1d255910 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa36c9bc740 10 client.25562 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa36c9bc740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa36c9bc740 20 client.25562 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d2d89f0 0x55fc1d2dade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa36c9bc740 4 client.25562 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc1d1b8530 con 0x55fc1d2d89f0 2024-07-29T00:37:30.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.437+0000 7fa362a00640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d2d89f0 0x55fc1d2dade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:30.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.438+0000 7fa34cc00640 10 client.25562 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.438+0000 7fa362a00640 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d2d89f0 0x55fc1d2dade0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa16540a9799fe3 server_cookie=a784d3c6e4c6c58c in_seq=0 out_seq=0 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.438+0000 7fa34cc00640 1 -- 172.21.15.175:0/2270787334 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc1d1b8530 con 0x55fc1d2d89f0 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.438+0000 7fa34cc00640 10 client.25562 handle_command_reply: tid=0 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.438+0000 7fa36c9bc740 1 client.25562 shutdown 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 msgr2=0x55fc1d24f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d24f060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005e30 tx=0x7fa35c02fd30 comp rx=0 tx=0).stop 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 shutdown_connections 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fc1d2d89f0 0x55fc1d2dade0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc1d255910 0x55fc1d24f060 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc1d24c0e0 0x55fc1d25e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 --2- 172.21.15.175:0/2270787334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1d24ab70 0x55fc1d25dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa34cc00640 0 client.25562 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:30.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 >> 172.21.15.175:0/2270787334 conn(0x55fc1d213950 msgr2=0x55fc1d245b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:30.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 shutdown_connections 2024-07-29T00:37:30.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 1 -- 172.21.15.175:0/2270787334 wait complete. 2024-07-29T00:37:30.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.439+0000 7fa36c9bc740 20 client.25562 trim_cache size 0 max 16384 2024-07-29T00:37:30.443 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:37:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa3480a2f10 msgr2=0x7fa3480a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa3480a2f10 0x7fa3480a53d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55fc1d248060 tx=0x7fa354005f90 comp rx=0 tx=0).stop 2024-07-29T00:37:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 msgr2=0x7fa36419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 0x7fa36419a6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa35000e9f0 tx=0x7fa35000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 shutdown_connections 2024-07-29T00:37:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa36410df00 0x7fa36419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa364074c30 0x7fa36419a6a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa3480a2f10 0x7fa3480a53d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 --2- 172.21.15.175:0/3446183295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364074310 0x7fa36419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 >> 172.21.15.175:0/3446183295 conn(0x7fa3640fd400 msgr2=0x7fa3640fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.440+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 shutdown_connections 2024-07-29T00:37:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:30.441+0000 7fa363e00640 1 -- 172.21.15.175:0/3446183295 wait complete. 2024-07-29T00:37:31.358 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:31.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:31 smithi175 ceph-mon[32138]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:31 smithi181 ceph-mon[36706]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:32.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 -- 172.21.15.175:0/3918427989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44741005c0 msgr2=0x7f44741009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:32.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 --2- 172.21.15.175:0/3918427989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44741005c0 0x7f44741009a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4464009900 tx=0x7f446402a410 comp rx=0 tx=0).stop 2024-07-29T00:37:32.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 -- 172.21.15.175:0/3918427989 shutdown_connections 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 --2- 172.21.15.175:0/3918427989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447410e170 0x7f4474110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 --2- 172.21.15.175:0/3918427989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4474100f70 0x7f447410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 --2- 172.21.15.175:0/3918427989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44741005c0 0x7f44741009a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 -- 172.21.15.175:0/3918427989 >> 172.21.15.175:0/3918427989 conn(0x7f44740fc490 msgr2=0x7f44740fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 -- 172.21.15.175:0/3918427989 shutdown_connections 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.057+0000 7f4478e00640 1 -- 172.21.15.175:0/3918427989 wait complete. 2024-07-29T00:37:32.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.058+0000 7f4478e00640 1 Processor -- start 2024-07-29T00:37:32.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 -- start start 2024-07-29T00:37:32.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44741005c0 0x7f447419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:32.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 0x7f447419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447410e170 0x7f447419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4474113ec0 con 0x7f4474100f70 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4474113d40 con 0x7f447410e170 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4478e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4474114040 con 0x7f44741005c0 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4471400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 0x7f447419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4471400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 0x7f447419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54504/0 (socket says 172.21.15.175:54504) 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4471400640 1 -- 172.21.15.175:0/3897087046 learned_addr learned my addr 172.21.15.175:0/3897087046 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4471e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44741005c0 0x7f447419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:32.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.059+0000 7f4472800640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447410e170 0x7f447419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471400640 1 -- 172.21.15.175:0/3897087046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44741005c0 msgr2=0x7f447419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471400640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44741005c0 0x7f447419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471400640 1 -- 172.21.15.175:0/3897087046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447410e170 msgr2=0x7f447419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471400640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447410e170 0x7f447419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471400640 1 -- 172.21.15.175:0/3897087046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447419f300 con 0x7f4474100f70 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44741005c0 0x7f447419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f4471400640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 0x7f447419a890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f446000e9f0 tx=0x7f446000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.060+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446000cd80 con 0x7f4474100f70 2024-07-29T00:37:32.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.061+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447419f5f0 con 0x7f4474100f70 2024-07-29T00:37:32.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.061+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4460010910 con 0x7f4474100f70 2024-07-29T00:37:32.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.061+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44600188f0 con 0x7f4474100f70 2024-07-29T00:37:32.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.061+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44741a6ed0 con 0x7f4474100f70 2024-07-29T00:37:32.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.062+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44741009a0 con 0x7f4474100f70 2024-07-29T00:37:32.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.065+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f4460010ac0 con 0x7f4474100f70 2024-07-29T00:37:32.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.067+0000 7f446aa00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f44580a2e10 0x7f44580a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:32.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.068+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f44600cfbd0 con 0x7f4474100f70 2024-07-29T00:37:32.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.068+0000 7f4471e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f44580a2e10 0x7f44580a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:32.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.069+0000 7f4471e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f44580a2e10 0x7f44580a52d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464005a40 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:32.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.072+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f446009c520 con 0x7f4474100f70 2024-07-29T00:37:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:31 smithi204 ceph-mon[37368]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:32.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:37:32.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:37:32.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.319+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f447402d5a0 con 0x7f4474100f70 2024-07-29T00:37:32.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.320+0000 7f446aa00640 1 -- 172.21.15.175:0/3897087046 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f446009be20 con 0x7f4474100f70 2024-07-29T00:37:32.321 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:37:32.321 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:37:32.321 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:37:32.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f44580a2e10 msgr2=0x7f44580a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:32.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f44580a2e10 0x7f44580a52d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464005a40 comp rx=0 tx=0).stop 2024-07-29T00:37:32.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 msgr2=0x7f447419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:32.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 0x7f447419a890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f446000e9f0 tx=0x7f446000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 shutdown_connections 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447410e170 0x7f447419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f44580a2e10 0x7f44580a52d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4474100f70 0x7f447419a890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 --2- 172.21.15.175:0/3897087046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44741005c0 0x7f447419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 >> 172.21.15.175:0/3897087046 conn(0x7f44740fc490 msgr2=0x7f44740fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 shutdown_connections 2024-07-29T00:37:32.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:32.323+0000 7f4478e00640 1 -- 172.21.15.175:0/3897087046 wait complete. 2024-07-29T00:37:32.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3897087046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:33.033 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3897087046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3897087046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:37:33.104 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 90 --expiration 149.7 --await-for 45.0' 2024-07-29T00:37:33.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db / --timeout 90 --expiration 149.7 --await-for 45.0 2024-07-29T00:37:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:33.639 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:33 smithi175 ceph-mon[32138]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.895+0000 7f04e1200640 1 -- 172.21.15.175:0/4242467992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1098c0 msgr2=0x7f04dc109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.895+0000 7f04e1200640 1 --2- 172.21.15.175:0/4242467992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1098c0 0x7f04dc109ca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009930 tx=0x7f04cc02a440 comp rx=0 tx=0).stop 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 -- 172.21.15.175:0/4242467992 shutdown_connections 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 --2- 172.21.15.175:0/4242467992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc104260 0x7f04dc110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 --2- 172.21.15.175:0/4242467992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04dc1038c0 0x7f04dc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 --2- 172.21.15.175:0/4242467992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1098c0 0x7f04dc109ca0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 -- 172.21.15.175:0/4242467992 >> 172.21.15.175:0/4242467992 conn(0x7f04dc0ff600 msgr2=0x7f04dc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:33.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 -- 172.21.15.175:0/4242467992 shutdown_connections 2024-07-29T00:37:33.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.896+0000 7f04e1200640 1 -- 172.21.15.175:0/4242467992 wait complete. 2024-07-29T00:37:33.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.897+0000 7f04e1200640 1 Processor -- start 2024-07-29T00:37:33.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 -- start start 2024-07-29T00:37:33.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 0x7f04dc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc104260 0x7f04dc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04dc1098c0 0x7f04dc19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc113d10 con 0x7f04dc1098c0 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc113b90 con 0x7f04dc1038c0 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.898+0000 7f04e1200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc113e90 con 0x7f04dc104260 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 0x7f04dc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 0x7f04dc19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42884/0 (socket says 172.21.15.175:42884) 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 -- 172.21.15.175:0/3095001595 learned_addr learned my addr 172.21.15.175:0/3095001595 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04dbe00640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04dc1098c0 0x7f04dc19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04daa00640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc104260 0x7f04dc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 -- 172.21.15.175:0/3095001595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc104260 msgr2=0x7f04dc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc104260 0x7f04dc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 -- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04dc1098c0 msgr2=0x7f04dc19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04dc1098c0 0x7f04dc19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04db400640 1 -- 172.21.15.175:0/3095001595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc19f160 con 0x7f04dc1038c0 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.899+0000 7f04daa00640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc104260 0x7f04dc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.900+0000 7f04db400640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 0x7f04dc19a1b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04cc02a960 tx=0x7f04cc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.900+0000 7f04d3e00640 1 -- 172.21.15.175:0/3095001595 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc03f070 con 0x7f04dc1038c0 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.900+0000 7f04d3e00640 1 -- 172.21.15.175:0/3095001595 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc031c90 con 0x7f04dc1038c0 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.900+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc19f3f0 con 0x7f04dc1038c0 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.900+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1a6d30 con 0x7f04dc1038c0 2024-07-29T00:37:33.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.901+0000 7f04d3e00640 1 -- 172.21.15.175:0/3095001595 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc035030 con 0x7f04dc1038c0 2024-07-29T00:37:33.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 Processor -- start 2024-07-29T00:37:33.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 -- start start 2024-07-29T00:37:33.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04d3e00640 1 -- 172.21.15.175:0/3095001595 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7f04cc042070 con 0x7f04dc1038c0 2024-07-29T00:37:33.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 0x560b834b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b6640 0x560b834c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834c0f00 0x560b834c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b8341b8a0 con 0x560b834b6640 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b833b3710 con 0x560b834b5d20 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.905+0000 7f04e5252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560b83417860 con 0x560b834c0f00 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 0x560b834b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 0x560b834b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42900/0 (socket says 172.21.15.175:42900) 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 -- 172.21.15.175:0/845189457 learned_addr learned my addr 172.21.15.175:0/845189457 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 -- 172.21.15.175:0/845189457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834c0f00 msgr2=0x560b834c32f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834c0f00 0x560b834c32f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 -- 172.21.15.175:0/845189457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b6640 msgr2=0x560b834c09c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b6640 0x560b834c09c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.906+0000 7f04dbe00640 1 -- 172.21.15.175:0/845189457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8318d6a0 con 0x560b834b5d20 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.907+0000 7f04dbe00640 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 0x560b834b6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009e60 tx=0x7f04d4010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7aa1258fe90987e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.907+0000 7f04d0c00640 1 -- 172.21.15.175:0/845189457 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4016070 con 0x560b834b5d20 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.907+0000 7f04d0c00640 1 -- 172.21.15.175:0/845189457 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4004d20 con 0x560b834b5d20 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.907+0000 7f04d0c00640 1 -- 172.21.15.175:0/845189457 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4012870 con 0x560b834b5d20 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.907+0000 7f04d3e00640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04c00a2f10 0x7f04c00a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 -- 172.21.15.175:0/845189457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 msgr2=0x560b834b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 0x560b834b6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009e60 tx=0x7f04d4010740 comp rx=0 tx=0).stop 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04d3e00640 1 -- 172.21.15.175:0/3095001595 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f04cc0efd30 con 0x7f04dc1038c0 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 -- 172.21.15.175:0/845189457 shutdown_connections 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834c0f00 0x560b834c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b6640 0x560b834c09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 --2- 172.21.15.175:0/845189457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b5d20 0x560b834b6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 -- 172.21.15.175:0/845189457 >> 172.21.15.175:0/845189457 conn(0x560b834b22b0 msgr2=0x560b834b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04e5252740 1 -- 172.21.15.175:0/845189457 shutdown_connections 2024-07-29T00:37:33.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.908+0000 7f04db400640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04c00a2f10 0x7f04c00a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.909+0000 7f04db400640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04c00a2f10 0x7f04c00a53d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0ed2e0 tx=0x7f04cc02a440 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:33.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.909+0000 7f04e5252740 1 -- 172.21.15.175:0/845189457 wait complete. 2024-07-29T00:37:33.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.910+0000 7f04e5252740 1 -- start start 2024-07-29T00:37:33.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.910+0000 7f04e5252740 1 Processor -- start 2024-07-29T00:37:33.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 0x560b834be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b834c3ad0 con 0x560b834b5d20 2024-07-29T00:37:33.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b833d68c0 con 0x560b834c0f00 2024-07-29T00:37:33.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560b834c3c50 con 0x560b834be840 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04e5252740 10 client.0 _ll_register_callbacks cb 0x560b833c2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04daa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 0x560b834be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.911+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 0x560b834be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42906/0 (socket says 172.21.15.175:42906) 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 -- 172.21.15.175:0/990246670 learned_addr learned my addr 172.21.15.175:0/990246670 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04dbe00640 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 -- 172.21.15.175:0/990246670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 msgr2=0x560b834ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 -- 172.21.15.175:0/990246670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 msgr2=0x560b834bddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 -- 172.21.15.175:0/990246670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b833b9bc0 con 0x560b834c0f00 2024-07-29T00:37:33.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04daa00640 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 0x560b834be300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009c80 tx=0x7f04c800e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f143c2e16a53834c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:33.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.912+0000 7f04c5600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:33.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.913+0000 7f04c5600640 1 -- 172.21.15.175:0/990246670 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8025060 con 0x560b834c0f00 2024-07-29T00:37:33.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.913+0000 7f04c5600640 1 -- 172.21.15.175:0/990246670 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f04c8004030 con 0x560b834c0f00 2024-07-29T00:37:33.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.913+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d4004d20 con 0x560b834c0f00 2024-07-29T00:37:33.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.913+0000 7f04c5600640 1 -- 172.21.15.175:0/990246670 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04c8004230 con 0x560b834c0f00 2024-07-29T00:37:33.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.913+0000 7f04e5252740 10 client.25580 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:33.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.913+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b834cffe0 con 0x560b834c0f00 2024-07-29T00:37:33.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.914+0000 7f04c5600640 1 -- 172.21.15.175:0/990246670 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4004d20 con 0x560b834c0f00 2024-07-29T00:37:33.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.914+0000 7f04c5600640 1 -- 172.21.15.175:0/990246670 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f04c800f040 con 0x560b834c0f00 2024-07-29T00:37:33.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.915+0000 7f04e5252740 10 client.25580 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:33.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.915+0000 7f04e5252740 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:33.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.915+0000 7f04e5252740 20 client.25580 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:33.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.915+0000 7f04e5252740 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834d13f0 0x560b834d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.915+0000 7f04e5252740 4 client.25580 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:33.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.915+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b833dbc90 con 0x560b834d13f0 2024-07-29T00:37:33.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.916+0000 7f04dbe00640 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834d13f0 0x560b834d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.916+0000 7f04dbe00640 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834d13f0 0x560b834d37e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f40ef7a637b27c server_cookie=8294ec0573d519ee in_seq=0 out_seq=0 2024-07-29T00:37:33.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.916+0000 7f04c5600640 10 client.25580 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:33.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.917+0000 7f04c5600640 1 -- 172.21.15.175:0/990246670 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b833dbc90 con 0x560b834d13f0 2024-07-29T00:37:33.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.917+0000 7f04c5600640 10 client.25580 handle_command_reply: tid=0 2024-07-29T00:37:33.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.917+0000 7f04e5252740 1 client.25580 shutdown 2024-07-29T00:37:33.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 msgr2=0x560b834be300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 0x560b834be300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009c80 tx=0x7f04c800e6c0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 shutdown_connections 2024-07-29T00:37:33.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834d13f0 0x560b834d37e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834c0f00 0x560b834be300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 --2- 172.21.15.175:0/990246670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 >> 172.21.15.175:0/990246670 conn(0x560b834b22b0 msgr2=0x560b834b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:33.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04c5600640 0 client.25580 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:33.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 shutdown_connections 2024-07-29T00:37:33.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 1 -- 172.21.15.175:0/990246670 wait complete. 2024-07-29T00:37:33.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.918+0000 7f04e5252740 20 client.25580 trim_cache size 0 max 16384 2024-07-29T00:37:33.946 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:33.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 Processor -- start 2024-07-29T00:37:33.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 -- start start 2024-07-29T00:37:33.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc00e850 con 0x560b834b5d20 2024-07-29T00:37:33.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc010390 con 0x560b834b22b0 2024-07-29T00:37:33.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.954+0000 7f04e5252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc00ef20 con 0x560b834be840 2024-07-29T00:37:33.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.955+0000 7f04daa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.955+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.955+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834bdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42914/0 (socket says 172.21.15.175:42914) 2024-07-29T00:37:33.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.955+0000 7f04dbe00640 1 -- 172.21.15.175:0/2294938043 learned_addr learned my addr 172.21.15.175:0/2294938043 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:33.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.955+0000 7f04daa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54536/0 (socket says 172.21.15.175:54536) 2024-07-29T00:37:33.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.955+0000 7f04dbe00640 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04dbe00640 1 -- 172.21.15.175:0/2294938043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 msgr2=0x560b834bcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04dbe00640 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04dbe00640 1 -- 172.21.15.175:0/2294938043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 msgr2=0x560b834ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04dbe00640 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04dbe00640 1 -- 172.21.15.175:0/2294938043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b834b6470 con 0x560b834b22b0 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04dbe00640 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834bdd80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005a90 tx=0x7f04d401c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c3c35aab8f7a57 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.956+0000 7f04c7e00640 1 -- 172.21.15.175:0/2294938043 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4023070 con 0x560b834b22b0 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.957+0000 7f04c7e00640 1 -- 172.21.15.175:0/2294938043 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4024040 con 0x560b834b22b0 2024-07-29T00:37:33.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.957+0000 7f04c7e00640 1 -- 172.21.15.175:0/2294938043 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4016040 con 0x560b834b22b0 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.957+0000 7f04e5252740 1 -- 172.21.15.175:0/2294938043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 msgr2=0x560b834bdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.957+0000 7f04e5252740 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834bdd80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005a90 tx=0x7f04d401c040 comp rx=0 tx=0).stop 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 -- 172.21.15.175:0/2294938043 shutdown_connections 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 --2- 172.21.15.175:0/2294938043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834bdd80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 -- 172.21.15.175:0/2294938043 >> 172.21.15.175:0/2294938043 conn(0x560b834be460 msgr2=0x560b834bc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:33.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 -- 172.21.15.175:0/2294938043 shutdown_connections 2024-07-29T00:37:33.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 -- 172.21.15.175:0/2294938043 wait complete. 2024-07-29T00:37:33.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.958+0000 7f04e5252740 1 -- start start 2024-07-29T00:37:33.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 Processor -- start 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc00fcc0 con 0x560b834b5d20 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc00f5f0 con 0x560b834b22b0 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc00c830 con 0x560b834be840 2024-07-29T00:37:33.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.959+0000 7f04e5252740 10 client.0 _ll_register_callbacks cb 0x7f04c80098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.960+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.960+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42918/0 (socket says 172.21.15.175:42918) 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.960+0000 7f04dbe00640 1 -- 172.21.15.175:0/4005104434 learned_addr learned my addr 172.21.15.175:0/4005104434 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.960+0000 7f04daa00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.960+0000 7f04dbe00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04dbe00640 1 -- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 msgr2=0x560b834c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04dbe00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04dbe00640 1 -- 172.21.15.175:0/4005104434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 msgr2=0x560b834bac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:33.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04dbe00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04dbe00640 1 -- 172.21.15.175:0/4005104434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b834bba60 con 0x560b834be840 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04dbe00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bb230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f04d4004e70 tx=0x7f04d4005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1704795a7164bcec server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.961+0000 7f04c5600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.962+0000 7f04c5600640 1 -- 172.21.15.175:0/4005104434 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4023030 con 0x560b834be840 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.962+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d4024040 con 0x560b834be840 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.962+0000 7f04c5600640 1 -- 172.21.15.175:0/4005104434 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f04d4005580 con 0x560b834be840 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.962+0000 7f04c5600640 1 -- 172.21.15.175:0/4005104434 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04d4016040 con 0x560b834be840 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.962+0000 7f04e5252740 10 client.25576 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.962+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b8353c230 con 0x560b834be840 2024-07-29T00:37:33.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.963+0000 7f04c5600640 1 -- 172.21.15.175:0/4005104434 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d402c8d0 con 0x560b834be840 2024-07-29T00:37:33.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.963+0000 7f04c5600640 1 -- 172.21.15.175:0/4005104434 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f04d400f040 con 0x560b834be840 2024-07-29T00:37:33.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.964+0000 7f04e5252740 10 client.25576 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:33.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.964+0000 7f04e5252740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:33.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.964+0000 7f04e5252740 20 client.25576 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:33.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.964+0000 7f04e5252740 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834c0f00 0x560b8353e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:33.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.964+0000 7f04e5252740 4 client.25576 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:37:33.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.964+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f04d4024040 con 0x560b834c0f00 2024-07-29T00:37:33.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.965+0000 7f04daa00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834c0f00 0x560b8353e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:33.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.965+0000 7f04daa00640 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834c0f00 0x560b8353e700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d981bd2fc353af79 server_cookie=4b905b78368a2ccb in_seq=0 out_seq=0 2024-07-29T00:37:33.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.965+0000 7f04c5600640 10 client.25576 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:33.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.966+0000 7f04c5600640 1 -- 172.21.15.175:0/4005104434 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f04d4024040 con 0x560b834c0f00 2024-07-29T00:37:33.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.966+0000 7f04c5600640 10 client.25576 handle_command_reply: tid=0 2024-07-29T00:37:33.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.966+0000 7f04e5252740 1 client.25576 shutdown 2024-07-29T00:37:33.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 msgr2=0x560b834bb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bb230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f04d4004e70 tx=0x7f04d4005b00 comp rx=0 tx=0).stop 2024-07-29T00:37:33.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 shutdown_connections 2024-07-29T00:37:33.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560b834c0f00 0x560b8353e700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b834be840 0x560b834bb230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b834b5d20 0x560b834bac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 --2- 172.21.15.175:0/4005104434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b834b22b0 0x560b834c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 >> 172.21.15.175:0/4005104434 conn(0x560b834be460 msgr2=0x560b834bceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:33.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.967+0000 7f04c5600640 0 client.25576 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.968+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 shutdown_connections 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.968+0000 7f04e5252740 1 -- 172.21.15.175:0/4005104434 wait complete. 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.968+0000 7f04e5252740 20 client.25576 trim_cache size 0 max 16384 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "a77ed333-b732-4634-acc2-3b8441164089", 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:37:33.969 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04c00a2f10 msgr2=0x7f04c00a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04c00a2f10 0x7f04c00a53d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0ed2e0 tx=0x7f04cc02a440 comp rx=0 tx=0).stop 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 msgr2=0x7f04dc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 0x7f04dc19a1b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04cc02a960 tx=0x7f04cc003b90 comp rx=0 tx=0).stop 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 shutdown_connections 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04c00a2f10 0x7f04c00a53d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04dc1098c0 0x7f04dc19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc104260 0x7f04dc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 --2- 172.21.15.175:0/3095001595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc1038c0 0x7f04dc19a1b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:33.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 >> 172.21.15.175:0/3095001595 conn(0x7f04dc0ff600 msgr2=0x7f04dc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:33.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.969+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 shutdown_connections 2024-07-29T00:37:33.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:33.970+0000 7f04e1200640 1 -- 172.21.15.175:0/3095001595 wait complete. 2024-07-29T00:37:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:33 smithi181 ceph-mon[36706]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:33 smithi204 ceph-mon[37368]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:37:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:34 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:37:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:34 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:37:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:34 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:37:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:34 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:37:34.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:37:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:34 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:37:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:34 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:37:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:34 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:37:34.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:34 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:37:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:34 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:37:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:34 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:37:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:34 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:37:35.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:34 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:37:35.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 -- 172.21.15.175:0/1728508117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 msgr2=0x7fb92010b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 --2- 172.21.15.175:0/1728508117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb92010b690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009900 tx=0x7fb90c02a4a0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 -- 172.21.15.175:0/1728508117 shutdown_connections 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 --2- 172.21.15.175:0/1728508117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92010bcb0 0x7fb92010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 --2- 172.21.15.175:0/1728508117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb92010b690 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 --2- 172.21.15.175:0/1728508117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 0x7fb920102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.532+0000 7fb926c00640 1 -- 172.21.15.175:0/1728508117 >> 172.21.15.175:0/1728508117 conn(0x7fb9200fc300 msgr2=0x7fb9200fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.533+0000 7fb926c00640 1 -- 172.21.15.175:0/1728508117 shutdown_connections 2024-07-29T00:37:35.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.533+0000 7fb926c00640 1 -- 172.21.15.175:0/1728508117 wait complete. 2024-07-29T00:37:35.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.534+0000 7fb926c00640 1 Processor -- start 2024-07-29T00:37:35.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 -- start start 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 0x7fb9201004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb920100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92010bcb0 0x7fb920100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111b60 con 0x7fb9201031a0 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201119e0 con 0x7fb92010bcb0 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.535+0000 7fb926c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111ce0 con 0x7fb920102880 2024-07-29T00:37:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb920100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb920100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54558/0 (socket says 172.21.15.175:54558) 2024-07-29T00:37:35.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 -- 172.21.15.175:0/2472407692 learned_addr learned my addr 172.21.15.175:0/2472407692 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:35.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb925800640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 0x7fb9201004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb926200640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92010bcb0 0x7fb920100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 -- 172.21.15.175:0/2472407692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 msgr2=0x7fb9201004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 0x7fb9201004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 -- 172.21.15.175:0/2472407692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92010bcb0 msgr2=0x7fb920100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92010bcb0 0x7fb920100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb924e00640 1 -- 172.21.15.175:0/2472407692 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9201a64e0 con 0x7fb9201031a0 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.536+0000 7fb925800640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 0x7fb9201004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.537+0000 7fb924e00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb920100a20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb90c004770 tx=0x7fb90c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.537+0000 7fb91e000640 1 -- 172.21.15.175:0/2472407692 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c03f070 con 0x7fb9201031a0 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.537+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9201a6770 con 0x7fb9201031a0 2024-07-29T00:37:35.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.538+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a6ca0 con 0x7fb9201031a0 2024-07-29T00:37:35.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.538+0000 7fb91e000640 1 -- 172.21.15.175:0/2472407692 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c005d10 con 0x7fb9201031a0 2024-07-29T00:37:35.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.538+0000 7fb91e000640 1 -- 172.21.15.175:0/2472407692 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c035030 con 0x7fb9201031a0 2024-07-29T00:37:35.540 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:37:35.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 Processor -- start 2024-07-29T00:37:35.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 -- start start 2024-07-29T00:37:35.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f22966a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f22966a580 0x55f229674900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f229677230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f2295c6a80 con 0x55f229669c60 2024-07-29T00:37:35.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.542+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f22955e080 con 0x55f229674e40 2024-07-29T00:37:35.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.543+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2295c2a40 con 0x55f22966a580 2024-07-29T00:37:35.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.543+0000 7fb926200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f22966a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.543+0000 7fb926200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f22966a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54568/0 (socket says 172.21.15.175:54568) 2024-07-29T00:37:35.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.543+0000 7fb926200640 1 -- 172.21.15.175:0/407891770 learned_addr learned my addr 172.21.15.175:0/407891770 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:35.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.543+0000 7fb925800640 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f22966a580 0x55f229674900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb926200640 1 -- 172.21.15.175:0/407891770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f22966a580 msgr2=0x55f229674900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb926200640 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f22966a580 0x55f229674900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb926200640 1 -- 172.21.15.175:0/407891770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 msgr2=0x55f229677230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:37:35.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb926200640 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f229677230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb926200640 1 -- 172.21.15.175:0/407891770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f229338b90 con 0x55f229669c60 2024-07-29T00:37:35.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb925800640 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f22966a580 0x55f229674900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:35.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.544+0000 7fb926200640 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f22966a040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb91800a4d0 tx=0x7fb918009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba5a822e2ef4060 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:35.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.545+0000 7fb912a00640 1 -- 172.21.15.175:0/407891770 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918018020 con 0x55f229669c60 2024-07-29T00:37:35.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.545+0000 7fb912a00640 1 -- 172.21.15.175:0/407891770 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb918014cf0 con 0x55f229669c60 2024-07-29T00:37:35.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.545+0000 7fb912a00640 1 -- 172.21.15.175:0/407891770 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918013dd0 con 0x55f229669c60 2024-07-29T00:37:35.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb91e000640 1 -- 172.21.15.175:0/2472407692 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7fb90c042070 con 0x7fb9201031a0 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 -- 172.21.15.175:0/407891770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 msgr2=0x55f22966a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f22966a040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb91800a4d0 tx=0x7fb918009650 comp rx=0 tx=0).stop 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 -- 172.21.15.175:0/407891770 shutdown_connections 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f229677230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f22966a580 0x55f229674900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/407891770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f22966a040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 -- 172.21.15.175:0/407891770 >> 172.21.15.175:0/407891770 conn(0x55f2296661f0 msgr2=0x55f2296665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 -- 172.21.15.175:0/407891770 shutdown_connections 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.546+0000 7fb92ac3d740 1 -- 172.21.15.175:0/407891770 wait complete. 2024-07-29T00:37:35.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.547+0000 7fb92ac3d740 1 -- start start 2024-07-29T00:37:35.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.547+0000 7fb92ac3d740 1 Processor -- start 2024-07-29T00:37:35.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229671c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f229672710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f229688a20 con 0x55f229669c60 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f229581750 con 0x55f22966a580 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f229688ba0 con 0x55f229674e40 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb92ac3d740 10 client.0 _ll_register_callbacks cb 0x55f2292d6090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb91e000640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb9080a2da0 0x7fb9080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb924e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f229672710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb924e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f229672710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35398/0 (socket says 172.21.15.175:35398) 2024-07-29T00:37:35.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.548+0000 7fb924e00640 1 -- 172.21.15.175:0/3726956094 learned_addr learned my addr 172.21.15.175:0/3726956094 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:35.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.549+0000 7fb91e000640 1 -- 172.21.15.175:0/2472407692 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb90c0efb30 con 0x7fb9201031a0 2024-07-29T00:37:35.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.549+0000 7fb926200640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229671c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.549+0000 7fb925800640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.549+0000 7fb926200640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb9080a2da0 0x7fb9080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.550+0000 7fb925800640 1 -- 172.21.15.175:0/3726956094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 msgr2=0x55f229672710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.550+0000 7fb925800640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f229672710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.550+0000 7fb925800640 1 -- 172.21.15.175:0/3726956094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 msgr2=0x55f229671c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.550+0000 7fb925800640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229671c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.550+0000 7fb925800640 1 -- 172.21.15.175:0/3726956094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22965d3b0 con 0x55f22966a580 2024-07-29T00:37:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.550+0000 7fb924e00640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f229672710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.551+0000 7fb925800640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296721d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb914013760 tx=0x7fb914013c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b763f1bc279a1f4b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.551+0000 7fb907400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.551+0000 7fb926200640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb9080a2da0 0x7fb9080a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb918013970 tx=0x7fb91800d040 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.551+0000 7fb907400640 1 -- 172.21.15.175:0/3726956094 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914020060 con 0x55f22966a580 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.551+0000 7fb907400640 1 -- 172.21.15.175:0/3726956094 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb914011dd0 con 0x55f22966a580 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.551+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918014cf0 con 0x55f22966a580 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb907400640 1 -- 172.21.15.175:0/3726956094 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb91400f450 con 0x55f22966a580 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb92ac3d740 10 client.25592 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f22965fd30 con 0x55f22966a580 2024-07-29T00:37:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb907400640 1 -- 172.21.15.175:0/3726956094 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb918014cf0 con 0x55f22966a580 2024-07-29T00:37:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb907400640 1 -- 172.21.15.175:0/3726956094 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb91401a030 con 0x55f22966a580 2024-07-29T00:37:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb92ac3d740 10 client.25592 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb92ac3d740 10 client.25592 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:37:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.552+0000 7fb92ac3d740 20 client.25592 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.553+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f2296611f0 0x55f229663610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.553+0000 7fb92ac3d740 4 client.25592 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.553+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2295d41b0 con 0x55f2296611f0 2024-07-29T00:37:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.553+0000 7fb924e00640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f2296611f0 0x55f229663610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.554+0000 7fb924e00640 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f2296611f0 0x55f229663610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6806d13b909378 server_cookie=a34eb41ffaef25f4 in_seq=0 out_seq=0 2024-07-29T00:37:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.554+0000 7fb907400640 10 client.25592 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.554+0000 7fb907400640 1 -- 172.21.15.175:0/3726956094 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f2295d41b0 con 0x55f2296611f0 2024-07-29T00:37:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.554+0000 7fb907400640 10 client.25592 handle_command_reply: tid=0 2024-07-29T00:37:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb92ac3d740 1 client.25592 shutdown 2024-07-29T00:37:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 msgr2=0x55f2296721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296721d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb914013760 tx=0x7fb914013c20 comp rx=0 tx=0).stop 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 shutdown_connections 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f2296611f0 0x55f229663610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f229672710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.556+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296721d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.556+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/3726956094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229671c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.556+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 >> 172.21.15.175:0/3726956094 conn(0x55f2296661f0 msgr2=0x55f22965cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.555+0000 7fb907400640 0 client.25592 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.556+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 shutdown_connections 2024-07-29T00:37:35.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.556+0000 7fb92ac3d740 1 -- 172.21.15.175:0/3726956094 wait complete. 2024-07-29T00:37:35.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.556+0000 7fb92ac3d740 20 client.25592 trim_cache size 0 max 16384 2024-07-29T00:37:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.590+0000 7fb92ac3d740 1 Processor -- start 2024-07-29T00:37:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 -- start start 2024-07-29T00:37:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229672560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296a7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f2296a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb90000e940 con 0x55f229669c60 2024-07-29T00:37:35.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb900010480 con 0x55f22966a580 2024-07-29T00:37:35.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb90000f010 con 0x55f229674e40 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb926200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f2296a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb926200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f2296a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35404/0 (socket says 172.21.15.175:35404) 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.591+0000 7fb924e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296a7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb926200640 1 -- 172.21.15.175:0/355022666 learned_addr learned my addr 172.21.15.175:0/355022666 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb925800640 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229672560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb924e00640 1 -- 172.21.15.175:0/355022666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 msgr2=0x55f2296a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb924e00640 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f2296a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb924e00640 1 -- 172.21.15.175:0/355022666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 msgr2=0x55f229672560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb924e00640 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229672560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb924e00640 1 -- 172.21.15.175:0/355022666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f229666020 con 0x55f22966a580 2024-07-29T00:37:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb926200640 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f2296a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:37:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb924e00640 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296a7f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0ebd50b125fb88 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.592+0000 7fb912000640 1 -- 172.21.15.175:0/355022666 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0c0200 con 0x55f22966a580 2024-07-29T00:37:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.593+0000 7fb912000640 1 -- 172.21.15.175:0/355022666 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c0be040 con 0x55f22966a580 2024-07-29T00:37:35.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.593+0000 7fb912000640 1 -- 172.21.15.175:0/355022666 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0c4630 con 0x55f22966a580 2024-07-29T00:37:35.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 -- 172.21.15.175:0/355022666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 msgr2=0x55f2296a7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296a7f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c0095c0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 -- 172.21.15.175:0/355022666 shutdown_connections 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229674e40 0x55f2296a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f22966a580 0x55f2296a7f60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/355022666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f229669c60 0x55f229672560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 -- 172.21.15.175:0/355022666 >> 172.21.15.175:0/355022666 conn(0x55f2296661f0 msgr2=0x55f22965cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 -- 172.21.15.175:0/355022666 shutdown_connections 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.594+0000 7fb92ac3d740 1 -- 172.21.15.175:0/355022666 wait complete. 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.595+0000 7fb92ac3d740 1 -- start start 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.595+0000 7fb92ac3d740 1 Processor -- start 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 0x55f22968de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f22966a580 0x55f22968e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f22966df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb90000fdb0 con 0x55f22966a580 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb90000f6e0 con 0x55f229674e40 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb90000c8c0 con 0x55f229669c60 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb92ac3d740 10 client.0 _ll_register_callbacks cb 0x7fb914009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb925800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 0x55f22968de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb925800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 0x55f22968de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35416/0 (socket says 172.21.15.175:35416) 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb925800640 1 -- 172.21.15.175:0/753756363 learned_addr learned my addr 172.21.15.175:0/753756363 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.596+0000 7fb926200640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f22966df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb924e00640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f22966a580 0x55f22968e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb925800640 1 -- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 msgr2=0x55f22966df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb925800640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f22966df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb925800640 1 -- 172.21.15.175:0/753756363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f22966a580 msgr2=0x55f22968e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb925800640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f22966a580 0x55f22968e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb925800640 1 -- 172.21.15.175:0/753756363 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22966cad0 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb924e00640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f22966a580 0x55f22968e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb926200640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f22966df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb925800640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 0x55f22968de80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb9140059f0 tx=0x7fb914009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=829d7a665729e501 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb907400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.597+0000 7fb907400640 1 -- 172.21.15.175:0/753756363 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400f450 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.598+0000 7fb907400640 1 -- 172.21.15.175:0/753756363 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb914011dd0 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.598+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb90c0be040 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.598+0000 7fb907400640 1 -- 172.21.15.175:0/753756363 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb9140050c0 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.598+0000 7fb92ac3d740 10 client.25594 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.598+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f22966f530 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb907400640 1 -- 172.21.15.175:0/753756363 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c0be040 con 0x55f229669c60 2024-07-29T00:37:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb907400640 1 -- 172.21.15.175:0/753756363 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb91401a040 con 0x55f229669c60 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb92ac3d740 10 client.25594 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb92ac3d740 10 client.25594 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb92ac3d740 20 client.25594 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f229709f30 0x55f22970c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb92ac3d740 4 client.25594 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.7, "await_for": 45.0}] 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.599+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.7, "await_for": 45.0}) -- 0x55f2295ce500 con 0x55f229709f30 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.600+0000 7fb924e00640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f229709f30 0x55f22970c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.600+0000 7fb924e00640 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f229709f30 0x55f22970c320 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d22b6948fa9da0 server_cookie=12a2743c8470e3c1 in_seq=0 out_seq=0 2024-07-29T00:37:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.600+0000 7fb907400640 10 client.25594 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:35.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.616+0000 7fb907400640 1 -- 172.21.15.175:0/753756363 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f2295ce500 con 0x55f229709f30 2024-07-29T00:37:35.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.616+0000 7fb907400640 10 client.25594 handle_command_reply: tid=0 2024-07-29T00:37:35.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.616+0000 7fb92ac3d740 1 client.25594 shutdown 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.616+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 msgr2=0x55f22968de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.616+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 0x55f22968de80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb9140059f0 tx=0x7fb914009760 comp rx=0 tx=0).stop 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 shutdown_connections 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f229709f30 0x55f22970c320 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f229674e40 0x55f22966df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f22966a580 0x55f22968e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 --2- 172.21.15.175:0/753756363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f229669c60 0x55f22968de80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 >> 172.21.15.175:0/753756363 conn(0x55f2296661f0 msgr2=0x55f229673300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:35.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb907400640 0 client.25594 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:35.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 shutdown_connections 2024-07-29T00:37:35.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.617+0000 7fb92ac3d740 1 -- 172.21.15.175:0/753756363 wait complete. 2024-07-29T00:37:35.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.618+0000 7fb92ac3d740 20 client.25594 trim_cache size 0 max 16384 2024-07-29T00:37:35.619 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:35.619 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 29, 2024-07-29T00:37:35.619 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:37:35.619 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 2, 2024-07-29T00:37:35.619 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:37:35.619 INFO:teuthology.orchestra.run.smithi175.stdout: "b5690169": { 2024-07-29T00:37:35.620 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 2, 2024-07-29T00:37:35.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:37:35.620 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:37:35.620 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:37:35.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 149.7, 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:37:35.621 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:37:35.622 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:37:35.622 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:37:35.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:35.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:35.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb9080a2da0 msgr2=0x7fb9080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb9080a2da0 0x7fb9080a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb918013970 tx=0x7fb91800d040 comp rx=0 tx=0).stop 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 msgr2=0x7fb920100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:35.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb920100a20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb90c004770 tx=0x7fb90c003b90 comp rx=0 tx=0).stop 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 shutdown_connections 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92010bcb0 0x7fb920100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb9080a2da0 0x7fb9080a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9201031a0 0x7fb920100a20 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb90c004770 tx=0x7fb90c003b90 comp rx=0 tx=0).stop 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 --2- 172.21.15.175:0/2472407692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb920102880 0x7fb9201004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.619+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 >> 172.21.15.175:0/2472407692 conn(0x7fb9200fc300 msgr2=0x7fb9200fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.620+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 shutdown_connections 2024-07-29T00:37:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:35.620+0000 7fb926c00640 1 -- 172.21.15.175:0/2472407692 wait complete. 2024-07-29T00:37:35.693 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:35 smithi175 ceph-mon[32138]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:35 smithi181 ceph-mon[36706]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:35 smithi204 ceph-mon[37368]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:37:36.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 -- 172.21.15.175:0/1692278838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 msgr2=0x7faa5c10e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 --2- 172.21.15.175:0/1692278838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c10e150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa54009ca0 tx=0x7faa5402a950 comp rx=0 tx=0).stop 2024-07-29T00:37:36.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 -- 172.21.15.175:0/1692278838 shutdown_connections 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 --2- 172.21.15.175:0/1692278838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c10e150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 --2- 172.21.15.175:0/1692278838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa5c1031a0 0x7faa5c10b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 --2- 172.21.15.175:0/1692278838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 0x7faa5c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.226+0000 7faa61c00640 1 -- 172.21.15.175:0/1692278838 >> 172.21.15.175:0/1692278838 conn(0x7faa5c0fc300 msgr2=0x7faa5c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.227+0000 7faa61c00640 1 -- 172.21.15.175:0/1692278838 shutdown_connections 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.227+0000 7faa61c00640 1 -- 172.21.15.175:0/1692278838 wait complete. 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.228+0000 7faa61c00640 1 Processor -- start 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.228+0000 7faa61c00640 1 -- start start 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 0x7faa5c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa5c1031a0 0x7faa5c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c111a70 con 0x7faa5c10bcb0 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1118f0 con 0x7faa5c1031a0 2024-07-29T00:37:36.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c111bf0 con 0x7faa5c102880 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa5b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa5c1031a0 0x7faa5c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa5be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 0x7faa5c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c19ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54618/0 (socket says 172.21.15.175:54618) 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.229+0000 7faa61200640 1 -- 172.21.15.175:0/2784742142 learned_addr learned my addr 172.21.15.175:0/2784742142 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa5be00640 1 -- 172.21.15.175:0/2784742142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa5c1031a0 msgr2=0x7faa5c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa5be00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa5c1031a0 0x7faa5c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa5be00640 1 -- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 msgr2=0x7faa5c19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa5be00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa5be00640 1 -- 172.21.15.175:0/2784742142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c19f0e0 con 0x7faa5c102880 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa5be00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 0x7faa5c19a130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00d6d0 tx=0x7faa4c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa58c00640 1 -- 172.21.15.175:0/2784742142 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c017070 con 0x7faa5c102880 2024-07-29T00:37:36.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.230+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5c19f3d0 con 0x7faa5c102880 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.231+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5c1a6cb0 con 0x7faa5c102880 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.231+0000 7faa58c00640 1 -- 172.21.15.175:0/2784742142 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4c009820 con 0x7faa5c102880 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.231+0000 7faa58c00640 1 -- 172.21.15.175:0/2784742142 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c004cd0 con 0x7faa5c102880 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 Processor -- start 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 -- start start 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x56519037def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037e430 0x5651903887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x56519038b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5651902e2ad0 con 0x56519037db10 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56519027aaf0 con 0x56519037e430 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.234+0000 7faa65bf7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5651902dea90 con 0x565190388cf0 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x56519037def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x56519037def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54624/0 (socket says 172.21.15.175:54624) 2024-07-29T00:37:36.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 -- 172.21.15.175:0/2975699994 learned_addr learned my addr 172.21.15.175:0/2975699994 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa5b400640 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x56519038b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 -- 172.21.15.175:0/2975699994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 msgr2=0x56519038b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x56519038b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 -- 172.21.15.175:0/2975699994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037e430 msgr2=0x5651903887b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037e430 0x5651903887b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.235+0000 7faa61200640 1 -- 172.21.15.175:0/2975699994 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651900543c0 con 0x56519037db10 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.236+0000 7faa58c00640 1 -- 172.21.15.175:0/2784742142 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 138067+0+0 (secure 0 0 0) 0x7faa4c005220 con 0x7faa5c102880 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.236+0000 7faa61200640 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x56519037def0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faa5402fef0 tx=0x7faa54005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a978fb79e65835 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.236+0000 7faa51600640 1 -- 172.21.15.175:0/2975699994 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54048070 con 0x56519037db10 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.236+0000 7faa51600640 1 -- 172.21.15.175:0/2975699994 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5402f940 con 0x56519037db10 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.236+0000 7faa51600640 1 -- 172.21.15.175:0/2975699994 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54034040 con 0x56519037db10 2024-07-29T00:37:36.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.237+0000 7faa65bf7740 1 -- 172.21.15.175:0/2975699994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 msgr2=0x56519037def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.237+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x56519037def0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faa5402fef0 tx=0x7faa54005fa0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 -- 172.21.15.175:0/2975699994 shutdown_connections 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x56519038b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037e430 0x5651903887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2975699994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x56519037def0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 -- 172.21.15.175:0/2975699994 >> 172.21.15.175:0/2975699994 conn(0x56519037a0a0 msgr2=0x56519037a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 -- 172.21.15.175:0/2975699994 shutdown_connections 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 -- 172.21.15.175:0/2975699994 wait complete. 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa58c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa400a2da0 0x7faa400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa58c00640 1 -- 172.21.15.175:0/2784742142 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7faa4c0cedf0 con 0x7faa5c102880 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa5b400640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa400a2da0 0x7faa400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.238+0000 7faa65bf7740 1 -- start start 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.239+0000 7faa65bf7740 1 Processor -- start 2024-07-29T00:37:36.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.239+0000 7faa5b400640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa400a2da0 0x7faa400a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56519037b420 tx=0x7faa48005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.239+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x565190385c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.239+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 0x565190386140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.239+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x565190386680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.239+0000 7faa65bf7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56519038b8c0 con 0x56519037e430 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.240+0000 7faa5b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 0x565190386140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.240+0000 7faa5b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 0x565190386140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54626/0 (socket says 172.21.15.175:54626) 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.240+0000 7faa5b400640 1 -- 172.21.15.175:0/3505645267 learned_addr learned my addr 172.21.15.175:0/3505645267 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.240+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56519029dbc0 con 0x56519037db10 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.240+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56519038ba40 con 0x565190388cf0 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.240+0000 7faa65bf7740 10 client.0 _ll_register_callbacks cb 0x565190289d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5be00640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x565190385c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa61200640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x565190386680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5b400640 1 -- 172.21.15.175:0/3505645267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 msgr2=0x565190386680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5b400640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x565190386680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5b400640 1 -- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 msgr2=0x565190385c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5b400640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x565190385c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5b400640 1 -- 172.21.15.175:0/3505645267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519038bde0 con 0x56519037e430 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa61200640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x565190386680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.241+0000 7faa5be00640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x565190385c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.242+0000 7faa5b400640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 0x565190386140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faa480045e0 tx=0x7faa48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28d4e9dcc1605f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.242+0000 7faa46000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.242+0000 7faa46000640 1 -- 172.21.15.175:0/3505645267 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4801e470 con 0x56519037e430 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.242+0000 7faa46000640 1 -- 172.21.15.175:0/3505645267 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7faa4801e610 con 0x56519037e430 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.243+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5402f940 con 0x56519037e430 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.243+0000 7faa46000640 1 -- 172.21.15.175:0/3505645267 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa4801e810 con 0x56519037e430 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.243+0000 7faa65bf7740 10 client.15504 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.243+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56519038e800 con 0x56519037e430 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa46000640 1 -- 172.21.15.175:0/3505645267 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5402f940 con 0x56519037e430 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa46000640 1 -- 172.21.15.175:0/3505645267 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa48027070 con 0x56519037e430 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa65bf7740 10 client.15504 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa65bf7740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa65bf7740 20 client.15504 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56519038fc60 0x5651903920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa65bf7740 4 client.15504 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.244+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651902f0040 con 0x56519038fc60 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.245+0000 7faa5be00640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56519038fc60 0x5651903920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.245+0000 7faa5be00640 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56519038fc60 0x5651903920e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9decef15e50c5de server_cookie=777d184b8d996b22 in_seq=0 out_seq=0 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.245+0000 7faa46000640 10 client.15504 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa46000640 1 -- 172.21.15.175:0/3505645267 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5651902f0040 con 0x56519038fc60 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa46000640 10 client.15504 handle_command_reply: tid=0 2024-07-29T00:37:36.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 client.15504 shutdown 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 msgr2=0x565190386140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 0x565190386140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faa480045e0 tx=0x7faa48004730 comp rx=0 tx=0).stop 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 shutdown_connections 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56519038fc60 0x5651903920e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565190388cf0 0x565190386680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037e430 0x565190386140 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3505645267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x565190385c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.246+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 >> 172.21.15.175:0/3505645267 conn(0x56519037a0a0 msgr2=0x56519037fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.247+0000 7faa46000640 0 client.15504 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.247+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 shutdown_connections 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.247+0000 7faa65bf7740 1 -- 172.21.15.175:0/3505645267 wait complete. 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.247+0000 7faa65bf7740 20 client.15504 trim_cache size 0 max 16384 2024-07-29T00:37:36.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.281+0000 7faa65bf7740 1 Processor -- start 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.281+0000 7faa65bf7740 1 -- start start 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x5651903805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x5651903b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565190388cf0 0x5651903b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa65bf7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c00e850 con 0x565190388cf0 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa65bf7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c010390 con 0x56519037db10 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa65bf7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c00ef20 con 0x56519037e430 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa61200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565190388cf0 0x5651903b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa5b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x5651903b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa5b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x5651903b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35466/0 (socket says 172.21.15.175:35466) 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa5be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x5651903805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa5b400640 1 -- 172.21.15.175:0/2254928560 learned_addr learned my addr 172.21.15.175:0/2254928560 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.282+0000 7faa61200640 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565190388cf0 0x5651903b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54630/0 (socket says 172.21.15.175:54630) 2024-07-29T00:37:36.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa5b400640 1 -- 172.21.15.175:0/2254928560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 msgr2=0x5651903805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa5b400640 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x5651903805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa5b400640 1 -- 172.21.15.175:0/2254928560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565190388cf0 msgr2=0x5651903b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa5b400640 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565190388cf0 0x5651903b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa5b400640 1 -- 172.21.15.175:0/2254928560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565190379b40 con 0x56519037e430 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa5b400640 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x5651903b35e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa48004470 tx=0x7faa48003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c9000d0722ff88 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa50c00640 1 -- 172.21.15.175:0/2254928560 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4801e570 con 0x56519037e430 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa50c00640 1 -- 172.21.15.175:0/2254928560 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa48038020 con 0x56519037e430 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.283+0000 7faa50c00640 1 -- 172.21.15.175:0/2254928560 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48033ae0 con 0x56519037e430 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 -- 172.21.15.175:0/2254928560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 msgr2=0x5651903b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x5651903b35e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa48004470 tx=0x7faa48003ef0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 -- 172.21.15.175:0/2254928560 shutdown_connections 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565190388cf0 0x5651903b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x5651903b35e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 --2- 172.21.15.175:0/2254928560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56519037db10 0x5651903805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.284+0000 7faa65bf7740 1 -- 172.21.15.175:0/2254928560 >> 172.21.15.175:0/2254928560 conn(0x56519037a0a0 msgr2=0x56519039c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.285+0000 7faa65bf7740 1 -- 172.21.15.175:0/2254928560 shutdown_connections 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.285+0000 7faa65bf7740 1 -- 172.21.15.175:0/2254928560 wait complete. 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.285+0000 7faa65bf7740 1 -- start start 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.285+0000 7faa65bf7740 1 Processor -- start 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x565190397a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x565190397f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565190388cf0 0x5651903949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c00fcc0 con 0x56519037db10 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c00f5f0 con 0x565190388cf0 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c00c830 con 0x56519037e430 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa65bf7740 10 client.0 _ll_register_callbacks cb 0x7faa480044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:37:36.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa5be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x565190397a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa5be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x565190397a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54638/0 (socket says 172.21.15.175:54638) 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa5be00640 1 -- 172.21.15.175:0/3327588401 learned_addr learned my addr 172.21.15.175:0/3327588401 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa5b400640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x565190397f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.286+0000 7faa61200640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565190388cf0 0x5651903949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa5be00640 1 -- 172.21.15.175:0/3327588401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 msgr2=0x565190397f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa5be00640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x565190397f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa5be00640 1 -- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565190388cf0 msgr2=0x5651903949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa5be00640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565190388cf0 0x5651903949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa5be00640 1 -- 172.21.15.175:0/3327588401 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565190394f10 con 0x56519037db10 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa61200640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565190388cf0 0x5651903949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa5be00640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x565190397a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00d6a0 tx=0x7faa4c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896c269411033b49 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa46000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa46000640 1 -- 172.21.15.175:0/3327588401 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c014030 con 0x56519037db10 2024-07-29T00:37:36.420 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa46000640 1 -- 172.21.15.175:0/3327588401 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7faa4c0046c0 con 0x56519037db10 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa48038020 con 0x56519037db10 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa46000640 1 -- 172.21.15.175:0/3327588401 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa4c020040 con 0x56519037db10 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa65bf7740 10 client.15510 fetch_fsmap learned FSMap version 29 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.287+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565190414de0 con 0x56519037db10 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa46000640 1 -- 172.21.15.175:0/3327588401 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa48038020 con 0x56519037db10 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa46000640 1 -- 172.21.15.175:0/3327588401 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa4c0b7070 con 0x56519037db10 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa65bf7740 10 client.15510 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa65bf7740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa65bf7740 20 client.15510 populate_metadata read hostname 'smithi175' 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5651904154e0 0x565190417900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa65bf7740 4 client.15510 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.288+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651902ea550 con 0x5651904154e0 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.289+0000 7faa5b400640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5651904154e0 0x565190417900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:37:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.289+0000 7faa5b400640 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5651904154e0 0x565190417900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac43bfe276cf02d server_cookie=4f694a89d59581f2 in_seq=0 out_seq=0 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.289+0000 7faa46000640 10 client.15510 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.289+0000 7faa46000640 1 -- 172.21.15.175:0/3327588401 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5651902ea550 con 0x5651904154e0 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.289+0000 7faa46000640 10 client.15510 handle_command_reply: tid=0 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.289+0000 7faa65bf7740 1 client.15510 shutdown 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 msgr2=0x565190397a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x565190397a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00d6a0 tx=0x7faa4c0bb040 comp rx=0 tx=0).stop 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 shutdown_connections 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5651904154e0 0x565190417900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565190388cf0 0x5651903949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519037e430 0x565190397f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 --2- 172.21.15.175:0/3327588401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56519037db10 0x565190397a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 >> 172.21.15.175:0/3327588401 conn(0x56519037a0a0 msgr2=0x565190387460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa46000640 0 client.15510 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:37:36.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 shutdown_connections 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 1 -- 172.21.15.175:0/3327588401 wait complete. 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: "a77ed333-b732-4634-acc2-3b8441164089": { 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "a77ed333-b732-4634-acc2-3b8441164089", 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:36.423 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.290+0000 7faa65bf7740 20 client.15510 trim_cache size 0 max 16384 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa400a2da0 msgr2=0x7faa400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa400a2da0 0x7faa400a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56519037b420 tx=0x7faa48005f90 comp rx=0 tx=0).stop 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 msgr2=0x7faa5c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 0x7faa5c19a130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00d6d0 tx=0x7faa4c00db90 comp rx=0 tx=0).stop 2024-07-29T00:37:36.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 shutdown_connections 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa400a2da0 0x7faa400a5260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c10bcb0 0x7faa5c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa5c1031a0 0x7faa5c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 --2- 172.21.15.175:0/2784742142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa5c102880 0x7faa5c19a130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 >> 172.21.15.175:0/2784742142 conn(0x7faa5c0fc300 msgr2=0x7faa5c0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.291+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 shutdown_connections 2024-07-29T00:37:36.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:37:36.292+0000 7faa61c00640 1 -- 172.21.15.175:0/2784742142 wait complete. 2024-07-29T00:37:37.429 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b5690169', quiesce took 7.0 seconds. Will release after: 22.7 2024-07-29T00:37:37.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:37 smithi175 ceph-mon[32138]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-29T00:37:37.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:37 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:37:37.928 INFO:tasks.cephfs.filesystem:scrub status for tag:a77ed333-b732-4634-acc2-3b8441164089 - {'path': '/', 'tag': 'a77ed333-b732-4634-acc2-3b8441164089', 'options': 'recursive,force'} 2024-07-29T00:37:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:37 smithi181 ceph-mon[36706]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-29T00:37:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:37 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:37:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:37 smithi204 ceph-mon[37368]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-29T00:37:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:37 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:37:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:39 smithi181 ceph-mon[36706]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:40.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:39 smithi175 ceph-mon[32138]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:39 smithi204 ceph-mon[37368]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:42.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:41 smithi181 ceph-mon[36706]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:42.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:41 smithi175 ceph-mon[32138]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:41 smithi204 ceph-mon[37368]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:43.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:43 smithi181 ceph-mon[36706]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-29T00:37:43.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:43 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:37:44.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:43 smithi175 ceph-mon[32138]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-29T00:37:44.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:43 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:37:44.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:43 smithi204 ceph-mon[37368]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-29T00:37:44.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:43 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:37:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:37:45.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:44 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:37:45.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:44 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:37:45.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:37:45.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:44 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:37:47.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:47 smithi175 ceph-mon[32138]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-07-29T00:37:47.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:47 smithi204 ceph-mon[37368]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-07-29T00:37:47.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:47 smithi181 ceph-mon[36706]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-07-29T00:37:49.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:49 smithi175 ceph-mon[32138]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:49.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:49 smithi204 ceph-mon[37368]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:49.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:49 smithi181 ceph-mon[36706]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:51.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:51 smithi175 ceph-mon[32138]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:51.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:51 smithi204 ceph-mon[37368]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:51.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:51 smithi181 ceph-mon[36706]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:53.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:53 smithi175 ceph-mon[32138]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-07-29T00:37:53.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:53 smithi204 ceph-mon[37368]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-07-29T00:37:53.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:53 smithi181 ceph-mon[36706]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-07-29T00:37:55.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:55 smithi175 ceph-mon[32138]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:55.301 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:55 smithi181 ceph-mon[36706]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:55.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:55 smithi204 ceph-mon[37368]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-29T00:37:57.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:57 smithi204 ceph-mon[37368]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-07-29T00:37:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:57 smithi181 ceph-mon[36706]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-07-29T00:37:58.033 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:57 smithi175 ceph-mon[32138]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-07-29T00:37:59.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:37:59 smithi204 ceph-mon[37368]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:37:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:37:59 smithi181 ceph-mon[36706]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:38:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:37:59 smithi175 ceph-mon[32138]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:38:00.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:38:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:01.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:01 smithi204 ceph-mon[37368]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:38:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:01 smithi181 ceph-mon[36706]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:38:02.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:01 smithi175 ceph-mon[32138]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-07-29T00:38:03.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:03 smithi204 ceph-mon[37368]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:38:03.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:03 smithi204 ceph-mon[37368]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-07-29T00:38:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:03 smithi181 ceph-mon[36706]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:38:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:03 smithi181 ceph-mon[36706]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-07-29T00:38:04.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:03 smithi175 ceph-mon[32138]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:38:04.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:03 smithi175 ceph-mon[32138]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-07-29T00:38:04.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 -- 172.21.15.175:0/2876454413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50106750 msgr2=0x7f9a50106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:04.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 --2- 172.21.15.175:0/2876454413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50106750 0x7f9a50106b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009900 tx=0x7f9a4002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 -- 172.21.15.175:0/2876454413 shutdown_connections 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 --2- 172.21.15.175:0/2876454413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a501010f0 0x7f9a5010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 --2- 172.21.15.175:0/2876454413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a50100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 --2- 172.21.15.175:0/2876454413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50106750 0x7f9a50106b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 -- 172.21.15.175:0/2876454413 >> 172.21.15.175:0/2876454413 conn(0x7f9a500fc470 msgr2=0x7f9a500fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 -- 172.21.15.175:0/2876454413 shutdown_connections 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.899+0000 7f9a57200640 1 -- 172.21.15.175:0/2876454413 wait complete. 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 Processor -- start 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 -- start start 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a5019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a501010f0 0x7f9a5019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:04.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a50106750 0x7f9a5019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50111cb0 con 0x7f9a50106750 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50111b30 con 0x7f9a501010f0 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.901+0000 7f9a57200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50111e30 con 0x7f9a50100750 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a5019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a501010f0 0x7f9a5019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a5019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49976/0 (socket says 172.21.15.175:49976) 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a55200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a50106750 0x7f9a5019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 -- 172.21.15.175:0/353356180 learned_addr learned my addr 172.21.15.175:0/353356180 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 -- 172.21.15.175:0/353356180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a501010f0 msgr2=0x7f9a5019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:04.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a501010f0 0x7f9a5019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 -- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a50106750 msgr2=0x7f9a5019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a50106750 0x7f9a5019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.902+0000 7f9a4fe00640 1 -- 172.21.15.175:0/353356180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5019f280 con 0x7f9a50100750 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.903+0000 7f9a4fe00640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a5019a2d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a4002a9e0 tx=0x7f9a40003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.903+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4003f070 con 0x7f9a50100750 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.903+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40031c30 con 0x7f9a50100750 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.903+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5019f510 con 0x7f9a50100750 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.903+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40035030 con 0x7f9a50100750 2024-07-29T00:38:04.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.903+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a501a6e50 con 0x7f9a50100750 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.905+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a50101e90 con 0x7f9a50100750 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.908+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9a40042070 con 0x7f9a50100750 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.909+0000 7f9a4cc00640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9a340a2d70 0x7f9a340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.909+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f9a400ef9a0 con 0x7f9a50100750 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.910+0000 7f9a4f400640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9a340a2d70 0x7f9a340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.910+0000 7f9a4f400640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9a340a2d70 0x7f9a340a5230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a5010d2e0 tx=0x7f9a3c005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:04.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:04.913+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9a40031de0 con 0x7f9a50100750 2024-07-29T00:38:05.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.147+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a501a7020 con 0x7f9a50100750 2024-07-29T00:38:05.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.149+0000 7f9a4cc00640 1 -- 172.21.15.175:0/353356180 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f9a400bbf50 con 0x7f9a50100750 2024-07-29T00:38:05.149 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:38:05.150 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:38:05.150 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:38:05.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9a340a2d70 msgr2=0x7f9a340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:05.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9a340a2d70 0x7f9a340a5230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a5010d2e0 tx=0x7f9a3c005f90 comp rx=0 tx=0).stop 2024-07-29T00:38:05.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 msgr2=0x7f9a5019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:05.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a5019a2d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a4002a9e0 tx=0x7f9a40003c10 comp rx=0 tx=0).stop 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 shutdown_connections 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9a340a2d70 0x7f9a340a5230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a50106750 0x7f9a5019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a501010f0 0x7f9a5019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 --2- 172.21.15.175:0/353356180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a50100750 0x7f9a5019a2d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 >> 172.21.15.175:0/353356180 conn(0x7f9a500fc470 msgr2=0x7f9a500fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 shutdown_connections 2024-07-29T00:38:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:05.152+0000 7f9a57200640 1 -- 172.21.15.175:0/353356180 wait complete. 2024-07-29T00:38:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:05 smithi204 ceph-mon[37368]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:05.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:05 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/353356180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:05 smithi181 ceph-mon[36706]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:05 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/353356180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:06.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:05 smithi175 ceph-mon[32138]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:06.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:05 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/353356180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:06.219 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id b5690169 --release --await' 2024-07-29T00:38:06.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db --set-id b5690169 --release --await 2024-07-29T00:38:06.656 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:07.045 INFO:tasks.workunit.client.0.smithi175.stderr:Updating files: 61% (7856/12778) Updating files: 62% (7923/12778) Updating files: 63% (8051/12778) Updating files: 64% (8178/12778) Updating files: 65% (8306/12778) Updating files: 66% (8434/12778) Updating files: 67% (8562/12778) Updating files: 68% (8690/12778) Updating files: 69% (8817/12778) Updating files: 70% (8945/12778) Updating files: 71% (9073/12778) Updating files: 72% (9201/12778) Updating files: 73% (9328/12778) Updating files: 74% (9456/12778) Updating files: 75% (9584/12778) Updating files: 76% (9712/12778) Updating files: 77% (9840/12778) Updating files: 78% (9967/12778) Updating files: 79% (10095/12778) Updating files: 80% (10223/12778) Updating files: 81% (10351/12778) Updating files: 82% (10478/12778) Updating files: 83% (10606/12778) Updating files: 84% (10734/12778) Updating files: 85% (10862/12778) Updating files: 86% (10990/12778) Updating files: 87% (11117/12778) Updating files: 88% (11245/12778) Updating files: 89% (11373/12778) Updating files: 90% (11501/12778) Updating files: 91% (11628/12778) Updating files: 92% (11756/12778) Updating files: 93% (11884/12778) Updating files: 94% (12012/12778) Updating files: 95% (12140/12778) Updating files: 96% (12267/12778) Updating files: 97% (12395/12778) Updating files: 97% (12450/12778) Updating files: 98% (12523/12778) Updating files: 99% (12651/12778) Updating files: 100% (12778/12778) Updating files: 100% (12778/12778), done. 2024-07-29T00:38:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 -- 172.21.15.175:0/2259872463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 msgr2=0x7fa220103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 --2- 172.21.15.175:0/2259872463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa220103ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009930 tx=0x7fa20c02a420 comp rx=0 tx=0).stop 2024-07-29T00:38:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 -- 172.21.15.175:0/2259872463 shutdown_connections 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 --2- 172.21.15.175:0/2259872463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa220104420 0x7fa220110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 --2- 172.21.15.175:0/2259872463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa220103ee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 --2- 172.21.15.175:0/2259872463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa220109a80 0x7fa220109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 -- 172.21.15.175:0/2259872463 >> 172.21.15.175:0/2259872463 conn(0x7fa2200ff780 msgr2=0x7fa220101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 -- 172.21.15.175:0/2259872463 shutdown_connections 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.343+0000 7fa227000640 1 -- 172.21.15.175:0/2259872463 wait complete. 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.344+0000 7fa227000640 1 Processor -- start 2024-07-29T00:38:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 -- start start 2024-07-29T00:38:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa22019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa220104420 0x7fa22019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa220109a80 0x7fa22019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113ed0 con 0x7fa220104420 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113d50 con 0x7fa220109a80 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.345+0000 7fa227000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa220114050 con 0x7fa220103a80 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa22019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa22019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49980/0 (socket says 172.21.15.175:49980) 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 -- 172.21.15.175:0/457714161 learned_addr learned my addr 172.21.15.175:0/457714161 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa225000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa220109a80 0x7fa22019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21f400640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa220104420 0x7fa22019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 -- 172.21.15.175:0/457714161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa220109a80 msgr2=0x7fa22019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa220109a80 0x7fa22019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 -- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa220104420 msgr2=0x7fa22019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa220104420 0x7fa22019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 -- 172.21.15.175:0/457714161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22019f380 con 0x7fa220103a80 2024-07-29T00:38:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.346+0000 7fa21fe00640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa22019a3d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e9f0 tx=0x7fa21000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.347+0000 7fa21cc00640 1 -- 172.21.15.175:0/457714161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21000cd80 con 0x7fa220103a80 2024-07-29T00:38:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.347+0000 7fa21cc00640 1 -- 172.21.15.175:0/457714161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa210005500 con 0x7fa220103a80 2024-07-29T00:38:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.347+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22019f670 con 0x7fa220103a80 2024-07-29T00:38:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.347+0000 7fa21cc00640 1 -- 172.21.15.175:0/457714161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210010690 con 0x7fa220103a80 2024-07-29T00:38:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.347+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a6f50 con 0x7fa220103a80 2024-07-29T00:38:07.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.350+0000 7fa2299d9740 1 Processor -- start 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 -- start start 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558d1860fcc0 con 0x558d186b4de0 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558d185a7bc0 con 0x558d186a9c00 2024-07-29T00:38:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa2299d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558d1860bc80 con 0x558d186aa520 2024-07-29T00:38:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa21f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa225000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa225000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186a9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53432/0 (socket says 172.21.15.175:53432) 2024-07-29T00:38:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.351+0000 7fa225000640 1 -- 172.21.15.175:0/212922106 learned_addr learned my addr 172.21.15.175:0/212922106 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa21cc00640 1 -- 172.21.15.175:0/457714161 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa2100046c0 con 0x7fa220103a80 2024-07-29T00:38:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa21f400640 1 -- 172.21.15.175:0/212922106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 msgr2=0x558d186b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa21f400640 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa21f400640 1 -- 172.21.15.175:0/212922106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 msgr2=0x558d186a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa21f400640 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa21f400640 1 -- 172.21.15.175:0/212922106 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1831f580 con 0x558d186b4de0 2024-07-29T00:38:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.352+0000 7fa225000640 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.353+0000 7fa21f400640 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b71d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa20c02a940 tx=0x7fa20c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd97a9a909abe93a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.353+0000 7fa215600640 1 -- 172.21.15.175:0/212922106 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c035030 con 0x558d186b4de0 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.353+0000 7fa215600640 1 -- 172.21.15.175:0/212922106 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20c0043c0 con 0x558d186b4de0 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.353+0000 7fa215600640 1 -- 172.21.15.175:0/212922106 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004d70 con 0x558d186b4de0 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa21cc00640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa2040a2cf0 0x7fa2040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa21cc00640 1 -- 172.21.15.175:0/457714161 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa2100cf8a0 con 0x7fa220103a80 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 -- 172.21.15.175:0/212922106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 msgr2=0x558d186b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b71d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa20c02a940 tx=0x7fa20c005e60 comp rx=0 tx=0).stop 2024-07-29T00:38:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa225000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa2040a2cf0 0x7fa2040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 -- 172.21.15.175:0/212922106 shutdown_connections 2024-07-29T00:38:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b71d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 --2- 172.21.15.175:0/212922106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.354+0000 7fa2299d9740 1 -- 172.21.15.175:0/212922106 >> 172.21.15.175:0/212922106 conn(0x558d186a6190 msgr2=0x558d186a6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.355+0000 7fa2299d9740 1 -- 172.21.15.175:0/212922106 shutdown_connections 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.355+0000 7fa2299d9740 1 -- 172.21.15.175:0/212922106 wait complete. 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.355+0000 7fa225000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa2040a2cf0 0x7fa2040a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa218005f40 tx=0x7fa2180096a0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.355+0000 7fa2299d9740 1 -- start start 2024-07-29T00:38:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 Processor -- start 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558d186b79b0 con 0x558d186b4de0 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558d185caca0 con 0x558d186a9c00 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558d186b7b30 con 0x558d186aa520 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.356+0000 7fa2299d9740 10 client.0 _ll_register_callbacks cb 0x558d185b6e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49996/0 (socket says 172.21.15.175:49996) 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21f400640 1 -- 172.21.15.175:0/2089379369 learned_addr learned my addr 172.21.15.175:0/2089379369 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21fe00640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa225000640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21fe00640 1 -- 172.21.15.175:0/2089379369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 msgr2=0x558d186b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21fe00640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21fe00640 1 -- 172.21.15.175:0/2089379369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 msgr2=0x558d186b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21fe00640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21fe00640 1 -- 172.21.15.175:0/2089379369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d183e9960 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.357+0000 7fa21f400640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa21fe00640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186b1df0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ccf40 tx=0x7fa21009b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8c3f2724ddf474 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa20a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa20a000640 1 -- 172.21.15.175:0/2089379369 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100cd400 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa20a000640 1 -- 172.21.15.175:0/2089379369 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa2100a4720 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa20c0043c0 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa20a000640 1 -- 172.21.15.175:0/2089379369 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa2100ba070 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa2299d9740 10 client.25628 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.358+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d186c3780 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa20a000640 1 -- 172.21.15.175:0/2089379369 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20c0043c0 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa20a000640 1 -- 172.21.15.175:0/2089379369 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa21009ac00 con 0x558d186a9c00 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa2299d9740 10 client.25628 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:07.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa2299d9740 10 client.25628 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:38:07.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa2299d9740 20 client.25628 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:07.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d186c4b90 0x558d186c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa2299d9740 4 client.25628 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:07.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d1861d230 con 0x558d186c4b90 2024-07-29T00:38:07.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.359+0000 7fa21f400640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d186c4b90 0x558d186c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.360+0000 7fa21f400640 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d186c4b90 0x558d186c6fe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a9400786b0b4a4 server_cookie=36dcb6d8cd5ea14d in_seq=0 out_seq=0 2024-07-29T00:38:07.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.360+0000 7fa20a000640 10 client.25628 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:07.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa20a000640 1 -- 172.21.15.175:0/2089379369 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558d1861d230 con 0x558d186c4b90 2024-07-29T00:38:07.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa20a000640 10 client.25628 handle_command_reply: tid=0 2024-07-29T00:38:07.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 client.25628 shutdown 2024-07-29T00:38:07.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 msgr2=0x558d186b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186b1df0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ccf40 tx=0x7fa21009b940 comp rx=0 tx=0).stop 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 shutdown_connections 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d186c4b90 0x558d186c6fe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2089379369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186b1df0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 >> 172.21.15.175:0/2089379369 conn(0x558d186a6190 msgr2=0x558d186abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa20a000640 0 client.25628 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 shutdown_connections 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.361+0000 7fa2299d9740 1 -- 172.21.15.175:0/2089379369 wait complete. 2024-07-29T00:38:07.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.362+0000 7fa2299d9740 20 client.25628 trim_cache size 0 max 16384 2024-07-29T00:38:07.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.380+0000 7fa2299d9740 1 Processor -- start 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.380+0000 7fa2299d9740 1 -- start start 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa2299d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201af8a0 con 0x558d186b4de0 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa2299d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b13e0 con 0x558d186a9c00 2024-07-29T00:38:07.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa2299d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201aff70 con 0x558d186aa520 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa21f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa225000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa21f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:50000/0 (socket says 172.21.15.175:50000) 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa21f400640 1 -- 172.21.15.175:0/3795179747 learned_addr learned my addr 172.21.15.175:0/3795179747 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa21f400640 1 -- 172.21.15.175:0/3795179747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 msgr2=0x558d186ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.381+0000 7fa21f400640 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa21f400640 1 -- 172.21.15.175:0/3795179747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 msgr2=0x558d186ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa21f400640 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa21f400640 1 -- 172.21.15.175:0/3795179747 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d186b7890 con 0x558d186aa520 2024-07-29T00:38:07.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa21fe00640 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:07.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa21f400640 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186cc8b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa20c004650 tx=0x7fa20c0049d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72049da2d201b751 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa214c00640 1 -- 172.21.15.175:0/3795179747 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c035040 con 0x558d186aa520 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa214c00640 1 -- 172.21.15.175:0/3795179747 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20c031c90 con 0x558d186aa520 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.382+0000 7fa214c00640 1 -- 172.21.15.175:0/3795179747 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c03b430 con 0x558d186aa520 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 -- 172.21.15.175:0/3795179747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 msgr2=0x558d186cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186cc8b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa20c004650 tx=0x7fa20c0049d0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 -- 172.21.15.175:0/3795179747 shutdown_connections 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186b4de0 0x558d186ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186aa520 0x558d186cc8b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 --2- 172.21.15.175:0/3795179747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186a9c00 0x558d186ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 -- 172.21.15.175:0/3795179747 >> 172.21.15.175:0/3795179747 conn(0x558d186a6190 msgr2=0x558d186b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:07.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 -- 172.21.15.175:0/3795179747 shutdown_connections 2024-07-29T00:38:07.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.383+0000 7fa2299d9740 1 -- 172.21.15.175:0/3795179747 wait complete. 2024-07-29T00:38:07.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.384+0000 7fa2299d9740 1 -- start start 2024-07-29T00:38:07.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.384+0000 7fa2299d9740 1 Processor -- start 2024-07-29T00:38:07.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.384+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186a9c00 0x558d186b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.384+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 0x558d186b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.384+0000 7fa2299d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186b4de0 0x558d1872e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa2299d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b0d10 con 0x558d186a9c00 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa2299d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b0640 con 0x558d186aa520 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa2299d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ad880 con 0x558d186b4de0 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa2299d9740 10 client.0 _ll_register_callbacks cb 0x7fa21000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 0x558d186b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186a9c00 0x558d186b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa225000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186b4de0 0x558d1872e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 0x558d186b93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:53458/0 (socket says 172.21.15.175:53458) 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 -- 172.21.15.175:0/2691191558 learned_addr learned my addr 172.21.15.175:0/2691191558 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 -- 172.21.15.175:0/2691191558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186b4de0 msgr2=0x558d1872e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186b4de0 0x558d1872e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 -- 172.21.15.175:0/2691191558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186a9c00 msgr2=0x558d186b8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:07.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186a9c00 0x558d186b8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.385+0000 7fa21f400640 1 -- 172.21.15.175:0/2691191558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d186b9cb0 con 0x558d186aa520 2024-07-29T00:38:07.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa21f400640 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 0x558d186b93b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa20c004650 tx=0x7fa20c032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b235dee1161f66 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:07.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa20a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:07.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa20a000640 1 -- 172.21.15.175:0/2691191558 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c035070 con 0x558d186aa520 2024-07-29T00:38:07.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa20a000640 1 -- 172.21.15.175:0/2691191558 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa20c003b40 con 0x558d186aa520 2024-07-29T00:38:07.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa20c031c90 con 0x558d186aa520 2024-07-29T00:38:07.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa20a000640 1 -- 172.21.15.175:0/2691191558 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa20c066050 con 0x558d186aa520 2024-07-29T00:38:07.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa2299d9740 10 client.25634 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:07.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.386+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d1872ff30 con 0x558d186aa520 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa20a000640 1 -- 172.21.15.175:0/2691191558 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20c031c90 con 0x558d186aa520 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa20a000640 1 -- 172.21.15.175:0/2691191558 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa20c061840 con 0x558d186aa520 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa2299d9740 10 client.25634 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa2299d9740 10 client.25634 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa2299d9740 20 client.25634 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d18730170 0x558d18732560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa2299d9740 4 client.25634 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b5690169", "release": true, "await": true}] 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.387+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b5690169", "release": true, "await": true}) -- 0x558d18617740 con 0x558d18730170 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.388+0000 7fa21fe00640 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d18730170 0x558d18732560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.388+0000 7fa20a000640 10 client.25634 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.388+0000 7fa21fe00640 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d18730170 0x558d18732560 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a52f7e5510f010 server_cookie=eb3bb9898f38c1ac in_seq=0 out_seq=0 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.390+0000 7fa20a000640 1 -- 172.21.15.175:0/2691191558 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558d18617740 con 0x558d18730170 2024-07-29T00:38:07.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.390+0000 7fa20a000640 10 client.25634 handle_command_reply: tid=0 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.390+0000 7fa2299d9740 1 client.25634 shutdown 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 msgr2=0x558d186b93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 0x558d186b93b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa20c004650 tx=0x7fa20c032ed0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 shutdown_connections 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558d18730170 0x558d18732560 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d186b4de0 0x558d1872e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d186aa520 0x558d186b93b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 --2- 172.21.15.175:0/2691191558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558d186a9c00 0x558d186b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 >> 172.21.15.175:0/2691191558 conn(0x558d186a6190 msgr2=0x558d186b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa20a000640 0 client.25634 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 shutdown_connections 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 1 -- 172.21.15.175:0/2691191558 wait complete. 2024-07-29T00:38:07.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.391+0000 7fa2299d9740 20 client.25634 trim_cache size 0 max 16384 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 29, 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 4, 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "b5690169": { 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 4, 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:38:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 149.7, 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa2040a2cf0 msgr2=0x7fa2040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa2040a2cf0 0x7fa2040a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa218005f40 tx=0x7fa2180096a0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 msgr2=0x7fa22019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa22019a3d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e9f0 tx=0x7fa21000edf0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 shutdown_connections 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa220109a80 0x7fa22019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa2040a2cf0 0x7fa2040a51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa220104420 0x7fa22019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 --2- 172.21.15.175:0/457714161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa220103a80 0x7fa22019a3d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 >> 172.21.15.175:0/457714161 conn(0x7fa2200ff780 msgr2=0x7fa220100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 shutdown_connections 2024-07-29T00:38:07.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:07.392+0000 7fa227000640 1 -- 172.21.15.175:0/457714161 wait complete. 2024-07-29T00:38:07.616 INFO:tasks.workunit.client.0.smithi175.stderr:Note: switching to 'ba82afec3628bf830c0830558bf3d3c3aea66701'. 2024-07-29T00:38:07.616 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.616 INFO:tasks.workunit.client.0.smithi175.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-29T00:38:07.616 INFO:tasks.workunit.client.0.smithi175.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-29T00:38:07.616 INFO:tasks.workunit.client.0.smithi175.stderr:state without impacting any branches by switching back to a branch. 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: git switch -c 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr:Or undo this operation with: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: git switch - 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-07-29T00:38:07.617 INFO:tasks.workunit.client.0.smithi175.stderr:HEAD is now at ba82afec362 Merge pull request #58892 from zdover23/wip-doc-2024-07-28-backport-58884-to-squid 2024-07-29T00:38:07.628 DEBUG:teuthology.orchestra.run.smithi175:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-29T00:38:07.720 INFO:tasks.workunit.client.0.smithi175.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-29T00:38:07.725 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-29T00:38:07.725 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-29T00:38:07.738 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:07 smithi175 ceph-mon[32138]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:38:07.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:07 smithi204 ceph-mon[37368]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:38:07.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:38:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:07 smithi181 ceph-mon[36706]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-07-29T00:38:08.543 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b5690169', total seconds elapsed: 38.1 2024-07-29T00:38:08.702 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:09.185 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-29T00:38:09.393 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-29T00:38:09.471 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-29T00:38:09.476 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-29T00:38:09.476 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-29T00:38:09.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 -- 172.21.15.175:0/1620762073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d8102a30 msgr2=0x7fb8d8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 --2- 172.21.15.175:0/1620762073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d8102a30 0x7fb8d8102e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009900 tx=0x7fb8cc02a480 comp rx=0 tx=0).stop 2024-07-29T00:38:09.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 -- 172.21.15.175:0/1620762073 shutdown_connections 2024-07-29T00:38:09.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 --2- 172.21.15.175:0/1620762073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d810be60 0x7fb8d810e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 --2- 172.21.15.175:0/1620762073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8103350 0x7fb8d810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 --2- 172.21.15.175:0/1620762073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d8102a30 0x7fb8d8102e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.479+0000 7fb8df400640 1 -- 172.21.15.175:0/1620762073 >> 172.21.15.175:0/1620762073 conn(0x7fb8d80fc490 msgr2=0x7fb8d80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.480+0000 7fb8df400640 1 -- 172.21.15.175:0/1620762073 shutdown_connections 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.480+0000 7fb8df400640 1 -- 172.21.15.175:0/1620762073 wait complete. 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 Processor -- start 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 -- start start 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8102a30 0x7fb8d8100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 0x7fb8d8100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d810be60 0x7fb8d8101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8111c60 con 0x7fb8d810be60 2024-07-29T00:38:09.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8111ae0 con 0x7fb8d8103350 2024-07-29T00:38:09.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.481+0000 7fb8df400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8111de0 con 0x7fb8d8102a30 2024-07-29T00:38:09.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 0x7fb8d8100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d810be60 0x7fb8d8101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 0x7fb8d8100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47532/0 (socket says 172.21.15.175:47532) 2024-07-29T00:38:09.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8dca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8102a30 0x7fb8d8100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2941983091 learned_addr learned my addr 172.21.15.175:0/2941983091 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:09.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2941983091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8102a30 msgr2=0x7fb8d8100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8102a30 0x7fb8d8100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d810be60 msgr2=0x7fb8d8101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d810be60 0x7fb8d8101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2941983091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d81a65b0 con 0x7fb8d8103350 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.482+0000 7fb8dd400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d810be60 0x7fb8d8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.483+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 0x7fb8d8100bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400e9f0 tx=0x7fb8c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.483+0000 7fb8d5600640 1 -- 172.21.15.175:0/2941983091 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400ccc0 con 0x7fb8d8103350 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.483+0000 7fb8d5600640 1 -- 172.21.15.175:0/2941983091 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4010910 con 0x7fb8d8103350 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.483+0000 7fb8d5600640 1 -- 172.21.15.175:0/2941983091 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4018920 con 0x7fb8d8103350 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.483+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d81a68a0 con 0x7fb8d8103350 2024-07-29T00:38:09.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.483+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81a6d90 con 0x7fb8d8103350 2024-07-29T00:38:09.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.487+0000 7fb8e1e15740 1 Processor -- start 2024-07-29T00:38:09.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 -- start start 2024-07-29T00:38:09.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8d5600640 1 -- 172.21.15.175:0/2941983091 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb8c4010430 con 0x7fb8d8103350 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da8cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55929d9f1ad0 con 0x55929da8c930 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55929d989af0 con 0x55929da8d250 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55929d9eda90 con 0x55929da97b10 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.488+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da8cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da8cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60802/0 (socket says 172.21.15.175:60802) 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 -- 172.21.15.175:0/2470397573 learned_addr learned my addr 172.21.15.175:0/2470397573 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dca00640 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 -- 172.21.15.175:0/2470397573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 msgr2=0x55929da99f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da99f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 -- 172.21.15.175:0/2470397573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 msgr2=0x55929da975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 -- 172.21.15.175:0/2470397573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55929d7633c0 con 0x55929da8c930 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dca00640 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.489+0000 7fb8dd400640 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da8cd10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0011790 tx=0x7fb8d0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573c28187fe980a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.490+0000 7fb8c9c00640 1 -- 172.21.15.175:0/2470397573 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003c30 con 0x55929da8c930 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.490+0000 7fb8c9c00640 1 -- 172.21.15.175:0/2470397573 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d000fe80 con 0x55929da8c930 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.490+0000 7fb8c9c00640 1 -- 172.21.15.175:0/2470397573 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00193e0 con 0x55929da8c930 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.490+0000 7fb8d5600640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8c00a2cf0 0x7fb8c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.490+0000 7fb8d5600640 1 -- 172.21.15.175:0/2941983091 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb8c40cf740 con 0x7fb8d8103350 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.490+0000 7fb8dca00640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8c00a2cf0 0x7fb8c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2470397573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 msgr2=0x55929da8cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da8cd10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0011790 tx=0x7fb8d0011c50 comp rx=0 tx=0).stop 2024-07-29T00:38:09.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2470397573 shutdown_connections 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2470397573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da8cd10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2470397573 >> 172.21.15.175:0/2470397573 conn(0x55929da88ec0 msgr2=0x55929da892a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2470397573 shutdown_connections 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2470397573 wait complete. 2024-07-29T00:38:09.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.491+0000 7fb8dca00640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8c00a2cf0 0x7fb8c00a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009230 tx=0x7fb8cc0058d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:09.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.492+0000 7fb8e1e15740 1 -- start start 2024-07-29T00:38:09.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.492+0000 7fb8e1e15740 1 Processor -- start 2024-07-29T00:38:09.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.492+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da94940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.492+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da94e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55929da9a6e0 con 0x55929da8c930 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55929d9acbc0 con 0x55929da8d250 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55929da9a860 con 0x55929da97b10 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8dca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da94940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8dca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da94940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60810/0 (socket says 172.21.15.175:60810) 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8dca00640 1 -- 172.21.15.175:0/2306551694 learned_addr learned my addr 172.21.15.175:0/2306551694 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8e1e15740 10 client.0 _ll_register_callbacks cb 0x55929d998d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da94e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8dd400640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2306551694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 msgr2=0x55929da953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2306551694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 msgr2=0x55929da94940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da94940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.493+0000 7fb8d7e00640 1 -- 172.21.15.175:0/2306551694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929daa3b50 con 0x55929da8d250 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da94e80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40cce10 tx=0x7fb8c4097ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10760c25aa5705e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8be000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8be000640 1 -- 172.21.15.175:0/2306551694 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c409bc90 con 0x55929da8d250 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8be000640 1 -- 172.21.15.175:0/2306551694 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb8c4010ac0 con 0x55929da8d250 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55929d9ff040 con 0x55929da8d250 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8be000640 1 -- 172.21.15.175:0/2306551694 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8c409ff90 con 0x55929da8d250 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8e1e15740 10 client.25646 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55929daa64b0 con 0x55929da8d250 2024-07-29T00:38:09.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.494+0000 7fb8be000640 1 -- 172.21.15.175:0/2306551694 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55929d9ff040 con 0x55929da8d250 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8be000640 1 -- 172.21.15.175:0/2306551694 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8c40cfb30 con 0x55929da8d250 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8e1e15740 10 client.25646 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8e1e15740 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8e1e15740 20 client.25646 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929daa7970 0x55929daa9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8e1e15740 4 client.25646 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55929d9f9550 con 0x55929daa7970 2024-07-29T00:38:09.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.495+0000 7fb8dca00640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929daa7970 0x55929daa9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.496+0000 7fb8dca00640 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929daa7970 0x55929daa9d60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89bd07990b89d60e server_cookie=fd43b6665bed030d in_seq=0 out_seq=0 2024-07-29T00:38:09.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.496+0000 7fb8be000640 10 client.25646 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:09.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.496+0000 7fb8be000640 1 -- 172.21.15.175:0/2306551694 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55929d9f9550 con 0x55929daa7970 2024-07-29T00:38:09.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.496+0000 7fb8be000640 10 client.25646 handle_command_reply: tid=0 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 client.25646 shutdown 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 msgr2=0x55929da94e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da94e80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40cce10 tx=0x7fb8c4097ff0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 shutdown_connections 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929daa7970 0x55929daa9d60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da97b10 0x55929da953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da94e80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/2306551694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da8c930 0x55929da94940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8be000640 0 client.25646 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 >> 172.21.15.175:0/2306551694 conn(0x55929da88ec0 msgr2=0x55929da8ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 shutdown_connections 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 1 -- 172.21.15.175:0/2306551694 wait complete. 2024-07-29T00:38:09.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.497+0000 7fb8e1e15740 20 client.25646 trim_cache size 0 max 16384 2024-07-29T00:38:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 Processor -- start 2024-07-29T00:38:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 -- start start 2024-07-29T00:38:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8c930 0x55929da8f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da95210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929daaf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81af8a0 con 0x55929da97b10 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81b13e0 con 0x55929da8d250 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81aff70 con 0x55929da8c930 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8dca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8c930 0x55929da8f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929daaf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8dca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8c930 0x55929da8f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46314/0 (socket says 172.21.15.175:46314) 2024-07-29T00:38:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.516+0000 7fb8dca00640 1 -- 172.21.15.175:0/3468470496 learned_addr learned my addr 172.21.15.175:0/3468470496 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da95210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8dd400640 1 -- 172.21.15.175:0/3468470496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8c930 msgr2=0x55929da8f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8dd400640 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8c930 0x55929da8f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8dd400640 1 -- 172.21.15.175:0/3468470496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 msgr2=0x55929da95210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8dd400640 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da95210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8dd400640 1 -- 172.21.15.175:0/3468470496 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55929daaa0d0 con 0x55929da97b10 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.517+0000 7fb8dd400640 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929daaf630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000a010 tx=0x7fb8d0005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad05bc26f2f316f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.518+0000 7fb8c9200640 1 -- 172.21.15.175:0/3468470496 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d001f040 con 0x55929da97b10 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.518+0000 7fb8c9200640 1 -- 172.21.15.175:0/3468470496 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d002a070 con 0x55929da97b10 2024-07-29T00:38:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.518+0000 7fb8c9200640 1 -- 172.21.15.175:0/3468470496 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0005340 con 0x55929da97b10 2024-07-29T00:38:09.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3468470496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 msgr2=0x55929daaf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929daaf630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000a010 tx=0x7fb8d0005c40 comp rx=0 tx=0).stop 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3468470496 shutdown_connections 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929daaf630 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8d250 0x55929da95210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3468470496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8c930 0x55929da8f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3468470496 >> 172.21.15.175:0/3468470496 conn(0x55929da88ec0 msgr2=0x55929da969c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3468470496 shutdown_connections 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3468470496 wait complete. 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 -- start start 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.519+0000 7fb8e1e15740 1 Processor -- start 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8c930 0x55929da9fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8d250 0x55929daa02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929da91380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81b0d10 con 0x55929da97b10 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81b0640 con 0x55929da8c930 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81ad880 con 0x55929da8d250 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8e1e15740 10 client.0 _ll_register_callbacks cb 0x7fb8c400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929da91380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929da91380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60828/0 (socket says 172.21.15.175:60828) 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8dd400640 1 -- 172.21.15.175:0/3455739562 learned_addr learned my addr 172.21.15.175:0/3455739562 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8d250 0x55929daa02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.520+0000 7fb8dca00640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8c930 0x55929da9fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8d7e00640 1 -- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8c930 msgr2=0x55929da9fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8c930 0x55929da9fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8d7e00640 1 -- 172.21.15.175:0/3455739562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 msgr2=0x55929da91380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929da91380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8d7e00640 1 -- 172.21.15.175:0/3455739562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929da91bb0 con 0x55929da8d250 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8dd400640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929da91380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8d7e00640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8d250 0x55929daa02d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c6a0 tx=0x7fb8c40ccef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d260be43d15733f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8be000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8be000640 1 -- 172.21.15.175:0/3455739562 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4010720 con 0x55929da8d250 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8be000640 1 -- 172.21.15.175:0/3455739562 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fb8c409b360 con 0x55929da8d250 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.521+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d002a070 con 0x55929da8d250 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8be000640 1 -- 172.21.15.175:0/3455739562 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8c40a4850 con 0x55929da8d250 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 10 client.25648 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55929db12230 con 0x55929da8d250 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8be000640 1 -- 172.21.15.175:0/3455739562 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d002a070 con 0x55929da8d250 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8be000640 1 -- 172.21.15.175:0/3455739562 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8c40ba070 con 0x55929da8d250 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 10 client.25648 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 10 client.25648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 20 client.25648 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929db127c0 0x55929db14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 4 client.25648 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.522+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55929d9760c0 con 0x55929db127c0 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8dca00640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929db127c0 0x55929db14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8dca00640 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929db127c0 0x55929db14bb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15871e86ddae3fc2 server_cookie=93886728f23a2fba in_seq=0 out_seq=0 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8be000640 10 client.25648 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8be000640 1 -- 172.21.15.175:0/3455739562 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55929d9760c0 con 0x55929db127c0 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8be000640 10 client.25648 handle_command_reply: tid=0 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8e1e15740 1 client.25648 shutdown 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8d250 msgr2=0x55929daa02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.523+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8d250 0x55929daa02d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c6a0 tx=0x7fb8c40ccef0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 shutdown_connections 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55929db127c0 0x55929db14bb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55929da97b10 0x55929da91380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55929da8d250 0x55929daa02d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 --2- 172.21.15.175:0/3455739562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929da8c930 0x55929da9fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8be000640 0 client.25648 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 >> 172.21.15.175:0/3455739562 conn(0x55929da88ec0 msgr2=0x55929daa2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 shutdown_connections 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 1 -- 172.21.15.175:0/3455739562 wait complete. 2024-07-29T00:38:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.524+0000 7fb8e1e15740 20 client.25648 trim_cache size 0 max 16384 2024-07-29T00:38:09.528 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:38:09.528 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:38:09.528 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:38:09.528 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:38:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8c00a2cf0 msgr2=0x7fb8c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8c00a2cf0 0x7fb8c00a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009230 tx=0x7fb8cc0058d0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 msgr2=0x7fb8d8100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 0x7fb8d8100bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400e9f0 tx=0x7fb8c400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 shutdown_connections 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8c00a2cf0 0x7fb8c00a51b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d810be60 0x7fb8d8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d8103350 0x7fb8d8100bd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 --2- 172.21.15.175:0/2941983091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8102a30 0x7fb8d8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 >> 172.21.15.175:0/2941983091 conn(0x7fb8d80fc490 msgr2=0x7fb8d80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 shutdown_connections 2024-07-29T00:38:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:09.525+0000 7fb8df400640 1 -- 172.21.15.175:0/2941983091 wait complete. 2024-07-29T00:38:09.551 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-29T00:38:09.559 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:38:09.559 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-29T00:38:09.599 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-07-29T00:38:09.600 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-07-29T00:38:09.600 DEBUG:teuthology.orchestra.run.smithi175:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba82afec3628bf830c0830558bf3d3c3aea66701 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-07-29T00:38:09.698 INFO:tasks.workunit.client.0.smithi175.stderr:+ dbench 1 2024-07-29T00:38:09.736 INFO:tasks.workunit.client.0.smithi175.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-07-29T00:38:09.736 INFO:tasks.workunit.client.0.smithi175.stdout: 2024-07-29T00:38:09.736 INFO:tasks.workunit.client.0.smithi175.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-07-29T00:38:09.745 INFO:tasks.workunit.client.0.smithi175.stdout:0 of 1 processes prepared for launch 0 sec 2024-07-29T00:38:09.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:09 smithi175 ceph-mon[32138]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:09 smithi204 ceph-mon[37368]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:09.845 INFO:tasks.workunit.client.0.smithi175.stdout:1 of 1 processes prepared for launch 0 sec 2024-07-29T00:38:09.845 INFO:tasks.workunit.client.0.smithi175.stdout:releasing clients 2024-07-29T00:38:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:09 smithi181 ceph-mon[36706]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:10.550 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:38:10.550 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:38:10.845 INFO:tasks.workunit.client.0.smithi175.stdout: 1 1886 26.37 MB/sec warmup 1 sec latency 16.417 ms 2024-07-29T00:38:11.017 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:11.721 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:11 smithi175 ceph-mon[32138]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:11.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 -- 172.21.15.175:0/3638379668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 msgr2=0x7f6ac8110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:11.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 --2- 172.21.15.175:0/3638379668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac8110510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009900 tx=0x7f6ab802a480 comp rx=0 tx=0).stop 2024-07-29T00:38:11.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 -- 172.21.15.175:0/3638379668 shutdown_connections 2024-07-29T00:38:11.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 --2- 172.21.15.175:0/3638379668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac8110510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:11.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 --2- 172.21.15.175:0/3638379668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac8074bd0 0x7f6ac810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:11.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 --2- 172.21.15.175:0/3638379668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac80742b0 0x7f6ac8074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:11.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 -- 172.21.15.175:0/3638379668 >> 172.21.15.175:0/3638379668 conn(0x7f6ac80fd5a0 msgr2=0x7f6ac80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:11.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 -- 172.21.15.175:0/3638379668 shutdown_connections 2024-07-29T00:38:11.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.746+0000 7f6ace400640 1 -- 172.21.15.175:0/3638379668 wait complete. 2024-07-29T00:38:11.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.747+0000 7f6ace400640 1 Processor -- start 2024-07-29T00:38:11.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 -- start start 2024-07-29T00:38:11.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac80742b0 0x7f6ac810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:11.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac8074bd0 0x7f6ac810aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac810b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8113e80 con 0x7f6ac8074bd0 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8113d00 con 0x7f6ac810e120 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ace400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8114000 con 0x7f6ac80742b0 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ac6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac8074bd0 0x7f6ac810aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac810b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ac7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac80742b0 0x7f6ac810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.748+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac810b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47602/0 (socket says 172.21.15.175:47602) 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.749+0000 7f6ac7e00640 1 -- 172.21.15.175:0/2614159495 learned_addr learned my addr 172.21.15.175:0/2614159495 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.749+0000 7f6ac7e00640 1 -- 172.21.15.175:0/2614159495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac80742b0 msgr2=0x7f6ac810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:11.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.749+0000 7f6ac7e00640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac80742b0 0x7f6ac810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:11.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.749+0000 7f6ac7e00640 1 -- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac8074bd0 msgr2=0x7f6ac810aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:11.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.749+0000 7f6ac7e00640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac8074bd0 0x7f6ac810aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:11.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.749+0000 7f6ac7e00640 1 -- 172.21.15.175:0/2614159495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac810b8c0 con 0x7f6ac810e120 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.750+0000 7f6ac7e00640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac810b030 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8004770 tx=0x7f6ab8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.750+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab803f070 con 0x7f6ac810e120 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.750+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab8031c90 con 0x7f6ac810e120 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.750+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac81aac20 con 0x7f6ac810e120 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.750+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8035030 con 0x7f6ac810e120 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.750+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac81ab1b0 con 0x7f6ac810e120 2024-07-29T00:38:11.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.751+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac8075960 con 0x7f6ac810e120 2024-07-29T00:38:11.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.753+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6ab8042070 con 0x7f6ac810e120 2024-07-29T00:38:11.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.755+0000 7f6abfe00640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6aac0a2d70 0x7f6aac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:11.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.755+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6ab80ef9e0 con 0x7f6ac810e120 2024-07-29T00:38:11.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.755+0000 7f6ac7400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6aac0a2d70 0x7f6aac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:11.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.756+0000 7f6ac7400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6aac0a2d70 0x7f6aac0a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00099a0 tx=0x7f6ac0009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:11.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:11.758+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6ab80bbff0 con 0x7f6ac810e120 2024-07-29T00:38:11.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:11 smithi204 ceph-mon[37368]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:11.845 INFO:tasks.workunit.client.0.smithi175.stdout: 1 4220 20.09 MB/sec warmup 2 sec latency 16.716 ms 2024-07-29T00:38:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:11 smithi181 ceph-mon[36706]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-29T00:38:12.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.007+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ac802d5a0 con 0x7f6ac810e120 2024-07-29T00:38:12.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.008+0000 7f6abfe00640 1 -- 172.21.15.175:0/2614159495 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11994 (secure 0 0 0) 0x7f6ab80c02f0 con 0x7f6ac810e120 2024-07-29T00:38:12.009 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:38:12.009 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":29,"btime":"2024-07-29T00:36:31:605031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:36:30.665877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:38:12.009 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 29 2024-07-29T00:38:12.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6aac0a2d70 msgr2=0x7f6aac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:12.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6aac0a2d70 0x7f6aac0a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00099a0 tx=0x7f6ac0009500 comp rx=0 tx=0).stop 2024-07-29T00:38:12.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 msgr2=0x7f6ac810b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:12.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac810b030 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8004770 tx=0x7f6ab8003b90 comp rx=0 tx=0).stop 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 shutdown_connections 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac810e120 0x7f6ac810b030 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6aac0a2d70 0x7f6aac0a5230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac8074bd0 0x7f6ac810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 --2- 172.21.15.175:0/2614159495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac80742b0 0x7f6ac810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 >> 172.21.15.175:0/2614159495 conn(0x7f6ac80fd5a0 msgr2=0x7f6ac80fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:12.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.012+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 shutdown_connections 2024-07-29T00:38:12.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:12.013+0000 7f6ace400640 1 -- 172.21.15.175:0/2614159495 wait complete. 2024-07-29T00:38:12.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:12 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2614159495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:12.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:12 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2614159495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:12.846 INFO:tasks.workunit.client.0.smithi175.stdout: 1 6578 17.55 MB/sec warmup 3 sec latency 12.143 ms 2024-07-29T00:38:12.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:38:12.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:12 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2614159495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:13.424 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:13.846 INFO:tasks.workunit.client.0.smithi175.stdout: 1 8504 16.06 MB/sec warmup 4 sec latency 11.551 ms 2024-07-29T00:38:13.945 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:13 smithi175 ceph-mon[32138]: pgmap v54: 97 pgs: 97 active+clean; 2.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 132 KiB/s wr, 43 op/s 2024-07-29T00:38:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:13 smithi181 ceph-mon[36706]: pgmap v54: 97 pgs: 97 active+clean; 2.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 132 KiB/s wr, 43 op/s 2024-07-29T00:38:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:13 smithi204 ceph-mon[37368]: pgmap v54: 97 pgs: 97 active+clean; 2.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 132 KiB/s wr, 43 op/s 2024-07-29T00:38:14.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 -- 172.21.15.175:0/762750182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104109aa0 msgr2=0x7fe104109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 --2- 172.21.15.175:0/762750182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104109aa0 0x7fe104109e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009930 tx=0x7fe0f802a420 comp rx=0 tx=0).stop 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 -- 172.21.15.175:0/762750182 shutdown_connections 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 --2- 172.21.15.175:0/762750182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe104104440 0x7fe104110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 --2- 172.21.15.175:0/762750182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe104103aa0 0x7fe104103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 --2- 172.21.15.175:0/762750182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104109aa0 0x7fe104109e80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 -- 172.21.15.175:0/762750182 >> 172.21.15.175:0/762750182 conn(0x7fe1040ff740 msgr2=0x7fe104101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 -- 172.21.15.175:0/762750182 shutdown_connections 2024-07-29T00:38:14.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.089+0000 7fe10c400640 1 -- 172.21.15.175:0/762750182 wait complete. 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.090+0000 7fe10c400640 1 Processor -- start 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.090+0000 7fe10c400640 1 -- start start 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe104103aa0 0x7fe10419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 0x7fe10419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe104109aa0 0x7fe10419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe104113e60 con 0x7fe104103aa0 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe104113ce0 con 0x7fe104109aa0 2024-07-29T00:38:14.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe104113fe0 con 0x7fe104104440 2024-07-29T00:38:14.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe109000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 0x7fe10419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe109000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 0x7fe10419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46350/0 (socket says 172.21.15.175:46350) 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe109000640 1 -- 172.21.15.175:0/3201464981 learned_addr learned my addr 172.21.15.175:0/3201464981 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe10a400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe104109aa0 0x7fe10419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.091+0000 7fe109a00640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe104103aa0 0x7fe10419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe109000640 1 -- 172.21.15.175:0/3201464981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe104109aa0 msgr2=0x7fe10419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe109000640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe104109aa0 0x7fe10419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe109000640 1 -- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe104103aa0 msgr2=0x7fe10419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe109000640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe104103aa0 0x7fe10419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe109000640 1 -- 172.21.15.175:0/3201464981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10419f370 con 0x7fe104104440 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe109000640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 0x7fe10419a900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000ea60 tx=0x7fe0f000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe102000640 1 -- 172.21.15.175:0/3201464981 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000cde0 con 0x7fe104104440 2024-07-29T00:38:14.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe102000640 1 -- 172.21.15.175:0/3201464981 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f00035f0 con 0x7fe104104440 2024-07-29T00:38:14.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe102000640 1 -- 172.21.15.175:0/3201464981 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0010670 con 0x7fe104104440 2024-07-29T00:38:14.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.092+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10419f660 con 0x7fe104104440 2024-07-29T00:38:14.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.093+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1041a6f40 con 0x7fe104104440 2024-07-29T00:38:14.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe102000640 1 -- 172.21.15.175:0/3201464981 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe0f00037a0 con 0x7fe104104440 2024-07-29T00:38:14.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 Processor -- start 2024-07-29T00:38:14.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 -- start start 2024-07-29T00:38:14.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e1ef0 0x5599e50e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e0980 0x5599e50e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50edb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5599e5046ab0 con 0x5599e50e1ef0 2024-07-29T00:38:14.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5599e4fdea80 con 0x5599e50eb720 2024-07-29T00:38:14.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.096+0000 7fe10eff8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5599e5042a70 con 0x5599e50e0980 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e0980 0x5599e50e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50edb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47636/0 (socket says 172.21.15.175:47636) 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 -- 172.21.15.175:0/2131920502 learned_addr learned my addr 172.21.15.175:0/2131920502 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe10a400640 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e1ef0 0x5599e50e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 -- 172.21.15.175:0/2131920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e0980 msgr2=0x5599e50e0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e0980 0x5599e50e0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 -- 172.21.15.175:0/2131920502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e1ef0 msgr2=0x5599e50e0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e1ef0 0x5599e50e0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.097+0000 7fe109000640 1 -- 172.21.15.175:0/2131920502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e4db83c0 con 0x5599e50eb720 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe102000640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0ec0a2d70 0x7fe0ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe109000640 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50edb10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00cce20 tx=0x7fe0f00d5ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf98c8ec90fe2d3b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe102000640 1 -- 172.21.15.175:0/3201464981 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe0f00cf810 con 0x7fe104104440 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe0f6a00640 1 -- 172.21.15.175:0/2131920502 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0097f50 con 0x5599e50eb720 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe10a400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0ec0a2d70 0x7fe0ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe0f6a00640 1 -- 172.21.15.175:0/2131920502 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f009c3b0 con 0x5599e50eb720 2024-07-29T00:38:14.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.098+0000 7fe0f6a00640 1 -- 172.21.15.175:0/2131920502 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f009b610 con 0x5599e50eb720 2024-07-29T00:38:14.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 -- 172.21.15.175:0/2131920502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 msgr2=0x5599e50edb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50edb10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00cce20 tx=0x7fe0f00d5ef0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10a400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0ec0a2d70 0x7fe0ec0a5230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005fd0 tx=0x7fe0fc009600 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 -- 172.21.15.175:0/2131920502 shutdown_connections 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50edb10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e0980 0x5599e50e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2131920502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e1ef0 0x5599e50e0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 -- 172.21.15.175:0/2131920502 >> 172.21.15.175:0/2131920502 conn(0x5599e50a9950 msgr2=0x5599e50dd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 -- 172.21.15.175:0/2131920502 shutdown_connections 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.099+0000 7fe10eff8740 1 -- 172.21.15.175:0/2131920502 wait complete. 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.100+0000 7fe10eff8740 1 -- start start 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.100+0000 7fe10eff8740 1 Processor -- start 2024-07-29T00:38:14.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50e0980 0x5599e50e8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50eb720 0x5599e50e9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5599e50ee4c0 con 0x5599e50eb720 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5599e5001c10 con 0x5599e50e0980 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5599e50ee640 con 0x5599e50e1ef0 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10eff8740 10 client.0 _ll_register_callbacks cb 0x5599e4fcaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe109000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe109000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50e8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46362/0 (socket says 172.21.15.175:46362) 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe109000640 1 -- 172.21.15.175:0/3165019704 learned_addr learned my addr 172.21.15.175:0/3165019704 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:14.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe10a400640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50eb720 0x5599e50e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.101+0000 7fe109a00640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50e0980 0x5599e50e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe109000640 1 -- 172.21.15.175:0/3165019704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50e0980 msgr2=0x5599e50e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe109000640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50e0980 0x5599e50e8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe109000640 1 -- 172.21.15.175:0/3165019704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50eb720 msgr2=0x5599e50e9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe109000640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50eb720 0x5599e50e9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe109000640 1 -- 172.21.15.175:0/3165019704 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e50051e0 con 0x5599e50e1ef0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe109000640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50e8c40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00d7730 tx=0x7fe0f0003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8613e4dd6fdfd59a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe0eb400640 1 -- 172.21.15.175:0/3165019704 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f009b7f0 con 0x5599e50e1ef0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe0eb400640 1 -- 172.21.15.175:0/3165019704 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe0f0098320 con 0x5599e50e1ef0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f009c3b0 con 0x5599e50e1ef0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.102+0000 7fe0eb400640 1 -- 172.21.15.175:0/3165019704 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0f009ba80 con 0x5599e50e1ef0 2024-07-29T00:38:14.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 10 client.25666 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599e50ee4c0 con 0x5599e50e1ef0 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe0eb400640 1 -- 172.21.15.175:0/3165019704 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f009c3b0 con 0x5599e50e1ef0 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe0eb400640 1 -- 172.21.15.175:0/3165019704 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0f00cd160 con 0x5599e50e1ef0 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 10 client.25666 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 10 client.25666 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 20 client.25666 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e50fb460 0x5599e50fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 4 client.25666 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e5054020 con 0x5599e50fb460 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.103+0000 7fe109a00640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e50fb460 0x5599e50fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.104+0000 7fe109a00640 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e50fb460 0x5599e50fd850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30b73badd881ff91 server_cookie=208051840ecec0cb in_seq=0 out_seq=0 2024-07-29T00:38:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.104+0000 7fe0eb400640 10 client.25666 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:14.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe0eb400640 1 -- 172.21.15.175:0/3165019704 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5599e5054020 con 0x5599e50fb460 2024-07-29T00:38:14.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe0eb400640 10 client.25666 handle_command_reply: tid=0 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 client.25666 shutdown 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 msgr2=0x5599e50e8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50e8c40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00d7730 tx=0x7fe0f0003ea0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 shutdown_connections 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50eb720 0x5599e50e9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50e8c40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e50fb460 0x5599e50fd850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 --2- 172.21.15.175:0/3165019704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50e0980 0x5599e50e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 >> 172.21.15.175:0/3165019704 conn(0x5599e50a9950 msgr2=0x5599e50e2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe0eb400640 0 client.25666 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 shutdown_connections 2024-07-29T00:38:14.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.105+0000 7fe10eff8740 1 -- 172.21.15.175:0/3165019704 wait complete. 2024-07-29T00:38:14.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.106+0000 7fe10eff8740 20 client.25666 trim_cache size 0 max 16384 2024-07-29T00:38:14.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 Processor -- start 2024-07-29T00:38:14.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 -- start start 2024-07-29T00:38:14.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e5102e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e51033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041afa00 con 0x5599e50e0980 2024-07-29T00:38:14.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b1540 con 0x5599e50eb720 2024-07-29T00:38:14.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.121+0000 7fe10eff8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b00d0 con 0x5599e50e1ef0 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e51033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe10a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e51033c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46366/0 (socket says 172.21.15.175:46366) 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 -- 172.21.15.175:0/321491448 learned_addr learned my addr 172.21.15.175:0/321491448 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109a00640 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e5102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 -- 172.21.15.175:0/321491448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 msgr2=0x5599e50e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50e9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 -- 172.21.15.175:0/321491448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 msgr2=0x5599e5102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e5102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 -- 172.21.15.175:0/321491448 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e50e34d0 con 0x5599e50e1ef0 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe109000640 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e51033c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe0f009b9b0 tx=0x7fe0f00cced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c713f1ed40939233 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:14.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.122+0000 7fe0f6000640 1 -- 172.21.15.175:0/321491448 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f009b7f0 con 0x5599e50e1ef0 2024-07-29T00:38:14.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.123+0000 7fe0f6000640 1 -- 172.21.15.175:0/321491448 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f009c3b0 con 0x5599e50e1ef0 2024-07-29T00:38:14.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.123+0000 7fe0f6000640 1 -- 172.21.15.175:0/321491448 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f009b610 con 0x5599e50e1ef0 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.123+0000 7fe10eff8740 1 -- 172.21.15.175:0/321491448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 msgr2=0x5599e51033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.123+0000 7fe10eff8740 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e51033c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe0f009b9b0 tx=0x7fe0f00cced0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 -- 172.21.15.175:0/321491448 shutdown_connections 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e50e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e51033c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 --2- 172.21.15.175:0/321491448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e5102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 -- 172.21.15.175:0/321491448 >> 172.21.15.175:0/321491448 conn(0x5599e50a9950 msgr2=0x5599e50e8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 -- 172.21.15.175:0/321491448 shutdown_connections 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 -- 172.21.15.175:0/321491448 wait complete. 2024-07-29T00:38:14.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 -- start start 2024-07-29T00:38:14.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.124+0000 7fe10eff8740 1 Processor -- start 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e50ef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e5166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b0e70 con 0x5599e50e0980 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b07a0 con 0x5599e50eb720 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041ad9e0 con 0x5599e50e1ef0 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe10eff8740 10 client.0 _ll_register_callbacks cb 0x7fe0f009f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.125+0000 7fe109a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e50ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe109a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e50ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60918/0 (socket says 172.21.15.175:60918) 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe109a00640 1 -- 172.21.15.175:0/2895967326 learned_addr learned my addr 172.21.15.175:0/2895967326 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:14.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe109000640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e5166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 -- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 msgr2=0x5599e50ef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50ef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 -- 172.21.15.175:0/2895967326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 msgr2=0x5599e50ef060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e50ef060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 -- 172.21.15.175:0/2895967326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e5166cc0 con 0x5599e50eb720 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe109a00640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e50ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe109000640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe10a400640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e5166620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005cc0 tx=0x7fe0fc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3428d587f947ddc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.126+0000 7fe0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe0eb400640 1 -- 172.21.15.175:0/2895967326 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004b20 con 0x5599e50eb720 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f009c3b0 con 0x5599e50eb720 2024-07-29T00:38:14.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe0eb400640 1 -- 172.21.15.175:0/2895967326 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe0fc004cc0 con 0x5599e50eb720 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe0eb400640 1 -- 172.21.15.175:0/2895967326 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0fc003970 con 0x5599e50eb720 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe10eff8740 10 client.25676 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599e516e090 con 0x5599e50eb720 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.127+0000 7fe0eb400640 1 -- 172.21.15.175:0/2895967326 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f009c3b0 con 0x5599e50eb720 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe0eb400640 1 -- 172.21.15.175:0/2895967326 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0fc00f040 con 0x5599e50eb720 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe10eff8740 10 client.25676 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe10eff8740 10 client.25676 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe10eff8740 20 client.25676 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e516e860 0x5599e5170c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe10eff8740 4 client.25676 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:14.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599e504e530 con 0x5599e516e860 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe109a00640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e516e860 0x5599e5170c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe109a00640 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e516e860 0x5599e5170c80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1583a659962e3a17 server_cookie=822c5634d0f019be in_seq=0 out_seq=0 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.128+0000 7fe0eb400640 10 client.25676 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe0eb400640 1 -- 172.21.15.175:0/2895967326 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599e504e530 con 0x5599e516e860 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe0eb400640 10 client.25676 handle_command_reply: tid=0 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 client.25676 shutdown 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 msgr2=0x5599e5166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e5166620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005cc0 tx=0x7fe0fc004a40 comp rx=0 tx=0).stop 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 shutdown_connections 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5599e516e860 0x5599e5170c80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e50eb720 0x5599e5166620 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e50e1ef0 0x5599e50ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 --2- 172.21.15.175:0/2895967326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599e50e0980 0x5599e50ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 >> 172.21.15.175:0/2895967326 conn(0x5599e50a9950 msgr2=0x5599e50db950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe0eb400640 0 client.25676 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 shutdown_connections 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 1 -- 172.21.15.175:0/2895967326 wait complete. 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.129+0000 7fe10eff8740 20 client.25676 trim_cache size 0 max 16384 2024-07-29T00:38:14.133 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0ec0a2d70 msgr2=0x7fe0ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0ec0a2d70 0x7fe0ec0a5230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005fd0 tx=0x7fe0fc009600 comp rx=0 tx=0).stop 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 msgr2=0x7fe10419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 0x7fe10419a900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000ea60 tx=0x7fe0f000ef20 comp rx=0 tx=0).stop 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 shutdown_connections 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe104109aa0 0x7fe10419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe104104440 0x7fe10419a900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0ec0a2d70 0x7fe0ec0a5230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 --2- 172.21.15.175:0/3201464981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe104103aa0 0x7fe10419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:14.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.130+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 >> 172.21.15.175:0/3201464981 conn(0x7fe1040ff740 msgr2=0x7fe104100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:14.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.131+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 shutdown_connections 2024-07-29T00:38:14.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:14.131+0000 7fe10c400640 1 -- 172.21.15.175:0/3201464981 wait complete. 2024-07-29T00:38:14.846 INFO:tasks.workunit.client.0.smithi175.stdout: 1 10990 15.66 MB/sec warmup 5 sec latency 8.601 ms 2024-07-29T00:38:15.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:38:15.683 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:15.846 INFO:tasks.workunit.client.0.smithi175.stdout: 1 13095 14.24 MB/sec warmup 6 sec latency 47.803 ms 2024-07-29T00:38:15.876 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:15 smithi175 ceph-mon[32138]: pgmap v55: 97 pgs: 97 active+clean; 2.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 109 KiB/s wr, 23 op/s 2024-07-29T00:38:15.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:15 smithi181 ceph-mon[36706]: pgmap v55: 97 pgs: 97 active+clean; 2.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 109 KiB/s wr, 23 op/s 2024-07-29T00:38:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:15 smithi204 ceph-mon[37368]: pgmap v55: 97 pgs: 97 active+clean; 2.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 109 KiB/s wr, 23 op/s 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 -- 172.21.15.175:0/2149109787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 msgr2=0x7f02f0108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 --2- 172.21.15.175:0/2149109787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f0108520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009900 tx=0x7f02e002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 -- 172.21.15.175:0/2149109787 shutdown_connections 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 --2- 172.21.15.175:0/2149109787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02f010d220 0x7f02f0114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 --2- 172.21.15.175:0/2149109787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f0108af0 0x7f02f010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 --2- 172.21.15.175:0/2149109787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f0108520 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 -- 172.21.15.175:0/2149109787 >> 172.21.15.175:0/2149109787 conn(0x7f02f0077890 msgr2=0x7f02f0079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 -- 172.21.15.175:0/2149109787 shutdown_connections 2024-07-29T00:38:16.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.382+0000 7f02f5000640 1 -- 172.21.15.175:0/2149109787 wait complete. 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.383+0000 7f02f5000640 1 Processor -- start 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 -- start start 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02f0108af0 0x7f02f019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f010d220 0x7f02f01a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0118270 con 0x7f02f010d220 2024-07-29T00:38:16.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01180f0 con 0x7f02f0108af0 2024-07-29T00:38:16.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02f5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01183f0 con 0x7f02f0108140 2024-07-29T00:38:16.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02f0108af0 0x7f02f019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46386/0 (socket says 172.21.15.175:46386) 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02ede00640 1 -- 172.21.15.175:0/3183818479 learned_addr learned my addr 172.21.15.175:0/3183818479 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.384+0000 7f02ee800640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f010d220 0x7f02f01a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02ede00640 1 -- 172.21.15.175:0/3183818479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02f0108af0 msgr2=0x7f02f019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02ede00640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02f0108af0 0x7f02f019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02ede00640 1 -- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f010d220 msgr2=0x7f02f01a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02ede00640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f010d220 0x7f02f01a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02ede00640 1 -- 172.21.15.175:0/3183818479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f01a36a0 con 0x7f02f0108140 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02ede00640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f019e6f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02e0004770 tx=0x7f02e0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02e6a00640 1 -- 172.21.15.175:0/3183818479 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e003f070 con 0x7f02f0108140 2024-07-29T00:38:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f01a3930 con 0x7f02f0108140 2024-07-29T00:38:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02e6a00640 1 -- 172.21.15.175:0/3183818479 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e0031ca0 con 0x7f02f0108140 2024-07-29T00:38:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.385+0000 7f02e6a00640 1 -- 172.21.15.175:0/3183818479 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0035030 con 0x7f02f0108140 2024-07-29T00:38:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.386+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f01ab270 con 0x7f02f0108140 2024-07-29T00:38:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.389+0000 7f02e6a00640 1 -- 172.21.15.175:0/3183818479 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f02e0042070 con 0x7f02f0108140 2024-07-29T00:38:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.389+0000 7f02f7c66740 1 Processor -- start 2024-07-29T00:38:16.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 -- start start 2024-07-29T00:38:16.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee78ef0 0x56122ee77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee77dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee84b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56122edddab0 con 0x56122ee78ef0 2024-07-29T00:38:16.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56122ed75a80 con 0x56122ee82720 2024-07-29T00:38:16.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.390+0000 7f02f7c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56122edd9a70 con 0x56122ee77980 2024-07-29T00:38:16.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee77dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee84b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02ee800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee78ef0 0x56122ee77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee84b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47686/0 (socket says 172.21.15.175:47686) 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 -- 172.21.15.175:0/3193193823 learned_addr learned my addr 172.21.15.175:0/3193193823 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e6a00640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02d40a2cf0 0x7f02d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 -- 172.21.15.175:0/3193193823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 msgr2=0x56122ee77dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee77dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 -- 172.21.15.175:0/3193193823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee78ef0 msgr2=0x56122ee77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee78ef0 0x56122ee77440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e7e00640 1 -- 172.21.15.175:0/3193193823 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122eb4f3c0 con 0x56122ee82720 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.391+0000 7f02e6a00640 1 -- 172.21.15.175:0/3183818479 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f02e00ef7e0 con 0x7f02f0108140 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.392+0000 7f02ee800640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02d40a2cf0 0x7f02d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.392+0000 7f02e7e00640 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee84b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02d800bcb0 tx=0x7f02d80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b93232a438747b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.392+0000 7f02df400640 1 -- 172.21.15.175:0/3193193823 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8004030 con 0x56122ee82720 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.392+0000 7f02df400640 1 -- 172.21.15.175:0/3193193823 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d80041d0 con 0x56122ee82720 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.392+0000 7f02df400640 1 -- 172.21.15.175:0/3193193823 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d80117b0 con 0x56122ee82720 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 -- 172.21.15.175:0/3193193823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 msgr2=0x56122ee84b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02ee800640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02d40a2cf0 0x7f02d40a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f02e8005c40 tx=0x7f02e8004a40 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee84b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02d800bcb0 tx=0x7f02d80107a0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 -- 172.21.15.175:0/3193193823 shutdown_connections 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee84b10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee77dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 --2- 172.21.15.175:0/3193193823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee78ef0 0x56122ee77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 -- 172.21.15.175:0/3193193823 >> 172.21.15.175:0/3193193823 conn(0x56122ee40950 msgr2=0x56122ee74010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 -- 172.21.15.175:0/3193193823 shutdown_connections 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.393+0000 7f02f7c66740 1 -- 172.21.15.175:0/3193193823 wait complete. 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 -- start start 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 Processor -- start 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee7f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee7fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee80140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56122ee854c0 con 0x56122ee77980 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56122ed98c10 con 0x56122ee82720 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56122ee85640 con 0x56122ee78ef0 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.394+0000 7f02f7c66740 10 client.0 _ll_register_callbacks cb 0x56122ed61a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:16.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee7fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ee800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee80140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee7fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46408/0 (socket says 172.21.15.175:46408) 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02e7e00640 1 -- 172.21.15.175:0/268489294 learned_addr learned my addr 172.21.15.175:0/268489294 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ede00640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee7f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ee800640 1 -- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 msgr2=0x56122ee7fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ee800640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee7fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ee800640 1 -- 172.21.15.175:0/268489294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 msgr2=0x56122ee7f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ee800640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee7f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ee800640 1 -- 172.21.15.175:0/268489294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122ed9c1e0 con 0x56122ee82720 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.395+0000 7f02ede00640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.396+0000 7f02ee800640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee80140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009f60 tx=0x7f02e8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec7e7b1f15ce6ee server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:16.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.396+0000 7f02d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.396+0000 7f02d3e00640 1 -- 172.21.15.175:0/268489294 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8003970 con 0x56122ee82720 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.396+0000 7f02d3e00640 1 -- 172.21.15.175:0/268489294 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f02e8003b10 con 0x56122ee82720 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.396+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d80041d0 con 0x56122ee82720 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02d3e00640 1 -- 172.21.15.175:0/268489294 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02e8003d10 con 0x56122ee82720 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 10 client.25688 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56122ee912b0 con 0x56122ee82720 2024-07-29T00:38:16.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02d3e00640 1 -- 172.21.15.175:0/268489294 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d80041d0 con 0x56122ee82720 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02d3e00640 1 -- 172.21.15.175:0/268489294 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02e801d920 con 0x56122ee82720 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 10 client.25688 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 10 client.25688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 20 client.25688 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122ee926c0 0x56122ee94b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 4 client.25688 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.397+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122edeb020 con 0x56122ee926c0 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.398+0000 7f02ede00640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122ee926c0 0x56122ee94b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.398+0000 7f02ede00640 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122ee926c0 0x56122ee94b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c50eea2b18270c0d server_cookie=365cb20825e6495f in_seq=0 out_seq=0 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.398+0000 7f02d3e00640 10 client.25688 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02d3e00640 1 -- 172.21.15.175:0/268489294 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56122edeb020 con 0x56122ee926c0 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02d3e00640 10 client.25688 handle_command_reply: tid=0 2024-07-29T00:38:16.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 client.25688 shutdown 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 msgr2=0x56122ee80140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee80140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009f60 tx=0x7f02e8004ab0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 shutdown_connections 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122ee926c0 0x56122ee94b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee80140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee7fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 --2- 172.21.15.175:0/268489294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 >> 172.21.15.175:0/268489294 conn(0x56122ee40950 msgr2=0x56122ee79d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02d3e00640 0 client.25688 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 shutdown_connections 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.399+0000 7f02f7c66740 1 -- 172.21.15.175:0/268489294 wait complete. 2024-07-29T00:38:16.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.400+0000 7f02f7c66740 20 client.25688 trim_cache size 0 max 16384 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.416+0000 7f02f7c66740 1 Processor -- start 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.416+0000 7f02f7c66740 1 -- start start 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee6f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee9a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02f7c66740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b3c80 con 0x56122ee77980 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02f7c66740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b57c0 con 0x56122ee82720 2024-07-29T00:38:16.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02f7c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b4350 con 0x56122ee78ef0 2024-07-29T00:38:16.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee6f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02ee800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee9a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee6f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46416/0 (socket says 172.21.15.175:46416) 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.417+0000 7f02e7e00640 1 -- 172.21.15.175:0/2120804080 learned_addr learned my addr 172.21.15.175:0/2120804080 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02ee800640 1 -- 172.21.15.175:0/2120804080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 msgr2=0x56122ee6f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02ee800640 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee6f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02ee800640 1 -- 172.21.15.175:0/2120804080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 msgr2=0x56122ee741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02ee800640 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02ee800640 1 -- 172.21.15.175:0/2120804080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122ee7e6d0 con 0x56122ee82720 2024-07-29T00:38:16.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02ee800640 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee9a450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02e8014040 tx=0x7f02e8010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff95a9de7733ce1e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02dea00640 1 -- 172.21.15.175:0/2120804080 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8009480 con 0x56122ee82720 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02dea00640 1 -- 172.21.15.175:0/2120804080 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e8036070 con 0x56122ee82720 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.418+0000 7f02dea00640 1 -- 172.21.15.175:0/2120804080 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8004310 con 0x56122ee82720 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 -- 172.21.15.175:0/2120804080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 msgr2=0x56122ee9a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee9a450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02e8014040 tx=0x7f02e8010430 comp rx=0 tx=0).stop 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 -- 172.21.15.175:0/2120804080 shutdown_connections 2024-07-29T00:38:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee82720 0x56122ee9a450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee78ef0 0x56122ee6f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 --2- 172.21.15.175:0/2120804080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee77980 0x56122ee741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 -- 172.21.15.175:0/2120804080 >> 172.21.15.175:0/2120804080 conn(0x56122ee40950 msgr2=0x56122ee73ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 -- 172.21.15.175:0/2120804080 shutdown_connections 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.419+0000 7f02f7c66740 1 -- 172.21.15.175:0/2120804080 wait complete. 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 -- start start 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 Processor -- start 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee8a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 0x56122ee8ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee82720 0x56122ee7b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b50f0 con 0x56122ee82720 2024-07-29T00:38:16.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b4a20 con 0x56122ee78ef0 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b1c60 con 0x56122ee77980 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.420+0000 7f02f7c66740 10 client.0 _ll_register_callbacks cb 0x7f02e8009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 0x56122ee8ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee8a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 0x56122ee8ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47712/0 (socket says 172.21.15.175:47712) 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 -- 172.21.15.175:0/1963068614 learned_addr learned my addr 172.21.15.175:0/1963068614 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02ee800640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee82720 0x56122ee7b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 -- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 msgr2=0x56122ee8a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee8a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 -- 172.21.15.175:0/1963068614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee82720 msgr2=0x56122ee7b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee82720 0x56122ee7b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 -- 172.21.15.175:0/1963068614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122ee7d630 con 0x56122ee78ef0 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02ee800640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee82720 0x56122ee7b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02ede00640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee8a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:16.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02e7e00640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 0x56122ee8ae10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02d8003f10 tx=0x7f02d800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=113f62f1ebf4c34c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.421+0000 7f02d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02d3e00640 1 -- 172.21.15.175:0/1963068614 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8005450 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02d3e00640 1 -- 172.21.15.175:0/1963068614 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f02d80055f0 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e8036070 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02d3e00640 1 -- 172.21.15.175:0/1963068614 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02d80035f0 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02f7c66740 10 client.25694 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56122eefc900 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02d3e00640 1 -- 172.21.15.175:0/1963068614 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e8036070 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02d3e00640 1 -- 172.21.15.175:0/1963068614 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02d8011a40 con 0x56122ee78ef0 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02f7c66740 10 client.25694 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02f7c66740 10 client.25694 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:38:16.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.422+0000 7f02f7c66740 20 client.25694 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02f7c66740 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122eefceb0 0x56122eeff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02f7c66740 4 client.25694 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56122ede5530 con 0x56122eefceb0 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02ede00640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122eefceb0 0x56122eeff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02ede00640 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122eefceb0 0x56122eeff2d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dbead08cbaa51d3 server_cookie=69b37d13e6e51873 in_seq=0 out_seq=0 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02d3e00640 10 client.25694 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02d3e00640 1 -- 172.21.15.175:0/1963068614 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56122ede5530 con 0x56122eefceb0 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02d3e00640 10 client.25694 handle_command_reply: tid=0 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.423+0000 7f02f7c66740 1 client.25694 shutdown 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 msgr2=0x56122ee8ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 0x56122ee8ae10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02d8003f10 tx=0x7f02d800bfd0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 shutdown_connections 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56122eefceb0 0x56122eeff2d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56122ee82720 0x56122ee7b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56122ee78ef0 0x56122ee8ae10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 --2- 172.21.15.175:0/1963068614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56122ee77980 0x56122ee8a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 >> 172.21.15.175:0/1963068614 conn(0x56122ee40950 msgr2=0x56122ee7f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02d3e00640 0 client.25694 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 shutdown_connections 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 1 -- 172.21.15.175:0/1963068614 wait complete. 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.424+0000 7f02f7c66740 20 client.25694 trim_cache size 0 max 16384 2024-07-29T00:38:16.428 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:16.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02d40a2cf0 msgr2=0x7f02d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02d40a2cf0 0x7f02d40a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f02e8005c40 tx=0x7f02e8004a40 comp rx=0 tx=0).stop 2024-07-29T00:38:16.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 msgr2=0x7f02f019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:16.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f019e6f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02e0004770 tx=0x7f02e0003c10 comp rx=0 tx=0).stop 2024-07-29T00:38:16.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 shutdown_connections 2024-07-29T00:38:16.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f02d40a2cf0 0x7f02d40a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02f010d220 0x7f02f01a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02f0108af0 0x7f02f019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 --2- 172.21.15.175:0/3183818479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f0108140 0x7f02f019e6f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:16.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 >> 172.21.15.175:0/3183818479 conn(0x7f02f0077890 msgr2=0x7f02f0078b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:16.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 shutdown_connections 2024-07-29T00:38:16.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:16.425+0000 7f02f5000640 1 -- 172.21.15.175:0/3183818479 wait complete. 2024-07-29T00:38:16.846 INFO:tasks.workunit.client.0.smithi175.stdout: 1 14976 13.98 MB/sec warmup 7 sec latency 16.138 ms 2024-07-29T00:38:17.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:38:17.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:17 smithi175 ceph-mon[32138]: pgmap v56: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 679 KiB/s wr, 90 op/s 2024-07-29T00:38:17.846 INFO:tasks.workunit.client.0.smithi175.stdout: 1 17557 13.99 MB/sec warmup 8 sec latency 11.338 ms 2024-07-29T00:38:17.887 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:17 smithi204 ceph-mon[37368]: pgmap v56: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 679 KiB/s wr, 90 op/s 2024-07-29T00:38:17.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:17 smithi181 ceph-mon[36706]: pgmap v56: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 679 KiB/s wr, 90 op/s 2024-07-29T00:38:18.008 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 -- 172.21.15.175:0/1765825693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 msgr2=0x7f3420114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 --2- 172.21.15.175:0/1765825693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 0x7f3420114b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3418009ca0 tx=0x7f341802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 -- 172.21.15.175:0/1765825693 shutdown_connections 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 --2- 172.21.15.175:0/1765825693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 0x7f3420114b90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 --2- 172.21.15.175:0/1765825693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3420107d10 0x7f3420108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 --2- 172.21.15.175:0/1765825693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342010dd50 0x7f342010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 -- 172.21.15.175:0/1765825693 >> 172.21.15.175:0/1765825693 conn(0x7f3420077670 msgr2=0x7f3420079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 -- 172.21.15.175:0/1765825693 shutdown_connections 2024-07-29T00:38:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.675+0000 7f3426200640 1 -- 172.21.15.175:0/1765825693 wait complete. 2024-07-29T00:38:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 Processor -- start 2024-07-29T00:38:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 -- start start 2024-07-29T00:38:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 0x7f342019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 0x7f342019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342010dd50 0x7f34201a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34201180b0 con 0x7f342010dd50 2024-07-29T00:38:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3420117f30 con 0x7f3420108670 2024-07-29T00:38:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.677+0000 7f3426200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3420118230 con 0x7f3420107d10 2024-07-29T00:38:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 0x7f342019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f341fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 0x7f342019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 0x7f342019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46446/0 (socket says 172.21.15.175:46446) 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 -- 172.21.15.175:0/259970761 learned_addr learned my addr 172.21.15.175:0/259970761 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3425800640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342010dd50 0x7f34201a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 -- 172.21.15.175:0/259970761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 msgr2=0x7f342019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 0x7f342019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 -- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342010dd50 msgr2=0x7f34201a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342010dd50 0x7f34201a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.678+0000 7f3424e00640 1 -- 172.21.15.175:0/259970761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34201a34c0 con 0x7f3420107d10 2024-07-29T00:38:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.679+0000 7f3424e00640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 0x7f342019e510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f341400e9f0 tx=0x7f341400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.679+0000 7f341d600640 1 -- 172.21.15.175:0/259970761 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400cd80 con 0x7f3420107d10 2024-07-29T00:38:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.679+0000 7f341d600640 1 -- 172.21.15.175:0/259970761 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414010910 con 0x7f3420107d10 2024-07-29T00:38:18.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.679+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201a37b0 con 0x7f3420107d10 2024-07-29T00:38:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.679+0000 7f341d600640 1 -- 172.21.15.175:0/259970761 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414018990 con 0x7f3420107d10 2024-07-29T00:38:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.679+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201ab090 con 0x7f3420107d10 2024-07-29T00:38:18.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 Processor -- start 2024-07-29T00:38:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 -- start start 2024-07-29T00:38:18.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e4549c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e4549cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557e45402ab0 con 0x557e454a7910 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557e4539aa80 con 0x557e4549e0e0 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f342a46b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557e453fea70 con 0x557e4549cb70 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f3425800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e4549c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f3425800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e4549c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47738/0 (socket says 172.21.15.175:47738) 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.683+0000 7f3425800640 1 -- 172.21.15.175:0/4206846457 learned_addr learned my addr 172.21.15.175:0/4206846457 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f341fe00640 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f341d600640 1 -- 172.21.15.175:0/259970761 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3414010ac0 con 0x7f3420107d10 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f3425800640 1 -- 172.21.15.175:0/4206846457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 msgr2=0x557e4549cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f3425800640 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e4549cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f3425800640 1 -- 172.21.15.175:0/4206846457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 msgr2=0x557e454a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f3425800640 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454a9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f3425800640 1 -- 172.21.15.175:0/4206846457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e451743c0 con 0x557e4549e0e0 2024-07-29T00:38:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.684+0000 7f3425800640 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e4549c630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3418004af0 tx=0x7f3418009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7941713e89df7c30 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.685+0000 7f3412000640 1 -- 172.21.15.175:0/4206846457 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418031da0 con 0x557e4549e0e0 2024-07-29T00:38:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.685+0000 7f3412000640 1 -- 172.21.15.175:0/4206846457 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3418036040 con 0x557e4549e0e0 2024-07-29T00:38:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.685+0000 7f3412000640 1 -- 172.21.15.175:0/4206846457 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418004330 con 0x557e4549e0e0 2024-07-29T00:38:18.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 -- 172.21.15.175:0/4206846457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 msgr2=0x557e4549c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e4549c630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3418004af0 tx=0x7f3418009800 comp rx=0 tx=0).stop 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 -- 172.21.15.175:0/4206846457 shutdown_connections 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e4549cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 --2- 172.21.15.175:0/4206846457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e4549c630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 -- 172.21.15.175:0/4206846457 >> 172.21.15.175:0/4206846457 conn(0x557e45465950 msgr2=0x557e45499200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 -- 172.21.15.175:0/4206846457 shutdown_connections 2024-07-29T00:38:18.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.686+0000 7f342a46b740 1 -- 172.21.15.175:0/4206846457 wait complete. 2024-07-29T00:38:18.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.687+0000 7f341d600640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f34080a2d00 0x7f34080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.687+0000 7f342a46b740 1 -- start start 2024-07-29T00:38:18.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.687+0000 7f341d600640 1 -- 172.21.15.175:0/259970761 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f34140cf760 con 0x7f3420107d10 2024-07-29T00:38:18.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.687+0000 7f342a46b740 1 Processor -- start 2024-07-29T00:38:18.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.687+0000 7f3425800640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f34080a2d00 0x7f34080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549cb70 0x557e454a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 0x557e454a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e454a7910 0x557e454a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557e454aa650 con 0x557e4549cb70 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557e453bdc10 con 0x557e454a7910 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557e454aa7d0 con 0x557e4549e0e0 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f342a46b740 10 client.0 _ll_register_callbacks cb 0x557e45386a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f341fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 0x557e454a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f341fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 0x557e454a4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46456/0 (socket says 172.21.15.175:46456) 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f341fe00640 1 -- 172.21.15.175:0/3982846115 learned_addr learned my addr 172.21.15.175:0/3982846115 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.688+0000 7f3424e00640 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549cb70 0x557e454a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f341fe00640 1 -- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e454a7910 msgr2=0x557e454a53e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f341fe00640 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e454a7910 0x557e454a53e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f341fe00640 1 -- 172.21.15.175:0/3982846115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549cb70 msgr2=0x557e454a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f341fe00640 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549cb70 0x557e454a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f341fe00640 1 -- 172.21.15.175:0/3982846115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e453c11e0 con 0x557e4549e0e0 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f3425800640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f34080a2d00 0x7f34080a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f34180318f0 tx=0x7f34180093f0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f341fe00640 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 0x557e454a4ea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f340c012700 tx=0x7f340c012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c558c8c57bf72b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f3406a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f3406a00640 1 -- 172.21.15.175:0/3982846115 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c004030 con 0x557e4549e0e0 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.689+0000 7f3406a00640 1 -- 172.21.15.175:0/3982846115 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f340c0041d0 con 0x557e4549e0e0 2024-07-29T00:38:18.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.690+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3418036040 con 0x557e4549e0e0 2024-07-29T00:38:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.690+0000 7f3406a00640 1 -- 172.21.15.175:0/3982846115 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f340c0043d0 con 0x557e4549e0e0 2024-07-29T00:38:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.690+0000 7f342a46b740 10 client.25699 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.690+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e454b64e0 con 0x557e4549e0e0 2024-07-29T00:38:18.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.690+0000 7f3406a00640 1 -- 172.21.15.175:0/3982846115 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3418036040 con 0x557e4549e0e0 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f3406a00640 1 -- 172.21.15.175:0/3982846115 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f340c00f880 con 0x557e4549e0e0 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f342a46b740 10 client.25699 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f342a46b740 10 client.25699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f342a46b740 20 client.25699 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f342a46b740 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e454b79f0 0x557e454b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f342a46b740 4 client.25699 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e45410020 con 0x557e454b79f0 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.691+0000 7f3424e00640 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e454b79f0 0x557e454b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.692+0000 7f3424e00640 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e454b79f0 0x557e454b9e30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0754a10a6d1302c server_cookie=9ae08fe0ae8c2473 in_seq=0 out_seq=0 2024-07-29T00:38:18.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.692+0000 7f3406a00640 10 client.25699 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f3406a00640 1 -- 172.21.15.175:0/3982846115 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557e45410020 con 0x557e454b79f0 2024-07-29T00:38:18.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f3406a00640 10 client.25699 handle_command_reply: tid=0 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 client.25699 shutdown 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 msgr2=0x557e454a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 0x557e454a4ea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f340c012700 tx=0x7f340c012b00 comp rx=0 tx=0).stop 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 shutdown_connections 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e454b79f0 0x557e454b9e30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e454a7910 0x557e454a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549e0e0 0x557e454a4ea0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 --2- 172.21.15.175:0/3982846115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549cb70 0x557e454a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f3406a00640 0 client.25699 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.693+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 >> 172.21.15.175:0/3982846115 conn(0x557e45465950 msgr2=0x557e4549ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.694+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 shutdown_connections 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.694+0000 7f342a46b740 1 -- 172.21.15.175:0/3982846115 wait complete. 2024-07-29T00:38:18.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.694+0000 7f342a46b740 20 client.25699 trim_cache size 0 max 16384 2024-07-29T00:38:18.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 Processor -- start 2024-07-29T00:38:18.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 -- start start 2024-07-29T00:38:18.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e454a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e454a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34201b3bb0 con 0x557e454a7910 2024-07-29T00:38:18.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34201b56f0 con 0x557e4549e0e0 2024-07-29T00:38:18.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.710+0000 7f342a46b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34201b4280 con 0x557e4549cb70 2024-07-29T00:38:18.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e454a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f341fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e454a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f341fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e454a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47750/0 (socket says 172.21.15.175:47750) 2024-07-29T00:38:18.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e454a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46468/0 (socket says 172.21.15.175:46468) 2024-07-29T00:38:18.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f341fe00640 1 -- 172.21.15.175:0/4151018861 learned_addr learned my addr 172.21.15.175:0/4151018861 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:18.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3425800640 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 -- 172.21.15.175:0/4151018861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 msgr2=0x557e454a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e454a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 -- 172.21.15.175:0/4151018861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 msgr2=0x557e454bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 -- 172.21.15.175:0/4151018861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e454ba4b0 con 0x557e4549cb70 2024-07-29T00:38:18.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f341fe00640 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e454a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:18.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3425800640 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:18.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.711+0000 7f3424e00640 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e454a5250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34140cce30 tx=0x7f3414018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd5d266f679afa29 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:18.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.712+0000 7f3411600640 1 -- 172.21.15.175:0/4151018861 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140a4870 con 0x557e4549cb70 2024-07-29T00:38:18.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.712+0000 7f3411600640 1 -- 172.21.15.175:0/4151018861 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414099f00 con 0x557e4549cb70 2024-07-29T00:38:18.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.712+0000 7f3411600640 1 -- 172.21.15.175:0/4151018861 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414018d90 con 0x557e4549cb70 2024-07-29T00:38:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 -- 172.21.15.175:0/4151018861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 msgr2=0x557e454a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e454a5250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34140cce30 tx=0x7f3414018420 comp rx=0 tx=0).stop 2024-07-29T00:38:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 -- 172.21.15.175:0/4151018861 shutdown_connections 2024-07-29T00:38:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e454a7910 0x557e454bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549e0e0 0x557e454a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 --2- 172.21.15.175:0/4151018861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e4549cb70 0x557e454a5250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 -- 172.21.15.175:0/4151018861 >> 172.21.15.175:0/4151018861 conn(0x557e45465950 msgr2=0x557e454a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:18.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 -- 172.21.15.175:0/4151018861 shutdown_connections 2024-07-29T00:38:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 -- 172.21.15.175:0/4151018861 wait complete. 2024-07-29T00:38:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.713+0000 7f342a46b740 1 -- start start 2024-07-29T00:38:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 Processor -- start 2024-07-29T00:38:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549cb70 0x557e454b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549e0e0 0x557e454a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 0x557e454a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34201b5020 con 0x557e4549e0e0 2024-07-29T00:38:18.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34201b4950 con 0x557e4549cb70 2024-07-29T00:38:18.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34201b1b90 con 0x557e454a7910 2024-07-29T00:38:18.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.714+0000 7f342a46b740 10 client.0 _ll_register_callbacks cb 0x7f340c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3424e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549cb70 0x557e454b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 0x557e454a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f341fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549e0e0 0x557e454a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 0x557e454a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46484/0 (socket says 172.21.15.175:46484) 2024-07-29T00:38:18.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3424e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549cb70 0x557e454b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47752/0 (socket says 172.21.15.175:47752) 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 -- 172.21.15.175:0/3776653586 learned_addr learned my addr 172.21.15.175:0/3776653586 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 -- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549cb70 msgr2=0x557e454b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549cb70 0x557e454b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 -- 172.21.15.175:0/3776653586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549e0e0 msgr2=0x557e454a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549e0e0 0x557e454a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 -- 172.21.15.175:0/3776653586 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e455216d0 con 0x557e454a7910 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3425800640 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 0x557e454a1f50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3418031a60 tx=0x7f3418031950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a4a3dded735491 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:18.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.715+0000 7f3406a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.716+0000 7f3406a00640 1 -- 172.21.15.175:0/3776653586 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341805d060 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.716+0000 7f3406a00640 1 -- 172.21.15.175:0/3776653586 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3418003970 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.716+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3414099f00 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.716+0000 7f3406a00640 1 -- 172.21.15.175:0/3776653586 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3418003b70 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.716+0000 7f342a46b740 10 client.25705 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e455227a0 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f3406a00640 1 -- 172.21.15.175:0/3776653586 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414099f00 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f3406a00640 1 -- 172.21.15.175:0/3776653586 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f341802f7b0 con 0x557e454a7910 2024-07-29T00:38:18.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 10 client.25705 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 10 client.25705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 20 client.25705 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e45522a80 0x557e45524e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 4 client.25705 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.717+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e4540a530 con 0x557e45522a80 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.718+0000 7f3424e00640 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e45522a80 0x557e45524e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.718+0000 7f3424e00640 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e45522a80 0x557e45524e70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5afe9a6896d8bc7 server_cookie=6a65a8cbc6c4a2b4 in_seq=0 out_seq=0 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.718+0000 7f3406a00640 10 client.25705 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:18.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.718+0000 7f3406a00640 1 -- 172.21.15.175:0/3776653586 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e4540a530 con 0x557e45522a80 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.718+0000 7f3406a00640 10 client.25705 handle_command_reply: tid=0 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.718+0000 7f342a46b740 1 client.25705 shutdown 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 msgr2=0x557e454a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 0x557e454a1f50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3418031a60 tx=0x7f3418031950 comp rx=0 tx=0).stop 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 shutdown_connections 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e454a7910 0x557e454a1f50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e4549e0e0 0x557e454a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557e45522a80 0x557e45524e70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 --2- 172.21.15.175:0/3776653586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e4549cb70 0x557e454b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 >> 172.21.15.175:0/3776653586 conn(0x557e45465950 msgr2=0x557e45498360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f3406a00640 0 client.25705 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 shutdown_connections 2024-07-29T00:38:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 1 -- 172.21.15.175:0/3776653586 wait complete. 2024-07-29T00:38:18.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.719+0000 7f342a46b740 20 client.25705 trim_cache size 0 max 16384 2024-07-29T00:38:18.732 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:18.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f34080a2d00 msgr2=0x7f34080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f34080a2d00 0x7f34080a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f34180318f0 tx=0x7f34180093f0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 msgr2=0x7f342019e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:18.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 0x7f342019e510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f341400e9f0 tx=0x7f341400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 shutdown_connections 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f34080a2d00 0x7f34080a51c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342010dd50 0x7f34201a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3420108670 0x7f342019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 --2- 172.21.15.175:0/259970761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3420107d10 0x7f342019e510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 >> 172.21.15.175:0/259970761 conn(0x7f3420077670 msgr2=0x7f3420078d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 shutdown_connections 2024-07-29T00:38:18.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:18.720+0000 7f3426200640 1 -- 172.21.15.175:0/259970761 wait complete. 2024-07-29T00:38:18.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 19303 13.52 MB/sec warmup 9 sec latency 16.610 ms 2024-07-29T00:38:19.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 21582 13.54 MB/sec warmup 10 sec latency 90.896 ms 2024-07-29T00:38:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:19 smithi181 ceph-mon[36706]: pgmap v57: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 679 KiB/s wr, 85 op/s 2024-07-29T00:38:20.005 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:19 smithi175 ceph-mon[32138]: pgmap v57: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 679 KiB/s wr, 85 op/s 2024-07-29T00:38:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:19 smithi204 ceph-mon[37368]: pgmap v57: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 679 KiB/s wr, 85 op/s 2024-07-29T00:38:20.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:38:20.565 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:20.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 23726 12.98 MB/sec warmup 11 sec latency 17.013 ms 2024-07-29T00:38:21.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 -- 172.21.15.175:0/991885357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076410 msgr2=0x7f64180767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 --2- 172.21.15.175:0/991885357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076410 0x7f64180767f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6408009900 tx=0x7f640802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 -- 172.21.15.175:0/991885357 shutdown_connections 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 --2- 172.21.15.175:0/991885357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418112720 0x7f6418114b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 --2- 172.21.15.175:0/991885357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418076d30 0x7f64181121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 --2- 172.21.15.175:0/991885357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076410 0x7f64180767f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 -- 172.21.15.175:0/991885357 >> 172.21.15.175:0/991885357 conn(0x7f64180ff780 msgr2=0x7f6418101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 -- 172.21.15.175:0/991885357 shutdown_connections 2024-07-29T00:38:21.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.234+0000 7f641da00640 1 -- 172.21.15.175:0/991885357 wait complete. 2024-07-29T00:38:21.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 Processor -- start 2024-07-29T00:38:21.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 -- start start 2024-07-29T00:38:21.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418076410 0x7f641819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076d30 0x7f641819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418112720 0x7f64181a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64181182e0 con 0x7f6418112720 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6418118160 con 0x7f6418076d30 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.235+0000 7f641da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6418118460 con 0x7f6418076410 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f640fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076d30 0x7f641819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418076410 0x7f641819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f640fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076d30 0x7f641819ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51300/0 (socket says 172.21.15.175:51300) 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f640fe00640 1 -- 172.21.15.175:0/2053150406 learned_addr learned my addr 172.21.15.175:0/2053150406 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6417400640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418112720 0x7f64181a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 -- 172.21.15.175:0/2053150406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076d30 msgr2=0x7f641819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076d30 0x7f641819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 -- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418112720 msgr2=0x7f64181a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418112720 0x7f64181a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 -- 172.21.15.175:0/2053150406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64181a36e0 con 0x7f6418076410 2024-07-29T00:38:21.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6417400640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418112720 0x7f64181a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.236+0000 7f6416a00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418076410 0x7f641819e730 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f640802a910 tx=0x7f6408003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.237+0000 7f640f400640 1 -- 172.21.15.175:0/2053150406 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640803f070 con 0x7f6418076410 2024-07-29T00:38:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.237+0000 7f640f400640 1 -- 172.21.15.175:0/2053150406 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6408031c90 con 0x7f6418076410 2024-07-29T00:38:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.237+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64181a3970 con 0x7f6418076410 2024-07-29T00:38:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.237+0000 7f640f400640 1 -- 172.21.15.175:0/2053150406 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408035030 con 0x7f6418076410 2024-07-29T00:38:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.237+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64181ab2b0 con 0x7f6418076410 2024-07-29T00:38:21.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.240+0000 7f640f400640 1 -- 172.21.15.175:0/2053150406 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6408042070 con 0x7f6418076410 2024-07-29T00:38:21.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 Processor -- start 2024-07-29T00:38:21.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 -- start start 2024-07-29T00:38:21.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50122440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce50088ab0 con 0x55ce50122980 2024-07-29T00:38:21.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ce50020a80 con 0x55ce50123ef0 2024-07-29T00:38:21.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f64204d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ce50084a70 con 0x55ce5012d720 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f6417400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f6417400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50122440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51302/0 (socket says 172.21.15.175:51302) 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.241+0000 7f6417400640 1 -- 172.21.15.175:0/1274374294 learned_addr learned my addr 172.21.15.175:0/1274374294 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6416a00640 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f640fe00640 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6417400640 1 -- 172.21.15.175:0/1274374294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 msgr2=0x55ce5012fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6417400640 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6417400640 1 -- 172.21.15.175:0/1274374294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 msgr2=0x55ce50122dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6417400640 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50122dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6417400640 1 -- 172.21.15.175:0/1274374294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce4fdfa3c0 con 0x55ce50123ef0 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f640fe00640 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:21.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6417400640 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50122440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6410009f80 tx=0x7f6410010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535b73ef64c46dd5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f640f400640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63fc0a2d70 0x7f63fc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6407e00640 1 -- 172.21.15.175:0/1274374294 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410011700 con 0x55ce50123ef0 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6407e00640 1 -- 172.21.15.175:0/1274374294 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6410011d80 con 0x55ce50123ef0 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.242+0000 7f6407e00640 1 -- 172.21.15.175:0/1274374294 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641001a380 con 0x55ce50123ef0 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.243+0000 7f640f400640 1 -- 172.21.15.175:0/2053150406 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f64080efdf0 con 0x7f6418076410 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.243+0000 7f6416a00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63fc0a2d70 0x7f63fc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.243+0000 7f64204d6740 1 -- 172.21.15.175:0/1274374294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 msgr2=0x55ce50122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.243+0000 7f64204d6740 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50122440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6410009f80 tx=0x7f6410010860 comp rx=0 tx=0).stop 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 -- 172.21.15.175:0/1274374294 shutdown_connections 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 --2- 172.21.15.175:0/1274374294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50122440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 -- 172.21.15.175:0/1274374294 >> 172.21.15.175:0/1274374294 conn(0x55ce500eb950 msgr2=0x55ce5011f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:21.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f6416a00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63fc0a2d70 0x7f63fc0a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ce5011fe70 tx=0x7f640802a3f0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:21.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 -- 172.21.15.175:0/1274374294 shutdown_connections 2024-07-29T00:38:21.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.244+0000 7f64204d6740 1 -- 172.21.15.175:0/1274374294 wait complete. 2024-07-29T00:38:21.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.245+0000 7f64204d6740 1 -- start start 2024-07-29T00:38:21.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.245+0000 7f64204d6740 1 Processor -- start 2024-07-29T00:38:21.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce5012a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce5012ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce501304c0 con 0x55ce50122980 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ce50043c10 con 0x55ce50123ef0 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ce50130640 con 0x55ce5012d720 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f64204d6740 10 client.0 _ll_register_callbacks cb 0x55ce5000ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f6417400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce5012a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f6417400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce5012a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47424/0 (socket says 172.21.15.175:47424) 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f6417400640 1 -- 172.21.15.175:0/508763341 learned_addr learned my addr 172.21.15.175:0/508763341 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f6417400640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.246+0000 7f640fe00640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce5012ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f640fe00640 1 -- 172.21.15.175:0/508763341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 msgr2=0x55ce5012b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f640fe00640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f640fe00640 1 -- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 msgr2=0x55ce5012a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f640fe00640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce5012a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f640fe00640 1 -- 172.21.15.175:0/508763341 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce500471e0 con 0x55ce50123ef0 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f6417400640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce5012a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f640fe00640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce5012ac80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55ce50123420 tx=0x7f6400013b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4cbb1705ee4e29 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f6404c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.247+0000 7f6404c00640 1 -- 172.21.15.175:0/508763341 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400011990 con 0x55ce50123ef0 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.248+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6410011d80 con 0x55ce50123ef0 2024-07-29T00:38:21.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.248+0000 7f6404c00640 1 -- 172.21.15.175:0/508763341 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6400011b30 con 0x55ce50123ef0 2024-07-29T00:38:21.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.248+0000 7f6404c00640 1 -- 172.21.15.175:0/508763341 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6400011d30 con 0x55ce50123ef0 2024-07-29T00:38:21.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.248+0000 7f64204d6740 10 client.25721 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:21.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.248+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce5013c2f0 con 0x55ce50123ef0 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.248+0000 7f6404c00640 1 -- 172.21.15.175:0/508763341 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6410011d80 con 0x55ce50123ef0 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f6404c00640 1 -- 172.21.15.175:0/508763341 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f640001a070 con 0x55ce50123ef0 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f64204d6740 10 client.25721 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f64204d6740 10 client.25721 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f64204d6740 20 client.25721 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f64204d6740 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce5013d700 0x55ce5013fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f64204d6740 4 client.25721 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce50096020 con 0x55ce5013d700 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.249+0000 7f6417400640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce5013d700 0x55ce5013fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.250+0000 7f6417400640 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce5013d700 0x55ce5013fb60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b5cf4db7a2c7e43 server_cookie=63a8e4ecbbf8e055 in_seq=0 out_seq=0 2024-07-29T00:38:21.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.250+0000 7f6404c00640 10 client.25721 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:21.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f6404c00640 1 -- 172.21.15.175:0/508763341 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce50096020 con 0x55ce5013d700 2024-07-29T00:38:21.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f6404c00640 10 client.25721 handle_command_reply: tid=0 2024-07-29T00:38:21.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 client.25721 shutdown 2024-07-29T00:38:21.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 msgr2=0x55ce5012ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce5012ac80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55ce50123420 tx=0x7f6400013b20 comp rx=0 tx=0).stop 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 shutdown_connections 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce5013d700 0x55ce5013fb60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce5012ac80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 --2- 172.21.15.175:0/508763341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce5012a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 >> 172.21.15.175:0/508763341 conn(0x55ce500eb950 msgr2=0x55ce50124ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.251+0000 7f6404c00640 0 client.25721 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.252+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 shutdown_connections 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.252+0000 7f64204d6740 1 -- 172.21.15.175:0/508763341 wait complete. 2024-07-29T00:38:21.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.252+0000 7f64204d6740 20 client.25721 trim_cache size 0 max 16384 2024-07-29T00:38:21.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 Processor -- start 2024-07-29T00:38:21.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 -- start start 2024-07-29T00:38:21.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50145430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50145970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64181b3c30 con 0x55ce50122980 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64181b5770 con 0x55ce50123ef0 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.267+0000 7f64204d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64181b4300 con 0x55ce5012d720 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f6417400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f6417400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47428/0 (socket says 172.21.15.175:47428) 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51322/0 (socket says 172.21.15.175:51322) 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f6417400640 1 -- 172.21.15.175:0/4050691625 learned_addr learned my addr 172.21.15.175:0/4050691625 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f6417400640 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 -- 172.21.15.175:0/4050691625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 msgr2=0x55ce5012bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 -- 172.21.15.175:0/4050691625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 msgr2=0x55ce50145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 -- 172.21.15.175:0/4050691625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce50128c90 con 0x55ce50123ef0 2024-07-29T00:38:21.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f6417400640 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:21.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.268+0000 7f640fe00640 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50145970 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6400023940 tx=0x7f6400009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b8e7251dbd104d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:21.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f6407400640 1 -- 172.21.15.175:0/4050691625 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400030070 con 0x55ce50123ef0 2024-07-29T00:38:21.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f6407400640 1 -- 172.21.15.175:0/4050691625 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6400011990 con 0x55ce50123ef0 2024-07-29T00:38:21.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f6407400640 1 -- 172.21.15.175:0/4050691625 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400003ad0 con 0x55ce50123ef0 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f64204d6740 1 -- 172.21.15.175:0/4050691625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 msgr2=0x55ce50145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f64204d6740 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50145970 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6400023940 tx=0x7f6400009900 comp rx=0 tx=0).stop 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f64204d6740 1 -- 172.21.15.175:0/4050691625 shutdown_connections 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.269+0000 7f64204d6740 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce5012bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50145970 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 --2- 172.21.15.175:0/4050691625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 -- 172.21.15.175:0/4050691625 >> 172.21.15.175:0/4050691625 conn(0x55ce500eb950 msgr2=0x55ce50129010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 -- 172.21.15.175:0/4050691625 shutdown_connections 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 -- 172.21.15.175:0/4050691625 wait complete. 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 -- start start 2024-07-29T00:38:21.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.270+0000 7f64204d6740 1 Processor -- start 2024-07-29T00:38:21.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50130890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50130dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce50131310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64181b50a0 con 0x55ce50122980 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64181b49d0 con 0x55ce50123ef0 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64181b1c10 con 0x55ce5012d720 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f64204d6740 10 client.0 _ll_register_callbacks cb 0x7f6400009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f640fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50130dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f6417400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50130890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f640fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50130dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51326/0 (socket says 172.21.15.175:51326) 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f640fe00640 1 -- 172.21.15.175:0/711166184 learned_addr learned my addr 172.21.15.175:0/711166184 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.271+0000 7f6417400640 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce50131310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f640fe00640 1 -- 172.21.15.175:0/711166184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 msgr2=0x55ce50131310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f640fe00640 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce50131310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f640fe00640 1 -- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 msgr2=0x55ce50130890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f640fe00640 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50130890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f640fe00640 1 -- 172.21.15.175:0/711166184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce501a63f0 con 0x55ce50123ef0 2024-07-29T00:38:21.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f640fe00640 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50130dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6400000c00 tx=0x7f6400005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a77996f7a2b59e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:21.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f6404c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:21.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.272+0000 7f6404c00640 1 -- 172.21.15.175:0/711166184 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400030040 con 0x55ce50123ef0 2024-07-29T00:38:21.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f6404c00640 1 -- 172.21.15.175:0/711166184 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f64000053a0 con 0x55ce50123ef0 2024-07-29T00:38:21.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6400011990 con 0x55ce50123ef0 2024-07-29T00:38:21.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f6404c00640 1 -- 172.21.15.175:0/711166184 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64000055a0 con 0x55ce50123ef0 2024-07-29T00:38:21.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 10 client.25727 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce501a76d0 con 0x55ce50123ef0 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f6404c00640 1 -- 172.21.15.175:0/711166184 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6400011990 con 0x55ce50123ef0 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f6404c00640 1 -- 172.21.15.175:0/711166184 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f640001a030 con 0x55ce50123ef0 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 10 client.25727 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 10 client.25727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 20 client.25727 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce501a7d30 0x55ce501aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 4 client.25727 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.273+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce50090530 con 0x55ce501a7d30 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.274+0000 7f6417400640 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce501a7d30 0x55ce501aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.274+0000 7f6417400640 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce501a7d30 0x55ce501aa150 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c9d515ac61b744f server_cookie=c621504fd1bd4133 in_seq=0 out_seq=0 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.274+0000 7f6404c00640 10 client.25727 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:21.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f6404c00640 1 -- 172.21.15.175:0/711166184 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce50090530 con 0x55ce501a7d30 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f6404c00640 10 client.25727 handle_command_reply: tid=0 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 client.25727 shutdown 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 msgr2=0x55ce50130dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50130dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6400000c00 tx=0x7f6400005990 comp rx=0 tx=0).stop 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 shutdown_connections 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55ce501a7d30 0x55ce501aa150 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce5012d720 0x55ce50131310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce50123ef0 0x55ce50130dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 --2- 172.21.15.175:0/711166184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce50122980 0x55ce50130890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f6404c00640 0 client.25727 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 >> 172.21.15.175:0/711166184 conn(0x55ce500eb950 msgr2=0x55ce5011d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 shutdown_connections 2024-07-29T00:38:21.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 1 -- 172.21.15.175:0/711166184 wait complete. 2024-07-29T00:38:21.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.275+0000 7f64204d6740 20 client.25727 trim_cache size 0 max 16384 2024-07-29T00:38:21.278 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:21.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63fc0a2d70 msgr2=0x7f63fc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63fc0a2d70 0x7f63fc0a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ce5011fe70 tx=0x7f640802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418076410 msgr2=0x7f641819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418076410 0x7f641819e730 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f640802a910 tx=0x7f6408003b20 comp rx=0 tx=0).stop 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 shutdown_connections 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63fc0a2d70 0x7f63fc0a5230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6418112720 0x7f64181a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6418076d30 0x7f641819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 --2- 172.21.15.175:0/2053150406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6418076410 0x7f641819e730 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 >> 172.21.15.175:0/2053150406 conn(0x7f64180ff780 msgr2=0x7f6418101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 shutdown_connections 2024-07-29T00:38:21.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:21.276+0000 7f641da00640 1 -- 172.21.15.175:0/2053150406 wait complete. 2024-07-29T00:38:21.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 25607 12.93 MB/sec warmup 12 sec latency 13.513 ms 2024-07-29T00:38:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:21 smithi181 ceph-mon[36706]: pgmap v58: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 679 KiB/s wr, 85 op/s 2024-07-29T00:38:22.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:21 smithi175 ceph-mon[32138]: pgmap v58: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 679 KiB/s wr, 85 op/s 2024-07-29T00:38:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:21 smithi204 ceph-mon[37368]: pgmap v58: 97 pgs: 97 active+clean; 8.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 679 KiB/s wr, 85 op/s 2024-07-29T00:38:22.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:38:22.750 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:22.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 28374 13.25 MB/sec warmup 13 sec latency 8.614 ms 2024-07-29T00:38:23.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 -- 172.21.15.175:0/4000122326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea8102050 msgr2=0x7f2ea810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 --2- 172.21.15.175:0/4000122326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea8102050 0x7f2ea810e580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009ca0 tx=0x7f2ea002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 -- 172.21.15.175:0/4000122326 shutdown_connections 2024-07-29T00:38:23.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 --2- 172.21.15.175:0/4000122326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea8102050 0x7f2ea810e580 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 --2- 172.21.15.175:0/4000122326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea81016b0 0x7f2ea8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 --2- 172.21.15.175:0/4000122326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea81076b0 0x7f2ea8107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 -- 172.21.15.175:0/4000122326 >> 172.21.15.175:0/4000122326 conn(0x7f2ea80fd430 msgr2=0x7f2ea80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:23.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 -- 172.21.15.175:0/4000122326 shutdown_connections 2024-07-29T00:38:23.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.431+0000 7f2ea7200640 1 -- 172.21.15.175:0/4000122326 wait complete. 2024-07-29T00:38:23.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 Processor -- start 2024-07-29T00:38:23.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 -- start start 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea81016b0 0x7f2ea8197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea8102050 0x7f2ea8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea81076b0 0x7f2ea819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8111b00 con 0x7f2ea8102050 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8111980 con 0x7f2ea81016b0 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.433+0000 7f2ea7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8111c80 con 0x7f2ea81076b0 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2ea5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea81016b0 0x7f2ea8197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2ea5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea81016b0 0x7f2ea8197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51340/0 (socket says 172.21.15.175:51340) 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2ea5e00640 1 -- 172.21.15.175:0/3484712929 learned_addr learned my addr 172.21.15.175:0/3484712929 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:23.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea8102050 0x7f2ea8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2ea6800640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea81076b0 0x7f2ea819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2e9fe00640 1 -- 172.21.15.175:0/3484712929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea81076b0 msgr2=0x7f2ea819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea81076b0 0x7f2ea819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2e9fe00640 1 -- 172.21.15.175:0/3484712929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea81016b0 msgr2=0x7f2ea8197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea81016b0 0x7f2ea8197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.434+0000 7f2e9fe00640 1 -- 172.21.15.175:0/3484712929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea819ced0 con 0x7f2ea8102050 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.435+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea8102050 0x7f2ea8198460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e9f0 tx=0x7f2e9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.435+0000 7f2e9ea00640 1 -- 172.21.15.175:0/3484712929 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000ccc0 con 0x7f2ea8102050 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.435+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea819d1c0 con 0x7f2ea8102050 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.435+0000 7f2e9ea00640 1 -- 172.21.15.175:0/3484712929 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e90010910 con 0x7f2ea8102050 2024-07-29T00:38:23.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.435+0000 7f2e9ea00640 1 -- 172.21.15.175:0/3484712929 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90018960 con 0x7f2ea8102050 2024-07-29T00:38:23.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.435+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea81a4aa0 con 0x7f2ea8102050 2024-07-29T00:38:23.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 Processor -- start 2024-07-29T00:38:23.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 -- start start 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c69a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69ab70 0x55ef1c69afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef1c600ab0 con 0x55ef1c69ab70 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ef1c598a80 con 0x55ef1c69c0e0 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.439+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef1c5fca70 con 0x55ef1c6a9d20 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c69a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c69a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51342/0 (socket says 172.21.15.175:51342) 2024-07-29T00:38:23.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 -- 172.21.15.175:0/3657672156 learned_addr learned my addr 172.21.15.175:0/3657672156 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:23.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69ab70 0x55ef1c69afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 -- 172.21.15.175:0/3657672156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 msgr2=0x55ef1c6ac110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:38:23.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6ac110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 -- 172.21.15.175:0/3657672156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69ab70 msgr2=0x55ef1c69afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69ab70 0x55ef1c69afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2ea6800640 1 -- 172.21.15.175:0/3657672156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1c3723c0 con 0x55ef1c69c0e0 2024-07-29T00:38:23.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.440+0000 7f2e9ea00640 1 -- 172.21.15.175:0/3484712929 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2e90010ac0 con 0x7f2ea8102050 2024-07-29T00:38:23.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.441+0000 7f2ea6800640 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c69a630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009870 tx=0x7f2ea00098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6776d122355e8c33 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:23.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.441+0000 7f2e97400640 1 -- 172.21.15.175:0/3657672156 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0031da0 con 0x55ef1c69c0e0 2024-07-29T00:38:23.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.441+0000 7f2e97400640 1 -- 172.21.15.175:0/3657672156 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea0036040 con 0x55ef1c69c0e0 2024-07-29T00:38:23.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.441+0000 7f2e97400640 1 -- 172.21.15.175:0/3657672156 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00043a0 con 0x55ef1c69c0e0 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/3657672156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 msgr2=0x55ef1c69a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c69a630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009870 tx=0x7f2ea00098a0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/3657672156 shutdown_connections 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69ab70 0x55ef1c69afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/3657672156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c69a630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/3657672156 >> 172.21.15.175:0/3657672156 conn(0x55ef1c663950 msgr2=0x55ef1c697200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:23.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/3657672156 shutdown_connections 2024-07-29T00:38:23.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.442+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/3657672156 wait complete. 2024-07-29T00:38:23.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.443+0000 7f2e9ea00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2e8c0a2c00 0x7f2e8c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.443+0000 7f2e9ea00640 1 -- 172.21.15.175:0/3484712929 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f2e900cf910 con 0x7f2ea8102050 2024-07-29T00:38:23.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.443+0000 7f2eaf9d5740 1 -- start start 2024-07-29T00:38:23.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.443+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2e8c0a2c00 0x7f2e8c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.443+0000 7f2eaf9d5740 1 Processor -- start 2024-07-29T00:38:23.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef1c6aca60 con 0x55ef1c69c0e0 2024-07-29T00:38:23.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ef1c5bbc10 con 0x55ef1c69ab70 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef1c6acbe0 con 0x55ef1c6a9d20 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2e8c0a2c00 0x7f2e8c0a50c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55ef1c698060 tx=0x7f2e98005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2eaf9d5740 10 client.0 _ll_register_callbacks cb 0x55ef1c584a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2ea5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2ea5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51352/0 (socket says 172.21.15.175:51352) 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.444+0000 7f2ea5e00640 1 -- 172.21.15.175:0/672658764 learned_addr learned my addr 172.21.15.175:0/672658764 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea6800640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea5e00640 1 -- 172.21.15.175:0/672658764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 msgr2=0x55ef1c6a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea5e00640 1 -- 172.21.15.175:0/672658764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 msgr2=0x55ef1c6a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea5e00640 1 -- 172.21.15.175:0/672658764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1c5bf1e0 con 0x55ef1c69ab70 2024-07-29T00:38:23.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:23.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a6d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e98004610 tx=0x7f2e98004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de41e8e5f942101 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:23.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.445+0000 7f2e8be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:23.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2e8be00640 1 -- 172.21.15.175:0/672658764 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98009e10 con 0x55ef1c69ab70 2024-07-29T00:38:23.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2e8be00640 1 -- 172.21.15.175:0/672658764 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f2e9801f610 con 0x55ef1c69ab70 2024-07-29T00:38:23.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea0036040 con 0x55ef1c69ab70 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2e8be00640 1 -- 172.21.15.175:0/672658764 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e9801f810 con 0x55ef1c69ab70 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2eaf9d5740 10 client.25739 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef1c6b88d0 con 0x55ef1c69ab70 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.446+0000 7f2e8be00640 1 -- 172.21.15.175:0/672658764 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea0036040 con 0x55ef1c69ab70 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2e8be00640 1 -- 172.21.15.175:0/672658764 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e98027020 con 0x55ef1c69ab70 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2eaf9d5740 10 client.25739 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2eaf9d5740 10 client.25739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2eaf9d5740 20 client.25739 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c6b9d90 0x55ef1c6bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2eaf9d5740 4 client.25739 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef1c60e020 con 0x55ef1c6b9d90 2024-07-29T00:38:23.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.447+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c6b9d90 0x55ef1c6bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.448+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c6b9d90 0x55ef1c6bc1c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822bd8639069c854 server_cookie=943adc52f7c48c95 in_seq=0 out_seq=0 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.448+0000 7f2e8be00640 10 client.25739 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.448+0000 7f2e8be00640 1 -- 172.21.15.175:0/672658764 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef1c60e020 con 0x55ef1c6b9d90 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.448+0000 7f2e8be00640 10 client.25739 handle_command_reply: tid=0 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.448+0000 7f2eaf9d5740 1 client.25739 shutdown 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 msgr2=0x55ef1c6a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a6d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e98004610 tx=0x7f2e98004730 comp rx=0 tx=0).stop 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 shutdown_connections 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c6b9d90 0x55ef1c6bc1c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/672658764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a6d30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2e8be00640 0 client.25739 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 >> 172.21.15.175:0/672658764 conn(0x55ef1c663950 msgr2=0x55ef1c69ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 shutdown_connections 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/672658764 wait complete. 2024-07-29T00:38:23.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.449+0000 7f2eaf9d5740 20 client.25739 trim_cache size 0 max 16384 2024-07-29T00:38:23.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 Processor -- start 2024-07-29T00:38:23.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 -- start start 2024-07-29T00:38:23.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ab310 con 0x55ef1c6a9d20 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ace50 con 0x55ef1c69c0e0 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.465+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ab9e0 con 0x55ef1c69ab70 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2ea6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51358/0 (socket says 172.21.15.175:51358) 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 -- 172.21.15.175:0/1167113546 learned_addr learned my addr 172.21.15.175:0/1167113546 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 -- 172.21.15.175:0/1167113546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c69ab70 msgr2=0x55ef1c6a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 -- 172.21.15.175:0/1167113546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c6a9d20 msgr2=0x55ef1c6c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6c1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 -- 172.21.15.175:0/1167113546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1c6a5ce0 con 0x55ef1c69c0e0 2024-07-29T00:38:23.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2ea6800640 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:23.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.466+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a8ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cf10 tx=0x7f2e90009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789db72f1255c48d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.467+0000 7f2e96a00640 1 -- 172.21.15.175:0/1167113546 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9009a0b0 con 0x55ef1c69c0e0 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.467+0000 7f2e96a00640 1 -- 172.21.15.175:0/1167113546 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9009f030 con 0x55ef1c69c0e0 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.467+0000 7f2e96a00640 1 -- 172.21.15.175:0/1167113546 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9009b460 con 0x55ef1c69c0e0 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.467+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/1167113546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 msgr2=0x55ef1c6a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.467+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a8ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cf10 tx=0x7f2e90009700 comp rx=0 tx=0).stop 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/1167113546 shutdown_connections 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a8ca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/1167113546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c69ab70 0x55ef1c6a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/1167113546 >> 172.21.15.175:0/1167113546 conn(0x55ef1c663950 msgr2=0x55ef1c6a8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:23.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/1167113546 shutdown_connections 2024-07-29T00:38:23.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/1167113546 wait complete. 2024-07-29T00:38:23.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.468+0000 7f2eaf9d5740 1 -- start start 2024-07-29T00:38:23.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 Processor -- start 2024-07-29T00:38:23.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ac780 con 0x55ef1c69c0e0 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ac0b0 con 0x55ef1c69ab70 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81a9310 con 0x55ef1c6a9d20 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2eaf9d5740 10 client.0 _ll_register_callbacks cb 0x7f2e98009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2ea5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2ea5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51370/0 (socket says 172.21.15.175:51370) 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2ea5e00640 1 -- 172.21.15.175:0/971322079 learned_addr learned my addr 172.21.15.175:0/971322079 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.469+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea6800640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea5e00640 1 -- 172.21.15.175:0/971322079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 msgr2=0x55ef1c6a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea5e00640 1 -- 172.21.15.175:0/971322079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 msgr2=0x55ef1c6a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea5e00640 1 -- 172.21.15.175:0/971322079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1c6a43f0 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2ea5e00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6b2340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e98012cb0 tx=0x7f2e98003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945b4f0c1bda6130 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2e8be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.470+0000 7f2e8be00640 1 -- 172.21.15.175:0/971322079 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98036030 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2e8be00640 1 -- 172.21.15.175:0/971322079 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f2e98018040 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9009f030 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2e8be00640 1 -- 172.21.15.175:0/971322079 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e9801f610 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2eaf9d5740 10 client.25745 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef1c724860 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2e8be00640 1 -- 172.21.15.175:0/971322079 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9009f030 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2e8be00640 1 -- 172.21.15.175:0/971322079 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e98027040 con 0x55ef1c69ab70 2024-07-29T00:38:23.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.471+0000 7f2eaf9d5740 10 client.25745 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:23.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2eaf9d5740 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:23.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2eaf9d5740 20 client.25745 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:23.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c724fb0 0x55ef1c7273d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:23.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2eaf9d5740 4 client.25745 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:23.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef1c608530 con 0x55ef1c724fb0 2024-07-29T00:38:23.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c724fb0 0x55ef1c7273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:23.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2e9fe00640 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c724fb0 0x55ef1c7273d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544b68b4f4624062 server_cookie=53c53ce3252d1f24 in_seq=0 out_seq=0 2024-07-29T00:38:23.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.472+0000 7f2e8be00640 10 client.25745 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:23.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2e8be00640 1 -- 172.21.15.175:0/971322079 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef1c608530 con 0x55ef1c724fb0 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2e8be00640 10 client.25745 handle_command_reply: tid=0 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 client.25745 shutdown 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 msgr2=0x55ef1c6b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6b2340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e98012cb0 tx=0x7f2e98003f90 comp rx=0 tx=0).stop 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 shutdown_connections 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ef1c724fb0 0x55ef1c7273d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef1c6a9d20 0x55ef1c6a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef1c69c0e0 0x55ef1c6a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 --2- 172.21.15.175:0/971322079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef1c69ab70 0x55ef1c6b2340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 >> 172.21.15.175:0/971322079 conn(0x55ef1c663950 msgr2=0x55ef1c6a8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2e8be00640 0 client.25745 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 shutdown_connections 2024-07-29T00:38:23.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 1 -- 172.21.15.175:0/971322079 wait complete. 2024-07-29T00:38:23.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.473+0000 7f2eaf9d5740 20 client.25745 trim_cache size 0 max 16384 2024-07-29T00:38:23.477 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2e8c0a2c00 msgr2=0x7f2e8c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2e8c0a2c00 0x7f2e8c0a50c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55ef1c698060 tx=0x7f2e98005f90 comp rx=0 tx=0).stop 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea8102050 msgr2=0x7f2ea8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea8102050 0x7f2ea8198460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e9f0 tx=0x7f2e9000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 shutdown_connections 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea81076b0 0x7f2ea819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2e8c0a2c00 0x7f2e8c0a50c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea8102050 0x7f2ea8198460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 --2- 172.21.15.175:0/3484712929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea81016b0 0x7f2ea8197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.474+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 >> 172.21.15.175:0/3484712929 conn(0x7f2ea80fd430 msgr2=0x7f2ea80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.475+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 shutdown_connections 2024-07-29T00:38:23.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:23.475+0000 7f2ea7200640 1 -- 172.21.15.175:0/3484712929 wait complete. 2024-07-29T00:38:23.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 29595 12.57 MB/sec warmup 14 sec latency 45.442 ms 2024-07-29T00:38:23.929 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:23 smithi175 ceph-mon[32138]: pgmap v59: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-07-29T00:38:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:23 smithi181 ceph-mon[36706]: pgmap v59: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-07-29T00:38:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:23 smithi204 ceph-mon[37368]: pgmap v59: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-07-29T00:38:24.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 32241 12.85 MB/sec warmup 15 sec latency 15.441 ms 2024-07-29T00:38:25.454 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:38:25.454 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:38:25.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 34311 12.50 MB/sec warmup 16 sec latency 11.398 ms 2024-07-29T00:38:25.942 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:25 smithi181 ceph-mon[36706]: pgmap v60: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-07-29T00:38:26.033 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:25 smithi175 ceph-mon[32138]: pgmap v60: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-07-29T00:38:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:25 smithi204 ceph-mon[37368]: pgmap v60: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.635+0000 7f2313800640 1 -- 172.21.15.175:0/795340302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 msgr2=0x7f230c10caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.635+0000 7f2313800640 1 --2- 172.21.15.175:0/795340302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 0x7f230c10caf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009900 tx=0x7f22f802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 -- 172.21.15.175:0/795340302 shutdown_connections 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 --2- 172.21.15.175:0/795340302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c10d220 0x7f230c114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 --2- 172.21.15.175:0/795340302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 0x7f230c10caf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 --2- 172.21.15.175:0/795340302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f230c108140 0x7f230c108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 -- 172.21.15.175:0/795340302 >> 172.21.15.175:0/795340302 conn(0x7f230c077890 msgr2=0x7f230c079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:26.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 -- 172.21.15.175:0/795340302 shutdown_connections 2024-07-29T00:38:26.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.636+0000 7f2313800640 1 -- 172.21.15.175:0/795340302 wait complete. 2024-07-29T00:38:26.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.637+0000 7f2313800640 1 Processor -- start 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.637+0000 7f2313800640 1 -- start start 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2313800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 0x7f230c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2313800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 0x7f230c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2313800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f230c10d220 0x7f230c1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2313800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f230c118270 con 0x7f230c10d220 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2313800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1180f0 con 0x7f230c108af0 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2313800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1183f0 con 0x7f230c108140 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2310e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 0x7f230c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2310e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 0x7f230c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38028/0 (socket says 172.21.15.175:38028) 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f2310e00640 1 -- 172.21.15.175:0/86670393 learned_addr learned my addr 172.21.15.175:0/86670393 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.638+0000 7f230be00640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 0x7f230c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2310e00640 1 -- 172.21.15.175:0/86670393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 msgr2=0x7f230c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2310e00640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 0x7f230c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2311800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f230c10d220 0x7f230c1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2310e00640 1 -- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f230c10d220 msgr2=0x7f230c1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2310e00640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f230c10d220 0x7f230c1a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2310e00640 1 -- 172.21.15.175:0/86670393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c1a2700 con 0x7f230c108140 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2310e00640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 0x7f230c19e7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00d950 tx=0x7f22fc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2309600640 1 -- 172.21.15.175:0/86670393 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc017070 con 0x7f230c108140 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2309600640 1 -- 172.21.15.175:0/86670393 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc009c30 con 0x7f230c108140 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c1a29f0 con 0x7f230c108140 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2309600640 1 -- 172.21.15.175:0/86670393 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc00f6b0 con 0x7f230c108140 2024-07-29T00:38:26.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.639+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1ab360 con 0x7f230c108140 2024-07-29T00:38:26.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.644+0000 7f2309600640 1 -- 172.21.15.175:0/86670393 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f22fc010460 con 0x7f230c108140 2024-07-29T00:38:26.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.644+0000 7f231647d740 1 Processor -- start 2024-07-29T00:38:26.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.644+0000 7f231647d740 1 -- start start 2024-07-29T00:38:26.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.644+0000 7f231647d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a6f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.644+0000 7f231647d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a7017e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f231647d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a704110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f231647d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56099a65c8a0 con 0x56099a701d20 2024-07-29T00:38:26.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f231647d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56099a5f4710 con 0x56099a6f7460 2024-07-29T00:38:26.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f231647d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56099a658860 con 0x56099a6f6b40 2024-07-29T00:38:26.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f2310e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a7017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f2311800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a6f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f2311800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a6f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38034/0 (socket says 172.21.15.175:38034) 2024-07-29T00:38:26.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f2311800640 1 -- 172.21.15.175:0/739317181 learned_addr learned my addr 172.21.15.175:0/739317181 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:26.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f230be00640 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a704110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f2311800640 1 -- 172.21.15.175:0/739317181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 msgr2=0x56099a7017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.645+0000 7f2311800640 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a7017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2311800640 1 -- 172.21.15.175:0/739317181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 msgr2=0x56099a704110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2311800640 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a704110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2311800640 1 -- 172.21.15.175:0/739317181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099a3ce6a0 con 0x56099a6f6b40 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2310e00640 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a7017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2309600640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f22f00a2d70 0x7f22f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2311800640 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a6f6f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2304009e20 tx=0x7f230400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c569986b19882aea server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2309600640 1 -- 172.21.15.175:0/86670393 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f22fc0cf3f0 con 0x7f230c108140 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2302000640 1 -- 172.21.15.175:0/739317181 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304011440 con 0x56099a6f6b40 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2302000640 1 -- 172.21.15.175:0/739317181 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304011ac0 con 0x56099a6f6b40 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.646+0000 7f2302000640 1 -- 172.21.15.175:0/739317181 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230401bc70 con 0x56099a6f6b40 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f2310e00640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f22f00a2d70 0x7f22f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 -- 172.21.15.175:0/739317181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 msgr2=0x56099a6f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a6f6f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2304009e20 tx=0x7f230400f730 comp rx=0 tx=0).stop 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 -- 172.21.15.175:0/739317181 shutdown_connections 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a704110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a7017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 --2- 172.21.15.175:0/739317181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a6f6f20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 -- 172.21.15.175:0/739317181 >> 172.21.15.175:0/739317181 conn(0x56099a6f30d0 msgr2=0x56099a6f34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:26.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f2310e00640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f22f00a2d70 0x7f22f00a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0d2fd0 tx=0x7f22fc0d2ea0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:26.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 -- 172.21.15.175:0/739317181 shutdown_connections 2024-07-29T00:38:26.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.647+0000 7f231647d740 1 -- 172.21.15.175:0/739317181 wait complete. 2024-07-29T00:38:26.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.648+0000 7f231647d740 1 -- start start 2024-07-29T00:38:26.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.648+0000 7f231647d740 1 Processor -- start 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a6f6b40 0x56099a6fec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a6ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a701d20 0x56099a6ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56099a7048f0 con 0x56099a6f6b40 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56099a6178c0 con 0x56099a6f7460 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56099a704a70 con 0x56099a701d20 2024-07-29T00:38:26.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f231647d740 10 client.0 _ll_register_callbacks cb 0x56099a603a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a6ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a6ff1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51410/0 (socket says 172.21.15.175:51410) 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 -- 172.21.15.175:0/424719605 learned_addr learned my addr 172.21.15.175:0/424719605 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f2311800640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a6f6b40 0x56099a6fec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f2311800640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a701d20 0x56099a6ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 -- 172.21.15.175:0/424719605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a701d20 msgr2=0x56099a6ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a701d20 0x56099a6ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 -- 172.21.15.175:0/424719605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a6f6b40 msgr2=0x56099a6fec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a6f6b40 0x56099a6fec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f230be00640 1 -- 172.21.15.175:0/424719605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56099a5fabc0 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.649+0000 7f2311800640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a701d20 0x56099a6ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.650+0000 7f230be00640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a6ff1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009900 tx=0x7f22f8004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa59b53325aa35ab server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.650+0000 7f22f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.650+0000 7f22f6a00640 1 -- 172.21.15.175:0/424719605 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8035040 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.650+0000 7f22f6a00640 1 -- 172.21.15.175:0/424719605 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f22f8005450 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.650+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2304011ac0 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.650+0000 7f22f6a00640 1 -- 172.21.15.175:0/424719605 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f22f8005650 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 10 client.25757 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56099a7109d0 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f22f6a00640 1 -- 172.21.15.175:0/424719605 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304011ac0 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f22f6a00640 1 -- 172.21.15.175:0/424719605 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f22f8003b50 con 0x56099a6f7460 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 10 client.25757 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:26.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 20 client.25757 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a711de0 0x56099a7141f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 4 client.25757 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.651+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56099a61cc90 con 0x56099a711de0 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.652+0000 7f2311800640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a711de0 0x56099a7141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.652+0000 7f2311800640 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a711de0 0x56099a7141f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1627f0d0d7c6e7cb server_cookie=6e199ce1eeb0b882 in_seq=0 out_seq=0 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.652+0000 7f22f6a00640 10 client.25757 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f22f6a00640 1 -- 172.21.15.175:0/424719605 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56099a61cc90 con 0x56099a711de0 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f22f6a00640 10 client.25757 handle_command_reply: tid=0 2024-07-29T00:38:26.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 client.25757 shutdown 2024-07-29T00:38:26.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 msgr2=0x56099a6ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a6ff1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009900 tx=0x7f22f8004420 comp rx=0 tx=0).stop 2024-07-29T00:38:26.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 shutdown_connections 2024-07-29T00:38:26.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a711de0 0x56099a7141f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a701d20 0x56099a6ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a6ff1a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 --2- 172.21.15.175:0/424719605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a6f6b40 0x56099a6fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 >> 172.21.15.175:0/424719605 conn(0x56099a6f30d0 msgr2=0x56099a6f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f22f6a00640 0 client.25757 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.653+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 shutdown_connections 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.654+0000 7f231647d740 1 -- 172.21.15.175:0/424719605 wait complete. 2024-07-29T00:38:26.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.654+0000 7f231647d740 20 client.25757 trim_cache size 0 max 16384 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 Processor -- start 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 -- start start 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a700bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a719ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a71a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1b3c80 con 0x56099a701d20 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1b57c0 con 0x56099a6f7460 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.675+0000 7f231647d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1b4350 con 0x56099a6f6b40 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.676+0000 7f230be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a719ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.676+0000 7f230be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a719ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51416/0 (socket says 172.21.15.175:51416) 2024-07-29T00:38:26.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.676+0000 7f230be00640 1 -- 172.21.15.175:0/430668471 learned_addr learned my addr 172.21.15.175:0/430668471 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:26.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.676+0000 7f230be00640 1 -- 172.21.15.175:0/430668471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 msgr2=0x56099a700bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.676+0000 7f230be00640 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a700bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.676+0000 7f230be00640 1 -- 172.21.15.175:0/430668471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 msgr2=0x56099a71a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f2311800640 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a71a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f230be00640 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a71a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f230be00640 1 -- 172.21.15.175:0/430668471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099a6fd1b0 con 0x56099a6f7460 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f2311800640 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a71a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f230be00640 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a719ac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009850 tx=0x7f22f80054f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b438d3192c759647 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f2301600640 1 -- 172.21.15.175:0/430668471 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8035030 con 0x56099a6f7460 2024-07-29T00:38:26.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f2301600640 1 -- 172.21.15.175:0/430668471 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f8041070 con 0x56099a6f7460 2024-07-29T00:38:26.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.677+0000 7f2301600640 1 -- 172.21.15.175:0/430668471 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f803b3d0 con 0x56099a6f7460 2024-07-29T00:38:26.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 -- 172.21.15.175:0/430668471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 msgr2=0x56099a719ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a719ac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009850 tx=0x7f22f80054f0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 -- 172.21.15.175:0/430668471 shutdown_connections 2024-07-29T00:38:26.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a71a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a719ac0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 --2- 172.21.15.175:0/430668471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a700bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 -- 172.21.15.175:0/430668471 >> 172.21.15.175:0/430668471 conn(0x56099a6f30d0 msgr2=0x56099a7004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 -- 172.21.15.175:0/430668471 shutdown_connections 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.678+0000 7f231647d740 1 -- 172.21.15.175:0/430668471 wait complete. 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.679+0000 7f231647d740 1 -- start start 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.679+0000 7f231647d740 1 Processor -- start 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.679+0000 7f231647d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a7059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.679+0000 7f231647d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a705ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.679+0000 7f231647d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a706420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.679+0000 7f231647d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1b50f0 con 0x56099a701d20 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f231647d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1b4a20 con 0x56099a6f7460 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f231647d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1b1c60 con 0x56099a6f6b40 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f231647d740 10 client.0 _ll_register_callbacks cb 0x7f22f80040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a7059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f230be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a705ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f230be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a705ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51420/0 (socket says 172.21.15.175:51420) 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a7059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38054/0 (socket says 172.21.15.175:38054) 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f230be00640 1 -- 172.21.15.175:0/72138280 learned_addr learned my addr 172.21.15.175:0/72138280 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a706420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 -- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 msgr2=0x56099a705ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a705ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 -- 172.21.15.175:0/72138280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 msgr2=0x56099a706420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a706420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f2311800640 1 -- 172.21.15.175:0/72138280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56099a77ba90 con 0x56099a6f6b40 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.680+0000 7f230be00640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a705ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f2311800640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a7059a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f230400fc30 tx=0x7f230401b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66097714d6ad9a55 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f22f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f22f6a00640 1 -- 172.21.15.175:0/72138280 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304026040 con 0x56099a6f6b40 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f22f6a00640 1 -- 172.21.15.175:0/72138280 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f2304009960 con 0x56099a6f6b40 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f8041070 con 0x56099a6f6b40 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f22f6a00640 1 -- 172.21.15.175:0/72138280 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f230401b980 con 0x56099a6f6b40 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f231647d740 10 client.25750 fetch_fsmap learned FSMap version 29 2024-07-29T00:38:26.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56099a77cd00 con 0x56099a6f6b40 2024-07-29T00:38:26.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.681+0000 7f22f6a00640 1 -- 172.21.15.175:0/72138280 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f8041070 con 0x56099a6f6b40 2024-07-29T00:38:26.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f22f6a00640 1 -- 172.21.15.175:0/72138280 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f230402c040 con 0x56099a6f6b40 2024-07-29T00:38:26.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f231647d740 10 client.25750 fetch_fsmap finished waiting for FSMap version 29 2024-07-29T00:38:26.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f231647d740 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:26.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f231647d740 20 client.25750 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f231647d740 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a77d150 0x56099a77f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f231647d740 4 client.25750 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56099a669e10 con 0x56099a77d150 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.682+0000 7f230be00640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a77d150 0x56099a77f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.683+0000 7f230be00640 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a77d150 0x56099a77f540 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64fb5857cf09149 server_cookie=f3a836a08297b7f6 in_seq=0 out_seq=0 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.683+0000 7f22f6a00640 10 client.25750 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f22f6a00640 1 -- 172.21.15.175:0/72138280 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56099a669e10 con 0x56099a77d150 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f22f6a00640 10 client.25750 handle_command_reply: tid=0 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 client.25750 shutdown 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 msgr2=0x56099a7059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a7059a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f230400fc30 tx=0x7f230401b850 comp rx=0 tx=0).stop 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 shutdown_connections 2024-07-29T00:38:26.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56099a77d150 0x56099a77f540 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56099a701d20 0x56099a706420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56099a6f7460 0x56099a705ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 --2- 172.21.15.175:0/72138280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56099a6f6b40 0x56099a7059a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 >> 172.21.15.175:0/72138280 conn(0x56099a6f30d0 msgr2=0x56099a6f2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f22f6a00640 0 client.25750 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 shutdown_connections 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 1 -- 172.21.15.175:0/72138280 wait complete. 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.685+0000 7f231647d740 20 client.25750 trim_cache size 0 max 16384 2024-07-29T00:38:26.688 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "a26cbe7e-fe26-4fb3-80c7-fa43aa128dc1", 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f22f00a2d70 msgr2=0x7f22f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f22f00a2d70 0x7f22f00a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0d2fd0 tx=0x7f22fc0d2ea0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 msgr2=0x7f230c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 0x7f230c19e7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00d950 tx=0x7f22fc00de10 comp rx=0 tx=0).stop 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 shutdown_connections 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f22f00a2d70 0x7f22f00a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f230c10d220 0x7f230c1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c108af0 0x7f230c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 --2- 172.21.15.175:0/86670393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f230c108140 0x7f230c19e7b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.686+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 >> 172.21.15.175:0/86670393 conn(0x7f230c077890 msgr2=0x7f230c078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.687+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 shutdown_connections 2024-07-29T00:38:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:26.687+0000 7f2313800640 1 -- 172.21.15.175:0/86670393 wait complete. 2024-07-29T00:38:26.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 36099 12.48 MB/sec warmup 17 sec latency 9.622 ms 2024-07-29T00:38:27.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:38:27.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 38435 12.48 MB/sec warmup 18 sec latency 11.393 ms 2024-07-29T00:38:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:27 smithi181 ceph-mon[36706]: pgmap v61: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-07-29T00:38:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:27 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:38:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:27 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:27 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:27 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:27 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:38:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:27 smithi175 ceph-mon[32138]: pgmap v61: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-07-29T00:38:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:27 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:38:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:27 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:27 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:27 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:27 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:38:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:27 smithi204 ceph-mon[37368]: pgmap v61: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-07-29T00:38:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:27 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:38:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:27 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:28.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:27 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:28.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:27 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:28.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:27 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:38:28.167 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:28.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 -- 172.21.15.175:0/3567024596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c112720 msgr2=0x7f996c114b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 --2- 172.21.15.175:0/3567024596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c112720 0x7f996c114b10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9964009ca0 tx=0x7f996402a8c0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 -- 172.21.15.175:0/3567024596 shutdown_connections 2024-07-29T00:38:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 --2- 172.21.15.175:0/3567024596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c112720 0x7f996c114b10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 --2- 172.21.15.175:0/3567024596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 0x7f996c1121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 --2- 172.21.15.175:0/3567024596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f996c076410 0x7f996c0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 -- 172.21.15.175:0/3567024596 >> 172.21.15.175:0/3567024596 conn(0x7f996c0ff760 msgr2=0x7f996c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 -- 172.21.15.175:0/3567024596 shutdown_connections 2024-07-29T00:38:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.838+0000 7f9972a00640 1 -- 172.21.15.175:0/3567024596 wait complete. 2024-07-29T00:38:28.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 Processor -- start 2024-07-29T00:38:28.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 -- start start 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 0x7f996c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 0x7f996c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f996c112720 0x7f996c1a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1182d0 con 0x7f996c112720 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f996c118150 con 0x7f996c076d30 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.839+0000 7f9972a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f996c118450 con 0x7f996c076410 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 0x7f996c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f9970a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f996c112720 0x7f996c1a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 0x7f996c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 0x7f996c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53360/0 (socket says 172.21.15.175:53360) 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 -- 172.21.15.175:0/200458265 learned_addr learned my addr 172.21.15.175:0/200458265 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 -- 172.21.15.175:0/200458265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 msgr2=0x7f996c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 0x7f996c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 -- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f996c112720 msgr2=0x7f996c1a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f996c112720 0x7f996c1a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996be00640 1 -- 172.21.15.175:0/200458265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1a3700 con 0x7f996c076410 2024-07-29T00:38:28.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.840+0000 7f996b400640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 0x7f996c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:28.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.841+0000 7f996be00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 0x7f996c19e750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f995c00e9f0 tx=0x7f995c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:28.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.841+0000 7f9968c00640 1 -- 172.21.15.175:0/200458265 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00cd80 con 0x7f996c076410 2024-07-29T00:38:28.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.841+0000 7f9968c00640 1 -- 172.21.15.175:0/200458265 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f995c010910 con 0x7f996c076410 2024-07-29T00:38:28.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.841+0000 7f9968c00640 1 -- 172.21.15.175:0/200458265 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c018990 con 0x7f996c076410 2024-07-29T00:38:28.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.841+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1a39f0 con 0x7f996c076410 2024-07-29T00:38:28.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.841+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1ab2d0 con 0x7f996c076410 2024-07-29T00:38:28.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.844+0000 7f9968c00640 1 -- 172.21.15.175:0/200458265 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f995c010ac0 con 0x7f996c076410 2024-07-29T00:38:28.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 Processor -- start 2024-07-29T00:38:28.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 -- start start 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3f0250 0x560eac3fa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3fab10 0x560eac3fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560eac354ad0 con 0x560eac3f0250 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560eac2ecaf0 con 0x560eac3fab10 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.845+0000 7f997543e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560eac350a90 con 0x560eac3ef930 2024-07-29T00:38:28.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3f0250 0x560eac3fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3f0250 0x560eac3fa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46704/0 (socket says 172.21.15.175:46704) 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f9970a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996be00640 1 -- 172.21.15.175:0/2333459133 learned_addr learned my addr 172.21.15.175:0/2333459133 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996b400640 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3fab10 0x560eac3fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f9968c00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f99500a2d70 0x7f99500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996b400640 1 -- 172.21.15.175:0/2333459133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 msgr2=0x560eac3efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996b400640 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3efd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996b400640 1 -- 172.21.15.175:0/2333459133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3f0250 msgr2=0x560eac3fa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996b400640 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3f0250 0x560eac3fa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f9968c00640 1 -- 172.21.15.175:0/200458265 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f995c0cf740 con 0x7f996c076410 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f9970a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f99500a2d70 0x7f99500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.846+0000 7f996b400640 1 -- 172.21.15.175:0/2333459133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eac0c63c0 con 0x560eac3fab10 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.847+0000 7f996b400640 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3fab10 0x560eac3fcf00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9958009f80 tx=0x7f99580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d04244b81f9d01 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.847+0000 7f9970a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f99500a2d70 0x7f99500a5230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9964005e30 tx=0x7f996402a8c0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:28.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.847+0000 7f9961200640 1 -- 172.21.15.175:0/2333459133 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99580040f0 con 0x560eac3fab10 2024-07-29T00:38:28.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39379 12.16 MB/sec warmup 19 sec latency 217.075 ms 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.847+0000 7f9961200640 1 -- 172.21.15.175:0/2333459133 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958005500 con 0x560eac3fab10 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.847+0000 7f9961200640 1 -- 172.21.15.175:0/2333459133 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958004dc0 con 0x560eac3fab10 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 -- 172.21.15.175:0/2333459133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3fab10 msgr2=0x560eac3fcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3fab10 0x560eac3fcf00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9958009f80 tx=0x7f99580118d0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 -- 172.21.15.175:0/2333459133 shutdown_connections 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3fab10 0x560eac3fcf00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3f0250 0x560eac3fa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 --2- 172.21.15.175:0/2333459133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 -- 172.21.15.175:0/2333459133 >> 172.21.15.175:0/2333459133 conn(0x560eac3ebec0 msgr2=0x560eac3ec2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 -- 172.21.15.175:0/2333459133 shutdown_connections 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.848+0000 7f997543e740 1 -- 172.21.15.175:0/2333459133 wait complete. 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.849+0000 7f997543e740 1 -- start start 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.849+0000 7f997543e740 1 Processor -- start 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.849+0000 7f997543e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3f7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.849+0000 7f997543e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.849+0000 7f997543e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.849+0000 7f997543e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560eac3fd6e0 con 0x560eac3fab10 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f997543e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560eac30fbc0 con 0x560eac3f0250 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f997543e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560eac3fd860 con 0x560eac3ef930 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f997543e740 10 client.0 _ll_register_callbacks cb 0x560eac2fbd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46688/0 (socket says 172.21.15.175:46688) 2024-07-29T00:38:28.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996b400640 1 -- 172.21.15.175:0/946897233 learned_addr learned my addr 172.21.15.175:0/946897233 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:28.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996be00640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f9970a00640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996be00640 1 -- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 msgr2=0x560eac3f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996be00640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996be00640 1 -- 172.21.15.175:0/946897233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 msgr2=0x560eac3f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996be00640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996be00640 1 -- 172.21.15.175:0/946897233 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eac406b50 con 0x560eac3ef930 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.850+0000 7f996b400640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:28.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f996be00640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3f7950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x560eac3ecd20 tx=0x7f995c09b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88d5935ad85361b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f9955600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f9955600640 1 -- 172.21.15.175:0/946897233 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c09ff90 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f9955600640 1 -- 172.21.15.175:0/946897233 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f995c0cd090 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958005500 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f9955600640 1 -- 172.21.15.175:0/946897233 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f995c0cd290 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f997543e740 10 client.25762 fetch_fsmap learned FSMap version 30 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560eac4094b0 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f9955600640 1 -- 172.21.15.175:0/946897233 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958005500 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.851+0000 7f9955600640 1 -- 172.21.15.175:0/946897233 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f995c0cfc50 con 0x560eac3ef930 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f997543e740 10 client.25762 fetch_fsmap finished waiting for FSMap version 30 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f997543e740 10 client.25762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:28.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f997543e740 20 client.25762 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f997543e740 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac40a8c0 0x560eac40cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f997543e740 4 client.25762 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eac362040 con 0x560eac40a8c0 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.852+0000 7f996b400640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac40a8c0 0x560eac40cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.853+0000 7f996b400640 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac40a8c0 0x560eac40cd30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d02b4b9255779c server_cookie=d713a7323a409934 in_seq=0 out_seq=0 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.853+0000 7f9955600640 10 client.25762 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.853+0000 7f9955600640 1 -- 172.21.15.175:0/946897233 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560eac362040 con 0x560eac40a8c0 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.853+0000 7f9955600640 10 client.25762 handle_command_reply: tid=0 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.853+0000 7f997543e740 1 client.25762 shutdown 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 msgr2=0x560eac3f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3f7950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x560eac3ecd20 tx=0x7f995c09b7d0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 shutdown_connections 2024-07-29T00:38:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac40a8c0 0x560eac40cd30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 --2- 172.21.15.175:0/946897233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac3f7950 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 >> 172.21.15.175:0/946897233 conn(0x560eac3ebec0 msgr2=0x560eac3f1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f9955600640 0 client.25762 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 shutdown_connections 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 1 -- 172.21.15.175:0/946897233 wait complete. 2024-07-29T00:38:28.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.854+0000 7f997543e740 20 client.25762 trim_cache size 0 max 16384 2024-07-29T00:38:28.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 Processor -- start 2024-07-29T00:38:28.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 -- start start 2024-07-29T00:38:28.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3ef930 0x560eac3f2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3fab10 0x560eac3eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b3ce0 con 0x560eac3ef930 2024-07-29T00:38:28.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b5850 con 0x560eac3f0250 2024-07-29T00:38:28.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.873+0000 7f997543e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b43b0 con 0x560eac3fab10 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3ef930 0x560eac3f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3ef930 0x560eac3f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46714/0 (socket says 172.21.15.175:46714) 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996be00640 1 -- 172.21.15.175:0/2203538456 learned_addr learned my addr 172.21.15.175:0/2203538456 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f9970a00640 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3fab10 0x560eac3eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 -- 172.21.15.175:0/2203538456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3fab10 msgr2=0x560eac3eade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3fab10 0x560eac3eade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 -- 172.21.15.175:0/2203538456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3ef930 msgr2=0x560eac3f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3ef930 0x560eac3f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 -- 172.21.15.175:0/2203538456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eac3ea440 con 0x560eac3f0250 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f9970a00640 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3fab10 0x560eac3eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:28.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.874+0000 7f996b400640 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f8220 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x560eac3f8f00 tx=0x7f9958004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75444337d422d6b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:28.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f9957e00640 1 -- 172.21.15.175:0/2203538456 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958009b30 con 0x560eac3f0250 2024-07-29T00:38:28.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f9957e00640 1 -- 172.21.15.175:0/2203538456 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958003ad0 con 0x560eac3f0250 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f9957e00640 1 -- 172.21.15.175:0/2203538456 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995802b5a0 con 0x560eac3f0250 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 -- 172.21.15.175:0/2203538456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 msgr2=0x560eac3f8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f8220 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x560eac3f8f00 tx=0x7f9958004920 comp rx=0 tx=0).stop 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 -- 172.21.15.175:0/2203538456 shutdown_connections 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3fab10 0x560eac3eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3f8220 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 --2- 172.21.15.175:0/2203538456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3ef930 0x560eac3f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 -- 172.21.15.175:0/2203538456 >> 172.21.15.175:0/2203538456 conn(0x560eac3ebec0 msgr2=0x560eac3ea6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.875+0000 7f997543e740 1 -- 172.21.15.175:0/2203538456 shutdown_connections 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.876+0000 7f997543e740 1 -- 172.21.15.175:0/2203538456 wait complete. 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.876+0000 7f997543e740 1 -- start start 2024-07-29T00:38:28.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.876+0000 7f997543e740 1 Processor -- start 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac403280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b5150 con 0x560eac3fab10 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b4a80 con 0x560eac3f0250 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b1cc0 con 0x560eac3ef930 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f997543e740 10 client.0 _ll_register_callbacks cb 0x7f995c0ccdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f9970a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46698/0 (socket says 172.21.15.175:46698) 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 -- 172.21.15.175:0/3657953868 learned_addr learned my addr 172.21.15.175:0/3657953868 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996be00640 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac403280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 -- 172.21.15.175:0/3657953868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 msgr2=0x560eac403280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac403280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 -- 172.21.15.175:0/3657953868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 msgr2=0x560eac3feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:28.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3feb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.877+0000 7f996b400640 1 -- 172.21.15.175:0/3657953868 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eac3ff340 con 0x560eac3f0250 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f996b400640 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3fe5b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f995802e040 tx=0x7f9958011f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a07f4fd98ab27c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f9955600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f9955600640 1 -- 172.21.15.175:0/3657953868 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958030040 con 0x560eac3f0250 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f9955600640 1 -- 172.21.15.175:0/3657953868 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f9958016040 con 0x560eac3f0250 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958003ad0 con 0x560eac3f0250 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f9955600640 1 -- 172.21.15.175:0/3657953868 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9958009b30 con 0x560eac3f0250 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f997543e740 10 client.25781 fetch_fsmap learned FSMap version 30 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.878+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560eac475670 con 0x560eac3f0250 2024-07-29T00:38:28.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f9955600640 1 -- 172.21.15.175:0/3657953868 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958003ad0 con 0x560eac3f0250 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f9955600640 1 -- 172.21.15.175:0/3657953868 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f995800f930 con 0x560eac3f0250 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f997543e740 10 client.25781 fetch_fsmap finished waiting for FSMap version 30 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f997543e740 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f997543e740 20 client.25781 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f997543e740 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac475ba0 0x560eac477fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f997543e740 4 client.25781 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560eac35c550 con 0x560eac475ba0 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.879+0000 7f996be00640 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac475ba0 0x560eac477fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f996be00640 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac475ba0 0x560eac477fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abcb989857f2e2fe server_cookie=3f31e5fccd1db641 in_seq=0 out_seq=0 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f9955600640 10 client.25781 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f9955600640 1 -- 172.21.15.175:0/3657953868 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560eac35c550 con 0x560eac475ba0 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f9955600640 10 client.25781 handle_command_reply: tid=0 2024-07-29T00:38:28.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 client.25781 shutdown 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 msgr2=0x560eac3fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3fe5b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f995802e040 tx=0x7f9958011f10 comp rx=0 tx=0).stop 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 shutdown_connections 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560eac475ba0 0x560eac477fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eac3fab10 0x560eac3feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560eac3f0250 0x560eac3fe5b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 --2- 172.21.15.175:0/3657953868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560eac3ef930 0x560eac403280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f9955600640 0 client.25781 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 >> 172.21.15.175:0/3657953868 conn(0x560eac3ebec0 msgr2=0x560eac405930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 shutdown_connections 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 1 -- 172.21.15.175:0/3657953868 wait complete. 2024-07-29T00:38:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.880+0000 7f997543e740 20 client.25781 trim_cache size 0 max 16384 2024-07-29T00:38:28.885 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:38:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:38:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:38:28.885 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f99500a2d70 msgr2=0x7f99500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f99500a2d70 0x7f99500a5230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9964005e30 tx=0x7f996402a8c0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 msgr2=0x7f996c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 0x7f996c19e750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f995c00e9f0 tx=0x7f995c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 shutdown_connections 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f99500a2d70 0x7f99500a5230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f996c112720 0x7f996c1a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f996c076d30 0x7f996c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 --2- 172.21.15.175:0/200458265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f996c076410 0x7f996c19e750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:28.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 >> 172.21.15.175:0/200458265 conn(0x7f996c0ff760 msgr2=0x7f996c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:28.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.881+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 shutdown_connections 2024-07-29T00:38:28.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:28.882+0000 7f9972a00640 1 -- 172.21.15.175:0/200458265 wait complete. 2024-07-29T00:38:28.896 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:28 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:38:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:28 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:38:29.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:28 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:38:29.775 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:29 smithi175 ceph-mon[32138]: pgmap v62: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-07-29T00:38:29.839 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:38:29.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:38:29.847 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39379 11.55 MB/sec warmup 20 sec latency 1217.147 ms 2024-07-29T00:38:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:29 smithi181 ceph-mon[36706]: pgmap v62: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-07-29T00:38:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:29 smithi204 ceph-mon[37368]: pgmap v62: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-07-29T00:38:30.299 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:30.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39427 11.01 MB/sec warmup 21 sec latency 2035.385 ms 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 -- 172.21.15.175:0/4140740373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c341005d0 msgr2=0x7f0c3410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 --2- 172.21.15.175:0/4140740373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c341005d0 0x7f0c3410e610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009ca0 tx=0x7f0c2c02a8c0 comp rx=0 tx=0).stop 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 -- 172.21.15.175:0/4140740373 shutdown_connections 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 --2- 172.21.15.175:0/4140740373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c341005d0 0x7f0c3410e610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 --2- 172.21.15.175:0/4140740373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c340ffc30 0x7f0c34100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 --2- 172.21.15.175:0/4140740373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c34107740 0x7f0c34107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 -- 172.21.15.175:0/4140740373 >> 172.21.15.175:0/4140740373 conn(0x7f0c34076d60 msgr2=0x7f0c340fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 -- 172.21.15.175:0/4140740373 shutdown_connections 2024-07-29T00:38:30.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.958+0000 7f0c3a600640 1 -- 172.21.15.175:0/4140740373 wait complete. 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.959+0000 7f0c3a600640 1 Processor -- start 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.959+0000 7f0c3a600640 1 -- start start 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c3a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340ffc30 0x7f0c3419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c3a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 0x7f0c3419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c3a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c34107740 0x7f0c3419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c3a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34111b90 con 0x7f0c341005d0 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c3a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34111a10 con 0x7f0c34107740 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c3a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34111d10 con 0x7f0c340ffc30 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 0x7f0c3419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 0x7f0c3419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46748/0 (socket says 172.21.15.175:46748) 2024-07-29T00:38:30.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c2be00640 1 -- 172.21.15.175:0/1287165625 learned_addr learned my addr 172.21.15.175:0/1287165625 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c39200640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340ffc30 0x7f0c3419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.960+0000 7f0c39c00640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c34107740 0x7f0c3419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c39200640 1 -- 172.21.15.175:0/1287165625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c34107740 msgr2=0x7f0c3419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c39200640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c34107740 0x7f0c3419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c39200640 1 -- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 msgr2=0x7f0c3419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c39200640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 0x7f0c3419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:30.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c39200640 1 -- 172.21.15.175:0/1287165625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3419f1e0 con 0x7f0c340ffc30 2024-07-29T00:38:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c2be00640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 0x7f0c3419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c39200640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340ffc30 0x7f0c3419a2d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9f0 tx=0x7f0c2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400ccc0 con 0x7f0c340ffc30 2024-07-29T00:38:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c24010910 con 0x7f0c340ffc30 2024-07-29T00:38:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3419f4d0 con 0x7f0c340ffc30 2024-07-29T00:38:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.961+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24018960 con 0x7f0c340ffc30 2024-07-29T00:38:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.962+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341a6db0 con 0x7f0c340ffc30 2024-07-29T00:38:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.963+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c34100090 con 0x7f0c340ffc30 2024-07-29T00:38:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.965+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0c24010ac0 con 0x7f0c340ffc30 2024-07-29T00:38:30.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.966+0000 7f0c32000640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0c180a2d70 0x7f0c180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:30.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.966+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0c240cfbe0 con 0x7f0c340ffc30 2024-07-29T00:38:30.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.967+0000 7f0c2be00640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0c180a2d70 0x7f0c180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:30.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.967+0000 7f0c2be00640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0c180a2d70 0x7f0c180a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c34102da0 tx=0x7f0c1c005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:30.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:30.968+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0c2409c130 con 0x7f0c340ffc30 2024-07-29T00:38:31.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.195+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c341a6f80 con 0x7f0c340ffc30 2024-07-29T00:38:31.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.196+0000 7f0c32000640 1 -- 172.21.15.175:0/1287165625 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11995 (secure 0 0 0) 0x7f0c240a0430 con 0x7f0c340ffc30 2024-07-29T00:38:31.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:38:31.197 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":30,"btime":"2024-07-29T00:38:27:624704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:27.433153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:38:31.197 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 30 2024-07-29T00:38:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0c180a2d70 msgr2=0x7f0c180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0c180a2d70 0x7f0c180a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c34102da0 tx=0x7f0c1c005f90 comp rx=0 tx=0).stop 2024-07-29T00:38:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340ffc30 msgr2=0x7f0c3419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340ffc30 0x7f0c3419a2d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9f0 tx=0x7f0c2400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:38:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 shutdown_connections 2024-07-29T00:38:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c34107740 0x7f0c3419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:31.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0c180a2d70 0x7f0c180a5230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:31.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c341005d0 0x7f0c3419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:31.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 --2- 172.21.15.175:0/1287165625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340ffc30 0x7f0c3419a2d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:31.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 >> 172.21.15.175:0/1287165625 conn(0x7f0c34076d60 msgr2=0x7f0c34104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:31.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 shutdown_connections 2024-07-29T00:38:31.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:31.200+0000 7f0c3a600640 1 -- 172.21.15.175:0/1287165625 wait complete. 2024-07-29T00:38:31.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39427 10.51 MB/sec warmup 22 sec latency 1111.325 ms 2024-07-29T00:38:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:31 smithi181 ceph-mon[36706]: pgmap v63: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-07-29T00:38:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:31 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1287165625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:32.026 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:31 smithi175 ceph-mon[32138]: pgmap v63: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-07-29T00:38:32.026 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:31 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1287165625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:31 smithi204 ceph-mon[37368]: pgmap v63: 97 pgs: 97 active+clean; 18 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-07-29T00:38:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:31 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1287165625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:38:32.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:38:32.546 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:32.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39427 10.06 MB/sec warmup 23 sec latency 2111.413 ms 2024-07-29T00:38:33.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 -- 172.21.15.175:0/2910489181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec102a80 msgr2=0x7fa0ec102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2910489181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec102a80 0x7fa0ec102e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009930 tx=0x7fa0e002a420 comp rx=0 tx=0).stop 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 -- 172.21.15.175:0/2910489181 shutdown_connections 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2910489181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec107f80 0x7fa0ec10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2910489181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0ec103430 0x7fa0ec107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2910489181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec102a80 0x7fa0ec102e60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 -- 172.21.15.175:0/2910489181 >> 172.21.15.175:0/2910489181 conn(0x7fa0ec0fc4c0 msgr2=0x7fa0ec0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 -- 172.21.15.175:0/2910489181 shutdown_connections 2024-07-29T00:38:33.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.212+0000 7fa0f4400640 1 -- 172.21.15.175:0/2910489181 wait complete. 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.213+0000 7fa0f4400640 1 Processor -- start 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.213+0000 7fa0f4400640 1 -- start start 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f4400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0ec102a80 0x7fa0ec19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f4400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec103430 0x7fa0ec19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f4400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 0x7fa0ec19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f4400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec111dd0 con 0x7fa0ec107f80 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f4400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec111c50 con 0x7fa0ec103430 2024-07-29T00:38:33.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f4400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec111f50 con 0x7fa0ec102a80 2024-07-29T00:38:33.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 0x7fa0ec19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 0x7fa0ec19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46764/0 (socket says 172.21.15.175:46764) 2024-07-29T00:38:33.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f2400640 1 -- 172.21.15.175:0/2336134541 learned_addr learned my addr 172.21.15.175:0/2336134541 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f1000640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec103430 0x7fa0ec19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0ec102a80 0x7fa0ec19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f1000640 1 -- 172.21.15.175:0/2336134541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0ec102a80 msgr2=0x7fa0ec19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.214+0000 7fa0f1000640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0ec102a80 0x7fa0ec19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f1000640 1 -- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 msgr2=0x7fa0ec19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f1000640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 0x7fa0ec19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f1000640 1 -- 172.21.15.175:0/2336134541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec19f460 con 0x7fa0ec103430 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f2400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 0x7fa0ec19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f1000640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec103430 0x7fa0ec19a9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800e9c0 tx=0x7fa0d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0ea000640 1 -- 172.21.15.175:0/2336134541 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800ccc0 con 0x7fa0ec103430 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0ea000640 1 -- 172.21.15.175:0/2336134541 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d8005020 con 0x7fa0ec103430 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0ea000640 1 -- 172.21.15.175:0/2336134541 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004a00 con 0x7fa0ec103430 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec19f750 con 0x7fa0ec103430 2024-07-29T00:38:33.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.215+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1a7030 con 0x7fa0ec103430 2024-07-29T00:38:33.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 Processor -- start 2024-07-29T00:38:33.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 -- start start 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd043440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd043dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd050b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0ea000640 1 -- 172.21.15.175:0/2336134541 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa0d80051d0 con 0x7fa0ec103430 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b1bcfa9ab0 con 0x55b1bd04e720 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b1bcf41a80 con 0x55b1bd044ef0 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.219+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1bcfa5a70 con 0x55b1bd043980 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd043440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd043440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46736/0 (socket says 172.21.15.175:46736) 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 -- 172.21.15.175:0/986834238 learned_addr learned my addr 172.21.15.175:0/986834238 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:33.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f1000640 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd050b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd043dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 -- 172.21.15.175:0/986834238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 msgr2=0x55b1bd043dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd043dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 -- 172.21.15.175:0/986834238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 msgr2=0x55b1bd050b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd050b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.220+0000 7fa0f2400640 1 -- 172.21.15.175:0/986834238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1bcd1b3c0 con 0x55b1bd044ef0 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0f2400640 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd043440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4010810 tx=0x7fa0e4010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7663d090cc963e99 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0dea00640 1 -- 172.21.15.175:0/986834238 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400eb30 con 0x55b1bd044ef0 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0dea00640 1 -- 172.21.15.175:0/986834238 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4015070 con 0x55b1bd044ef0 2024-07-29T00:38:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0dea00640 1 -- 172.21.15.175:0/986834238 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40116a0 con 0x55b1bd044ef0 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0ea000640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa0d40a2d70 0x7fa0d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0ea000640 1 -- 172.21.15.175:0/2336134541 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa0d80cf3a0 con 0x7fa0ec103430 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.221+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa0d40a2d70 0x7fa0d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/986834238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 msgr2=0x55b1bd043440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd043440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4010810 tx=0x7fa0e4010cd0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/986834238 shutdown_connections 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd050b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd043dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/986834238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd043440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/986834238 >> 172.21.15.175:0/986834238 conn(0x55b1bd00c950 msgr2=0x55b1bd040010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/986834238 shutdown_connections 2024-07-29T00:38:33.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/986834238 wait complete. 2024-07-29T00:38:33.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa0d40a2d70 0x7fa0d40a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0006010 tx=0x7fa0e0000bc0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:33.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.222+0000 7fa0f6e1a740 1 -- start start 2024-07-29T00:38:33.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 Processor -- start 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd043980 0x55b1bd04b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd044ef0 0x55b1bd04bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd04c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b1bd0514c0 con 0x55b1bd044ef0 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b1bcf64c10 con 0x55b1bd043980 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b1bd051640 con 0x55b1bd04e720 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.223+0000 7fa0f6e1a740 10 client.0 _ll_register_callbacks cb 0x55b1bcf2da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd04c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd044ef0 0x55b1bd04bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd043980 0x55b1bd04b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd04c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53418/0 (socket says 172.21.15.175:53418) 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f2400640 1 -- 172.21.15.175:0/3819526425 learned_addr learned my addr 172.21.15.175:0/3819526425 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:33.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 -- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 msgr2=0x55b1bd04c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd04c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 -- 172.21.15.175:0/3819526425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd044ef0 msgr2=0x55b1bd04bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd044ef0 0x55b1bd04bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 -- 172.21.15.175:0/3819526425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1bcf681e0 con 0x55b1bd043980 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f2400640 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd04c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd043980 0x55b1bd04b6e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa0e002a940 tx=0x7fa0e0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ae0da3dd4c2f95 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.224+0000 7fa0d3400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0d3400640 1 -- 172.21.15.175:0/3819526425 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e004e070 con 0x55b1bd043980 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e4015070 con 0x55b1bd043980 2024-07-29T00:38:33.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0d3400640 1 -- 172.21.15.175:0/3819526425 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa0e0004b90 con 0x55b1bd043980 2024-07-29T00:38:33.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0d3400640 1 -- 172.21.15.175:0/3819526425 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0e005a050 con 0x55b1bd043980 2024-07-29T00:38:33.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0f6e1a740 10 client.25799 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:33.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b1bd05d2c0 con 0x55b1bd043980 2024-07-29T00:38:33.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.225+0000 7fa0d3400640 1 -- 172.21.15.175:0/3819526425 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4015070 con 0x55b1bd043980 2024-07-29T00:38:33.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0d3400640 1 -- 172.21.15.175:0/3819526425 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa0e00037b0 con 0x55b1bd043980 2024-07-29T00:38:33.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f6e1a740 10 client.25799 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:33.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f6e1a740 10 client.25799 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f6e1a740 20 client.25799 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd05e6d0 0x55b1bd060b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f6e1a740 4 client.25799 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1bcfb7020 con 0x55b1bd05e6d0 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.226+0000 7fa0f1000640 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd05e6d0 0x55b1bd060b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.227+0000 7fa0f1000640 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd05e6d0 0x55b1bd060b00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8bc8d26aabbbad36 server_cookie=7237edd3a2dfb70a in_seq=0 out_seq=0 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.227+0000 7fa0d3400640 10 client.25799 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0d3400640 1 -- 172.21.15.175:0/3819526425 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b1bcfb7020 con 0x55b1bd05e6d0 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0d3400640 10 client.25799 handle_command_reply: tid=0 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 client.25799 shutdown 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd043980 msgr2=0x55b1bd04b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd043980 0x55b1bd04b6e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa0e002a940 tx=0x7fa0e0009260 comp rx=0 tx=0).stop 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 shutdown_connections 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd04c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd044ef0 0x55b1bd04bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd05e6d0 0x55b1bd060b00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/3819526425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd043980 0x55b1bd04b6e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 >> 172.21.15.175:0/3819526425 conn(0x55b1bd00c950 msgr2=0x55b1bd045d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0d3400640 0 client.25799 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.228+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 shutdown_connections 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.229+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/3819526425 wait complete. 2024-07-29T00:38:33.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.229+0000 7fa0f6e1a740 20 client.25799 trim_cache size 0 max 16384 2024-07-29T00:38:33.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 Processor -- start 2024-07-29T00:38:33.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 -- start start 2024-07-29T00:38:33.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd04bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd0668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1af8b0 con 0x55b1bd04e720 2024-07-29T00:38:33.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1b14b0 con 0x55b1bd044ef0 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.243+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1aff80 con 0x55b1bd043980 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd04bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd0668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd04bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53434/0 (socket says 172.21.15.175:53434) 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1562836492 learned_addr learned my addr 172.21.15.175:0/1562836492 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1000640 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1562836492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 msgr2=0x55b1bd0663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1562836492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 msgr2=0x55b1bd0668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd0668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1562836492 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1bd040d10 con 0x55b1bd043980 2024-07-29T00:38:33.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f2400640 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd0668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.244+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd04bdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0e002a460 tx=0x7fa0e0003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a00abdfdc5bc6f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.245+0000 7fa0de000640 1 -- 172.21.15.175:0/1562836492 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e004e070 con 0x55b1bd043980 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.245+0000 7fa0de000640 1 -- 172.21.15.175:0/1562836492 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e0063070 con 0x55b1bd043980 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.245+0000 7fa0de000640 1 -- 172.21.15.175:0/1562836492 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0035040 con 0x55b1bd043980 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.245+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1562836492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 msgr2=0x55b1bd04bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.245+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd04bdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0e002a460 tx=0x7fa0e0003f80 comp rx=0 tx=0).stop 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1562836492 shutdown_connections 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd04e720 0x55b1bd0668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1562836492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd043980 0x55b1bd04bdb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1562836492 >> 172.21.15.175:0/1562836492 conn(0x55b1bd00c950 msgr2=0x55b1bd04b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1562836492 shutdown_connections 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1562836492 wait complete. 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 -- start start 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.246+0000 7fa0f6e1a740 1 Processor -- start 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 0x55b1bd0c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd051eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1b0d20 con 0x55b1bd043980 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1b0650 con 0x55b1bd044ef0 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1ad890 con 0x55b1bd04e720 2024-07-29T00:38:33.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f6e1a740 10 client.0 _ll_register_callbacks cb 0x7fa0e0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:33.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 0x55b1bd0c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 0x55b1bd0c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46820/0 (socket says 172.21.15.175:46820) 2024-07-29T00:38:33.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1116180934 learned_addr learned my addr 172.21.15.175:0/1116180934 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:33.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f2400640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd051eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.247+0000 7fa0f1000640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 msgr2=0x55b1bd051eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd051eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1116180934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 msgr2=0x55b1bd0c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f1a00640 1 -- 172.21.15.175:0/1116180934 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1bd0525a0 con 0x55b1bd043980 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f1a00640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 0x55b1bd0c1660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0031550 tx=0x7fa0e0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aab27b2aaee87d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0d3400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0d3400640 1 -- 172.21.15.175:0/1116180934 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e006b020 con 0x55b1bd043980 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0d3400640 1 -- 172.21.15.175:0/1116180934 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa0e00035f0 con 0x55b1bd043980 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.248+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e0063070 con 0x55b1bd043980 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0d3400640 1 -- 172.21.15.175:0/1116180934 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0e0076050 con 0x55b1bd043980 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 10 client.15696 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b1bd0c8ec0 con 0x55b1bd043980 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0d3400640 1 -- 172.21.15.175:0/1116180934 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e0063070 con 0x55b1bd043980 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0d3400640 1 -- 172.21.15.175:0/1116180934 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa0e004a6b0 con 0x55b1bd043980 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 10 client.15696 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 10 client.15696 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 20 client.15696 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd0c94c0 0x55b1bd0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 4 client.15696 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.249+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1bcfb1530 con 0x55b1bd0c94c0 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.250+0000 7fa0f1000640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd0c94c0 0x55b1bd0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.250+0000 7fa0d3400640 10 client.15696 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.250+0000 7fa0f1000640 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd0c94c0 0x55b1bd0cb8e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=458fc506ad629554 server_cookie=ff7c3af26c16f9cc in_seq=0 out_seq=0 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.250+0000 7fa0d3400640 1 -- 172.21.15.175:0/1116180934 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1bcfb1530 con 0x55b1bd0c94c0 2024-07-29T00:38:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.250+0000 7fa0d3400640 10 client.15696 handle_command_reply: tid=0 2024-07-29T00:38:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.250+0000 7fa0f6e1a740 1 client.15696 shutdown 2024-07-29T00:38:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 msgr2=0x55b1bd0c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 0x55b1bd0c1660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0031550 tx=0x7fa0e0031400 comp rx=0 tx=0).stop 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 shutdown_connections 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1bd04e720 0x55b1bd051eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55b1bd0c94c0 0x55b1bd0cb8e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b1bd044ef0 0x55b1bd0c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 --2- 172.21.15.175:0/1116180934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b1bd043980 0x55b1bd0c1660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 >> 172.21.15.175:0/1116180934 conn(0x55b1bd00c950 msgr2=0x55b1bd04bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0d3400640 0 client.15696 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 shutdown_connections 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 1 -- 172.21.15.175:0/1116180934 wait complete. 2024-07-29T00:38:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.251+0000 7fa0f6e1a740 20 client.15696 trim_cache size 0 max 16384 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa0d40a2d70 msgr2=0x7fa0d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa0d40a2d70 0x7fa0d40a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0006010 tx=0x7fa0e0000bc0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec103430 msgr2=0x7fa0ec19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec103430 0x7fa0ec19a9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800e9c0 tx=0x7fa0d800ee80 comp rx=0 tx=0).stop 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 shutdown_connections 2024-07-29T00:38:33.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa0d40a2d70 0x7fa0d40a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ec107f80 0x7fa0ec19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec103430 0x7fa0ec19a9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 --2- 172.21.15.175:0/2336134541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0ec102a80 0x7fa0ec19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:33.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 >> 172.21.15.175:0/2336134541 conn(0x7fa0ec0fc4c0 msgr2=0x7fa0ec100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:33.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 shutdown_connections 2024-07-29T00:38:33.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:33.252+0000 7fa0f4400640 1 -- 172.21.15.175:0/2336134541 wait complete. 2024-07-29T00:38:33.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39427 9.64 MB/sec warmup 24 sec latency 3111.552 ms 2024-07-29T00:38:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:33 smithi181 ceph-mon[36706]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-07-29T00:38:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:33 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:38:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:33 smithi175 ceph-mon[32138]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-07-29T00:38:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:33 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:38:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:33 smithi204 ceph-mon[37368]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-07-29T00:38:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:33 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:38:34.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:38:34.776 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:34.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39427 9.25 MB/sec warmup 25 sec latency 4111.670 ms 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 -- 172.21.15.175:0/662372746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 msgr2=0x7f26fc074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 --2- 172.21.15.175:0/662372746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 0x7f26fc074660 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009ca0 tx=0x7f26ec02a950 comp rx=0 tx=0).stop 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 -- 172.21.15.175:0/662372746 shutdown_connections 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 --2- 172.21.15.175:0/662372746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26fc1126d0 0x7f26fc114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 --2- 172.21.15.175:0/662372746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc112190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 --2- 172.21.15.175:0/662372746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 0x7f26fc074660 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 -- 172.21.15.175:0/662372746 >> 172.21.15.175:0/662372746 conn(0x7f26fc0fd400 msgr2=0x7f26fc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 -- 172.21.15.175:0/662372746 shutdown_connections 2024-07-29T00:38:35.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.430+0000 7f2701000640 1 -- 172.21.15.175:0/662372746 wait complete. 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.431+0000 7f2701000640 1 Processor -- start 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 -- start start 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 0x7f26fc19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26fc1126d0 0x7f26fc1a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc118090 con 0x7f26fc1126d0 2024-07-29T00:38:35.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc117f10 con 0x7f26fc074280 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f2701000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc118210 con 0x7f26fc074ba0 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f26f2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f26fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 0x7f26fc19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f26f2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53452/0 (socket says 172.21.15.175:53452) 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.432+0000 7f26f2a00640 1 -- 172.21.15.175:0/712255309 learned_addr learned my addr 172.21.15.175:0/712255309 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26fbe00640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26fc1126d0 0x7f26fc1a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f2a00640 1 -- 172.21.15.175:0/712255309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 msgr2=0x7f26fc19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f2a00640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 0x7f26fc19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f2a00640 1 -- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26fc1126d0 msgr2=0x7f26fc1a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f2a00640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26fc1126d0 0x7f26fc1a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f2a00640 1 -- 172.21.15.175:0/712255309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc1a35a0 con 0x7f26fc074ba0 2024-07-29T00:38:35.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f2a00640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc19eb30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e9f0 tx=0x7f26e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:35.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f8c00640 1 -- 172.21.15.175:0/712255309 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e400cd80 con 0x7f26fc074ba0 2024-07-29T00:38:35.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.433+0000 7f26f8c00640 1 -- 172.21.15.175:0/712255309 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e4005500 con 0x7f26fc074ba0 2024-07-29T00:38:35.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.434+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc1a3890 con 0x7f26fc074ba0 2024-07-29T00:38:35.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.434+0000 7f26f8c00640 1 -- 172.21.15.175:0/712255309 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4010690 con 0x7f26fc074ba0 2024-07-29T00:38:35.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.434+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1ab170 con 0x7f26fc074ba0 2024-07-29T00:38:35.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 Processor -- start 2024-07-29T00:38:35.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f26f8c00640 1 -- 172.21.15.175:0/712255309 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f26e40046c0 con 0x7f26fc074ba0 2024-07-29T00:38:35.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 -- start start 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bc0e0 0x5626110ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110bafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562611020ab0 con 0x5626110bab70 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562610fb8a80 con 0x5626110c5910 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.437+0000 7f2705278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56261101ca70 con 0x5626110bc0e0 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110bafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bc0e0 0x5626110ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110bafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46850/0 (socket says 172.21.15.175:46850) 2024-07-29T00:38:35.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fb400640 1 -- 172.21.15.175:0/1777793555 learned_addr learned my addr 172.21.15.175:0/1777793555 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:35.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26f2a00640 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fbe00640 1 -- 172.21.15.175:0/1777793555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 msgr2=0x5626110c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fbe00640 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fbe00640 1 -- 172.21.15.175:0/1777793555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 msgr2=0x5626110bafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fbe00640 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110bafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.438+0000 7f26fbe00640 1 -- 172.21.15.175:0/1777793555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562610d923c0 con 0x5626110bc0e0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26fbe00640 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bc0e0 0x5626110ba630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009b50 tx=0x7f26f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe15936a7ce7572 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26f8c00640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26e00a2cf0 0x7f26e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26f0c00640 1 -- 172.21.15.175:0/1777793555 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4004050 con 0x5626110bc0e0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26f0c00640 1 -- 172.21.15.175:0/1777793555 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f4005500 con 0x5626110bc0e0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26f0c00640 1 -- 172.21.15.175:0/1777793555 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4004cc0 con 0x5626110bc0e0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26f8c00640 1 -- 172.21.15.175:0/712255309 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f26e40cf8b0 con 0x7f26fc074ba0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.439+0000 7f26fb400640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26e00a2cf0 0x7f26e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 -- 172.21.15.175:0/1777793555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bc0e0 msgr2=0x5626110ba630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bc0e0 0x5626110ba630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009b50 tx=0x7f26f4011810 comp rx=0 tx=0).stop 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 -- 172.21.15.175:0/1777793555 shutdown_connections 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110bafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 --2- 172.21.15.175:0/1777793555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bc0e0 0x5626110ba630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 -- 172.21.15.175:0/1777793555 >> 172.21.15.175:0/1777793555 conn(0x562611083950 msgr2=0x5626110b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:35.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f26fb400640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26e00a2cf0 0x7f26e00a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5626110b8060 tx=0x7f26ec004a60 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:35.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 -- 172.21.15.175:0/1777793555 shutdown_connections 2024-07-29T00:38:35.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.440+0000 7f2705278740 1 -- 172.21.15.175:0/1777793555 wait complete. 2024-07-29T00:38:35.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.441+0000 7f2705278740 1 -- start start 2024-07-29T00:38:35.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.441+0000 7f2705278740 1 Processor -- start 2024-07-29T00:38:35.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.441+0000 7f2705278740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.441+0000 7f2705278740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bc0e0 0x5626110c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.441+0000 7f2705278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f2705278740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5626110c86b0 con 0x5626110bab70 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f2705278740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562610fdbc10 con 0x5626110bc0e0 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f2705278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5626110c8830 con 0x5626110c5910 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f2705278740 10 client.0 _ll_register_callbacks cb 0x562610fa4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26fb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26fb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46862/0 (socket says 172.21.15.175:46862) 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26fb400640 1 -- 172.21.15.175:0/1358908909 learned_addr learned my addr 172.21.15.175:0/1358908909 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26f2a00640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bc0e0 0x5626110c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26fbe00640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26f2a00640 1 -- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 msgr2=0x5626110c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26f2a00640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26f2a00640 1 -- 172.21.15.175:0/1358908909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 msgr2=0x5626110c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26f2a00640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26f2a00640 1 -- 172.21.15.175:0/1358908909 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562610fdf1e0 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.442+0000 7f26fb400640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f26f2a00640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bc0e0 0x5626110c2d60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26e40ccf50 tx=0x7f26e4004ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a587cf3fd8521dd6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f26e9600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f26e9600640 1 -- 172.21.15.175:0/1358908909 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e409b6b0 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f26e9600640 1 -- 172.21.15.175:0/1358908909 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f26e40a4730 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f4005500 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f26e9600640 1 -- 172.21.15.175:0/1358908909 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f26e40cd410 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f2705278740 10 client.25817 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5626110d4430 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.443+0000 7f26e9600640 1 -- 172.21.15.175:0/1358908909 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f4005500 con 0x5626110bc0e0 2024-07-29T00:38:35.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f26e9600640 1 -- 172.21.15.175:0/1358908909 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f26e409b2c0 con 0x5626110bc0e0 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f2705278740 10 client.25817 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f2705278740 10 client.25817 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f2705278740 20 client.25817 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f2705278740 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5626110d5840 0x5626110d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f2705278740 4 client.25817 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56261102e020 con 0x5626110d5840 2024-07-29T00:38:35.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.444+0000 7f26fb400640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5626110d5840 0x5626110d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.445+0000 7f26fb400640 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5626110d5840 0x5626110d7c30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28d25bd1f1cbade0 server_cookie=dc23f13686d60f8d in_seq=0 out_seq=0 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.445+0000 7f26e9600640 10 client.25817 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f26e9600640 1 -- 172.21.15.175:0/1358908909 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56261102e020 con 0x5626110d5840 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f26e9600640 10 client.25817 handle_command_reply: tid=0 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 client.25817 shutdown 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bc0e0 msgr2=0x5626110c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bc0e0 0x5626110c2d60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26e40ccf50 tx=0x7f26e4004ae0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 shutdown_connections 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5626110d5840 0x5626110d7c30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bc0e0 0x5626110c2d60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 --2- 172.21.15.175:0/1358908909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bab70 0x5626110c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 >> 172.21.15.175:0/1358908909 conn(0x562611083950 msgr2=0x5626110bcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f26e9600640 0 client.25817 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 shutdown_connections 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.446+0000 7f2705278740 1 -- 172.21.15.175:0/1358908909 wait complete. 2024-07-29T00:38:35.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.447+0000 7f2705278740 20 client.25817 trim_cache size 0 max 16384 2024-07-29T00:38:35.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 Processor -- start 2024-07-29T00:38:35.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 -- start start 2024-07-29T00:38:35.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 0x5626110dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110dda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1b3af0 con 0x5626110bc0e0 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1b56f0 con 0x5626110bab70 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f2705278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1b41c0 con 0x5626110c5910 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f26fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 0x5626110dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f26fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 0x5626110dd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46818/0 (socket says 172.21.15.175:46818) 2024-07-29T00:38:35.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f26fb400640 1 -- 172.21.15.175:0/1566393547 learned_addr learned my addr 172.21.15.175:0/1566393547 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:35.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.461+0000 7f26fbe00640 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26f2a00640 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26fb400640 1 -- 172.21.15.175:0/1566393547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 msgr2=0x5626110b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26fb400640 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26fb400640 1 -- 172.21.15.175:0/1566393547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 msgr2=0x5626110dda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26fb400640 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110dda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26fb400640 1 -- 172.21.15.175:0/1566393547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626110b7740 con 0x5626110bab70 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.462+0000 7f26fb400640 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 0x5626110dd500 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26ec005e30 tx=0x7f26ec042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6b9e8e153fd1fb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f26ebe00640 1 -- 172.21.15.175:0/1566393547 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec034040 con 0x5626110bab70 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f26ebe00640 1 -- 172.21.15.175:0/1566393547 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec058070 con 0x5626110bab70 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f26ebe00640 1 -- 172.21.15.175:0/1566393547 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0539e0 con 0x5626110bab70 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 -- 172.21.15.175:0/1566393547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 msgr2=0x5626110dd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 0x5626110dd500 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26ec005e30 tx=0x7f26ec042000 comp rx=0 tx=0).stop 2024-07-29T00:38:35.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 -- 172.21.15.175:0/1566393547 shutdown_connections 2024-07-29T00:38:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110c5910 0x5626110b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 --2- 172.21.15.175:0/1566393547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110bab70 0x5626110dd500 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 -- 172.21.15.175:0/1566393547 >> 172.21.15.175:0/1566393547 conn(0x562611083950 msgr2=0x5626110bdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 -- 172.21.15.175:0/1566393547 shutdown_connections 2024-07-29T00:38:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.463+0000 7f2705278740 1 -- 172.21.15.175:0/1566393547 wait complete. 2024-07-29T00:38:35.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.464+0000 7f2705278740 1 -- start start 2024-07-29T00:38:35.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.464+0000 7f2705278740 1 Processor -- start 2024-07-29T00:38:35.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bab70 0x5626110cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1b4f60 con 0x5626110bc0e0 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1b4890 con 0x5626110c5910 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1b1ad0 con 0x5626110bab70 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f2705278740 10 client.0 _ll_register_callbacks cb 0x7f26e400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26f2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26f2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110bf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46876/0 (socket says 172.21.15.175:46876) 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bab70 0x5626110cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26f2a00640 1 -- 172.21.15.175:0/3169511208 learned_addr learned my addr 172.21.15.175:0/3169511208 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fbe00640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fb400640 1 -- 172.21.15.175:0/3169511208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 msgr2=0x5626110bf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fb400640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110bf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fb400640 1 -- 172.21.15.175:0/3169511208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 msgr2=0x5626110bf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fb400640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110bf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26fb400640 1 -- 172.21.15.175:0/3169511208 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626110bfcb0 con 0x5626110bab70 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.465+0000 7f26f2a00640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f26fb400640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bab70 0x5626110cdec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26ec055040 tx=0x7f26ec02ffb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa85b1adb4a8a65 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f26e9600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f26e9600640 1 -- 172.21.15.175:0/3169511208 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec060020 con 0x5626110bab70 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f26e9600640 1 -- 172.21.15.175:0/3169511208 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f26ec0537c0 con 0x5626110bab70 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ec058070 con 0x5626110bab70 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f26e9600640 1 -- 172.21.15.175:0/3169511208 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f26ec06b050 con 0x5626110bab70 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f2705278740 10 client.25810 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.466+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562611140160 con 0x5626110bab70 2024-07-29T00:38:35.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f26e9600640 1 -- 172.21.15.175:0/3169511208 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec058070 con 0x5626110bab70 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f26e9600640 1 -- 172.21.15.175:0/3169511208 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f26ec05c450 con 0x5626110bab70 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f2705278740 10 client.25810 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f2705278740 10 client.25810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f2705278740 20 client.25810 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f2705278740 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x562611140870 0x562611142c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f2705278740 4 client.25810 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562611028530 con 0x562611140870 2024-07-29T00:38:35.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.467+0000 7f26f2a00640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x562611140870 0x562611142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:35.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f26f2a00640 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x562611140870 0x562611142c90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a6cfa5be79ed5f2 server_cookie=a86acf00d6cfd196 in_seq=0 out_seq=0 2024-07-29T00:38:35.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f26e9600640 10 client.25810 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:35.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f26e9600640 1 -- 172.21.15.175:0/3169511208 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562611028530 con 0x562611140870 2024-07-29T00:38:35.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f26e9600640 10 client.25810 handle_command_reply: tid=0 2024-07-29T00:38:35.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 client.25810 shutdown 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bab70 msgr2=0x5626110cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bab70 0x5626110cdec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26ec055040 tx=0x7f26ec02ffb0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 shutdown_connections 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x562611140870 0x562611142c90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626110c5910 0x5626110bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626110bc0e0 0x5626110bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 --2- 172.21.15.175:0/3169511208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626110bab70 0x5626110cdec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 >> 172.21.15.175:0/3169511208 conn(0x562611083950 msgr2=0x5626110b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.468+0000 7f26e9600640 0 client.25810 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 shutdown_connections 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2705278740 1 -- 172.21.15.175:0/3169511208 wait complete. 2024-07-29T00:38:35.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2705278740 20 client.25810 trim_cache size 0 max 16384 2024-07-29T00:38:35.473 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:35.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26e00a2cf0 msgr2=0x7f26e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2701000640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26e00a2cf0 0x7f26e00a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5626110b8060 tx=0x7f26ec004a60 comp rx=0 tx=0).stop 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 msgr2=0x7f26fc19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2701000640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc19eb30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e9f0 tx=0x7f26e400edf0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.469+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 shutdown_connections 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26e00a2cf0 0x7f26e00a51b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26fc1126d0 0x7f26fc1a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26fc074ba0 0x7f26fc19eb30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 --2- 172.21.15.175:0/712255309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc074280 0x7f26fc19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 >> 172.21.15.175:0/712255309 conn(0x7f26fc0fd400 msgr2=0x7f26fc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 shutdown_connections 2024-07-29T00:38:35.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:35.470+0000 7f2701000640 1 -- 172.21.15.175:0/712255309 wait complete. 2024-07-29T00:38:35.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39439 8.90 MB/sec warmup 26 sec latency 4943.606 ms 2024-07-29T00:38:35.985 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:35 smithi175 ceph-mon[32138]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 763 KiB/s wr, 104 op/s 2024-07-29T00:38:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:35 smithi181 ceph-mon[36706]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 763 KiB/s wr, 104 op/s 2024-07-29T00:38:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:35 smithi204 ceph-mon[37368]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 763 KiB/s wr, 104 op/s 2024-07-29T00:38:36.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:38:36.848 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39439 8.57 MB/sec warmup 27 sec latency 1137.807 ms 2024-07-29T00:38:37.022 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:37.680 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:37 smithi175 ceph-mon[32138]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 782 KiB/s wr, 113 op/s 2024-07-29T00:38:37.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 -- 172.21.15.175:0/1576754964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 msgr2=0x7ff538107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 --2- 172.21.15.175:0/1576754964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff538107aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff528009930 tx=0x7ff52802a420 comp rx=0 tx=0).stop 2024-07-29T00:38:37.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 -- 172.21.15.175:0/1576754964 shutdown_connections 2024-07-29T00:38:37.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 --2- 172.21.15.175:0/1576754964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538102060 0x7ff53810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 --2- 172.21.15.175:0/1576754964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5381016c0 0x7ff538101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 --2- 172.21.15.175:0/1576754964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff538107aa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 -- 172.21.15.175:0/1576754964 >> 172.21.15.175:0/1576754964 conn(0x7ff5380fd400 msgr2=0x7ff5380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 -- 172.21.15.175:0/1576754964 shutdown_connections 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.679+0000 7ff537e00640 1 -- 172.21.15.175:0/1576754964 wait complete. 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.680+0000 7ff537e00640 1 Processor -- start 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.680+0000 7ff537e00640 1 -- start start 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.680+0000 7ff537e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 0x7ff53819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff538102060 0x7ff53819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff53819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111a80 con 0x7ff5381076c0 2024-07-29T00:38:37.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111900 con 0x7ff5381016c0 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111c00 con 0x7ff538102060 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff53819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff536a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 0x7ff53819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff53819eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46902/0 (socket says 172.21.15.175:46902) 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff536a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 0x7ff53819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46840/0 (socket says 172.21.15.175:46840) 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 -- 172.21.15.175:0/2281070201 learned_addr learned my addr 172.21.15.175:0/2281070201 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff536000640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff538102060 0x7ff53819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 -- 172.21.15.175:0/2281070201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff538102060 msgr2=0x7ff53819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff538102060 0x7ff53819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 -- 172.21.15.175:0/2281070201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 msgr2=0x7ff53819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 0x7ff53819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff537400640 1 -- 172.21.15.175:0/2281070201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53819f190 con 0x7ff5381076c0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff536000640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff538102060 0x7ff53819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.681+0000 7ff536a00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 0x7ff53819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.682+0000 7ff537400640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff53819eab0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff53000eb10 tx=0x7ff53000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.682+0000 7ff52f400640 1 -- 172.21.15.175:0/2281070201 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530004b20 con 0x7ff5381076c0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.682+0000 7ff52f400640 1 -- 172.21.15.175:0/2281070201 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff530004cc0 con 0x7ff5381076c0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.682+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53819f480 con 0x7ff5381076c0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.682+0000 7ff52f400640 1 -- 172.21.15.175:0/2281070201 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530010760 con 0x7ff5381076c0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.682+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5381a6d60 con 0x7ff5381076c0 2024-07-29T00:38:37.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.685+0000 7ff540411740 1 Processor -- start 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.685+0000 7ff52f400640 1 -- 172.21.15.175:0/2281070201 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff530005020 con 0x7ff5381076c0 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 -- start start 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628ea4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628ea4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c628e0aab0 con 0x55c628eaf910 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c628da2a80 con 0x55c628ea60e0 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.686+0000 7ff540411740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c628e06a70 con 0x55c628ea4b70 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628ea4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628ea4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628ea4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53530/0 (socket says 172.21.15.175:53530) 2024-07-29T00:38:37.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 -- 172.21.15.175:0/227618831 learned_addr learned my addr 172.21.15.175:0/227618831 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628ea4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46842/0 (socket says 172.21.15.175:46842) 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536000640 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 -- 172.21.15.175:0/227618831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 msgr2=0x55c628ea4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628ea4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 -- 172.21.15.175:0/227618831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 msgr2=0x55c628eb1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff536a00640 1 -- 172.21.15.175:0/227618831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c628b7c3c0 con 0x55c628ea4b70 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.687+0000 7ff52f400640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff51c0a2d70 0x7ff51c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.688+0000 7ff536a00640 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628ea4fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff528000c00 tx=0x7ff528005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edbb137ff311e8fd server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:37.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.688+0000 7ff52f400640 1 -- 172.21.15.175:0/2281070201 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7ff5300cf960 con 0x7ff5381076c0 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.688+0000 7ff536000640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff51c0a2d70 0x7ff51c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.688+0000 7ff523e00640 1 -- 172.21.15.175:0/227618831 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528035040 con 0x55c628ea4b70 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.688+0000 7ff523e00640 1 -- 172.21.15.175:0/227618831 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528031830 con 0x55c628ea4b70 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.688+0000 7ff523e00640 1 -- 172.21.15.175:0/227618831 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528003e60 con 0x55c628ea4b70 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 -- 172.21.15.175:0/227618831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 msgr2=0x55c628ea4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628ea4fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff528000c00 tx=0x7ff528005d50 comp rx=0 tx=0).stop 2024-07-29T00:38:37.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff536000640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff51c0a2d70 0x7ff51c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55c628ea5610 tx=0x7ff524005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 -- 172.21.15.175:0/227618831 shutdown_connections 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628ea4fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 --2- 172.21.15.175:0/227618831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628ea4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 -- 172.21.15.175:0/227618831 >> 172.21.15.175:0/227618831 conn(0x55c628e6d950 msgr2=0x55c628ea1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 -- 172.21.15.175:0/227618831 shutdown_connections 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.689+0000 7ff540411740 1 -- 172.21.15.175:0/227618831 wait complete. 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.690+0000 7ff540411740 1 -- start start 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.690+0000 7ff540411740 1 Processor -- start 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea4b70 0x55c628eac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea60e0 0x55c628eacdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628ead300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c628eb2650 con 0x55c628ea4b70 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c628dc5c10 con 0x55c628eaf910 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c628eb27d0 con 0x55c628ea60e0 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff540411740 10 client.0 _ll_register_callbacks cb 0x55c628d8ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff537400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628ead300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff537400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628ead300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:46858/0 (socket says 172.21.15.175:46858) 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.691+0000 7ff537400640 1 -- 172.21.15.175:0/1340461826 learned_addr learned my addr 172.21.15.175:0/1340461826 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff536a00640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea4b70 0x55c628eac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff537400640 1 -- 172.21.15.175:0/1340461826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea60e0 msgr2=0x55c628eacdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:38:37.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff537400640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea60e0 0x55c628eacdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff537400640 1 -- 172.21.15.175:0/1340461826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea4b70 msgr2=0x55c628eac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff537400640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea4b70 0x55c628eac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff537400640 1 -- 172.21.15.175:0/1340461826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c628dc91e0 con 0x55c628eaf910 2024-07-29T00:38:37.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff536a00640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea4b70 0x55c628eac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:37.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff537400640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628ead300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5300ccfd0 tx=0x7ff53009b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4adab0478da045 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:37.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff520c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:37.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff520c00640 1 -- 172.21.15.175:0/1340461826 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5300a47e0 con 0x55c628eaf910 2024-07-29T00:38:37.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff520c00640 1 -- 172.21.15.175:0/1340461826 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7ff530005310 con 0x55c628eaf910 2024-07-29T00:38:37.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.692+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff528031830 con 0x55c628eaf910 2024-07-29T00:38:37.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff520c00640 1 -- 172.21.15.175:0/1340461826 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff5300a01b0 con 0x55c628eaf910 2024-07-29T00:38:37.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 10 client.25835 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:37.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c628ebe3c0 con 0x55c628eaf910 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff520c00640 1 -- 172.21.15.175:0/1340461826 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528031830 con 0x55c628eaf910 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff520c00640 1 -- 172.21.15.175:0/1340461826 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff53009acc0 con 0x55c628eaf910 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 10 client.25835 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 10 client.25835 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 20 client.25835 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628ebf880 0x55c628ec1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 4 client.25835 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.693+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c628e18020 con 0x55c628ebf880 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.694+0000 7ff536a00640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628ebf880 0x55c628ec1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.694+0000 7ff536a00640 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628ebf880 0x55c628ec1d40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28771d28a65ad403 server_cookie=f3684fb63cf53a11 in_seq=0 out_seq=0 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.694+0000 7ff520c00640 10 client.25835 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff520c00640 1 -- 172.21.15.175:0/1340461826 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c628e18020 con 0x55c628ebf880 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff520c00640 10 client.25835 handle_command_reply: tid=0 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 client.25835 shutdown 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 msgr2=0x55c628ead300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628ead300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5300ccfd0 tx=0x7ff53009b820 comp rx=0 tx=0).stop 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 shutdown_connections 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628ebf880 0x55c628ec1d40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628ead300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea60e0 0x55c628eacdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 --2- 172.21.15.175:0/1340461826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea4b70 0x55c628eac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 >> 172.21.15.175:0/1340461826 conn(0x55c628e6d950 msgr2=0x55c628ea6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:37.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.695+0000 7ff520c00640 0 client.25835 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:37.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.696+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 shutdown_connections 2024-07-29T00:38:37.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.696+0000 7ff540411740 1 -- 172.21.15.175:0/1340461826 wait complete. 2024-07-29T00:38:37.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.696+0000 7ff540411740 20 client.25835 trim_cache size 0 max 16384 2024-07-29T00:38:37.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 Processor -- start 2024-07-29T00:38:37.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 -- start start 2024-07-29T00:38:37.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea60e0 0x55c628eb1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628e9fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381af740 con 0x55c628ea60e0 2024-07-29T00:38:37.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b1340 con 0x55c628eaf910 2024-07-29T00:38:37.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.710+0000 7ff540411740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381afe10 con 0x55c628ea4b70 2024-07-29T00:38:37.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea60e0 0x55c628eb1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53540/0 (socket says 172.21.15.175:53540) 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 -- 172.21.15.175:0/1268717738 learned_addr learned my addr 172.21.15.175:0/1268717738 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff537400640 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628e9fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 -- 172.21.15.175:0/1268717738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 msgr2=0x55c628e9fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628e9fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 -- 172.21.15.175:0/1268717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea60e0 msgr2=0x55c628eb1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea60e0 0x55c628eb1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536a00640 1 -- 172.21.15.175:0/1268717738 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c628ea76c0 con 0x55c628ea4b70 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.711+0000 7ff536000640 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea60e0 0x55c628eb1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.712+0000 7ff536a00640 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb0d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff528031470 tx=0x7ff528005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db95887cfe38c3b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:37.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.712+0000 7ff523400640 1 -- 172.21.15.175:0/1268717738 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528056070 con 0x55c628ea4b70 2024-07-29T00:38:37.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.712+0000 7ff523400640 1 -- 172.21.15.175:0/1268717738 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5280035f0 con 0x55c628ea4b70 2024-07-29T00:38:37.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.712+0000 7ff523400640 1 -- 172.21.15.175:0/1268717738 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528035040 con 0x55c628ea4b70 2024-07-29T00:38:37.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 -- 172.21.15.175:0/1268717738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 msgr2=0x55c628eb0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb0d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff528031470 tx=0x7ff528005fd0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 -- 172.21.15.175:0/1268717738 shutdown_connections 2024-07-29T00:38:37.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628eaf910 0x55c628e9fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628ea60e0 0x55c628eb1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 --2- 172.21.15.175:0/1268717738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb0d00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 -- 172.21.15.175:0/1268717738 >> 172.21.15.175:0/1268717738 conn(0x55c628e6d950 msgr2=0x55c628e9c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 -- 172.21.15.175:0/1268717738 shutdown_connections 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 -- 172.21.15.175:0/1268717738 wait complete. 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.713+0000 7ff540411740 1 -- start start 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 Processor -- start 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628eb7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b0bb0 con 0x55c628eaf910 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b04e0 con 0x55c628ea60e0 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381ad720 con 0x55c628ea4b70 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.714+0000 7ff540411740 10 client.0 _ll_register_callbacks cb 0x7ff53009bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff537400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53548/0 (socket says 172.21.15.175:53548) 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 -- 172.21.15.175:0/3093317972 learned_addr learned my addr 172.21.15.175:0/3093317972 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536000640 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628eb7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 -- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 msgr2=0x55c628eb7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628eb7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 -- 172.21.15.175:0/3093317972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 msgr2=0x55c628eb2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 -- 172.21.15.175:0/3093317972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c628ec81f0 con 0x55c628ea4b70 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff536a00640 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb7a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5280096c0 tx=0x7ff528005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d402620980de9d34 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.715+0000 7ff520c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff520c00640 1 -- 172.21.15.175:0/3093317972 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528056070 con 0x55c628ea4b70 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff520c00640 1 -- 172.21.15.175:0/3093317972 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7ff5280317b0 con 0x55c628ea4b70 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5280035f0 con 0x55c628ea4b70 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff520c00640 1 -- 172.21.15.175:0/3093317972 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff528035040 con 0x55c628ea4b70 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff540411740 10 client.25825 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c628f2aba0 con 0x55c628ea4b70 2024-07-29T00:38:37.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.716+0000 7ff520c00640 1 -- 172.21.15.175:0/3093317972 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5280035f0 con 0x55c628ea4b70 2024-07-29T00:38:37.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff520c00640 1 -- 172.21.15.175:0/3093317972 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff5280513c0 con 0x55c628ea4b70 2024-07-29T00:38:37.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff540411740 10 client.25825 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:37.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff540411740 10 client.25825 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:38:37.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff540411740 20 client.25825 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff540411740 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628f2add0 0x55c628f2d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff540411740 4 client.25825 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c628e12530 con 0x55c628f2add0 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.717+0000 7ff536000640 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628f2add0 0x55c628f2d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff536000640 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628f2add0 0x55c628f2d1c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf3168040c85301c server_cookie=604d69fa739d3b4a in_seq=0 out_seq=0 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff520c00640 10 client.25825 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff520c00640 1 -- 172.21.15.175:0/3093317972 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c628e12530 con 0x55c628f2add0 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff520c00640 10 client.25825 handle_command_reply: tid=0 2024-07-29T00:38:37.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 client.25825 shutdown 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 msgr2=0x55c628eb7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb7a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5280096c0 tx=0x7ff528005850 comp rx=0 tx=0).stop 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 shutdown_connections 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c628eaf910 0x55c628eb2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55c628f2add0 0x55c628f2d1c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c628ea60e0 0x55c628eb7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 --2- 172.21.15.175:0/3093317972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c628ea4b70 0x55c628eb7a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff520c00640 0 client.25825 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.718+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 >> 172.21.15.175:0/3093317972 conn(0x55c628e6d950 msgr2=0x55c628e9fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 shutdown_connections 2024-07-29T00:38:37.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff540411740 1 -- 172.21.15.175:0/3093317972 wait complete. 2024-07-29T00:38:37.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff540411740 20 client.25825 trim_cache size 0 max 16384 2024-07-29T00:38:37.724 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:37.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff51c0a2d70 msgr2=0x7ff51c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff537e00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff51c0a2d70 0x7ff51c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55c628ea5610 tx=0x7ff524005f90 comp rx=0 tx=0).stop 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 msgr2=0x7ff53819eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.719+0000 7ff537e00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff53819eab0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff53000eb10 tx=0x7ff53000c2d0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 shutdown_connections 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff51c0a2d70 0x7ff51c0a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5381076c0 0x7ff53819eab0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff538102060 0x7ff53819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 --2- 172.21.15.175:0/2281070201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5381016c0 0x7ff53819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 >> 172.21.15.175:0/2281070201 conn(0x7ff5380fd400 msgr2=0x7ff53810ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 shutdown_connections 2024-07-29T00:38:37.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:37.720+0000 7ff537e00640 1 -- 172.21.15.175:0/2281070201 wait complete. 2024-07-29T00:38:37.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39439 8.26 MB/sec warmup 28 sec latency 2137.917 ms 2024-07-29T00:38:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:37 smithi181 ceph-mon[36706]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 782 KiB/s wr, 113 op/s 2024-07-29T00:38:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:37 smithi204 ceph-mon[37368]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 782 KiB/s wr, 113 op/s 2024-07-29T00:38:38.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:38:38.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39439 7.98 MB/sec warmup 29 sec latency 3138.012 ms 2024-07-29T00:38:39.180 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:39.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39439 7.71 MB/sec warmup 30 sec latency 4138.123 ms 2024-07-29T00:38:39.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 -- 172.21.15.175:0/3302809473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c1038c0 msgr2=0x7f8a2c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 --2- 172.21.15.175:0/3302809473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c103d20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009900 tx=0x7f8a1402a3f0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.860 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:39 smithi175 ceph-mon[32138]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 250 KiB/s wr, 46 op/s 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 -- 172.21.15.175:0/3302809473 shutdown_connections 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 --2- 172.21.15.175:0/3302809473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c104260 0x7f8a2c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 --2- 172.21.15.175:0/3302809473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c103d20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 --2- 172.21.15.175:0/3302809473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1098c0 0x7f8a2c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 -- 172.21.15.175:0/3302809473 >> 172.21.15.175:0/3302809473 conn(0x7f8a2c0ff5c0 msgr2=0x7f8a2c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 -- 172.21.15.175:0/3302809473 shutdown_connections 2024-07-29T00:38:39.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.860+0000 7f8a31600640 1 -- 172.21.15.175:0/3302809473 wait complete. 2024-07-29T00:38:39.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.861+0000 7f8a31600640 1 Processor -- start 2024-07-29T00:38:39.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.861+0000 7f8a31600640 1 -- start start 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a31600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a31600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c104260 0x7f8a2c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a31600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c1098c0 0x7f8a2c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a31600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c113cd0 con 0x7f8a2c1038c0 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a31600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c113b50 con 0x7f8a2c104260 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a31600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c113e50 con 0x7f8a2c1098c0 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53878/0 (socket says 172.21.15.175:53878) 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a2be00640 1 -- 172.21.15.175:0/3186355188 learned_addr learned my addr 172.21.15.175:0/3186355188 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a30c00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c1098c0 0x7f8a2c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.862+0000 7f8a23e00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c104260 0x7f8a2c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a2be00640 1 -- 172.21.15.175:0/3186355188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c1098c0 msgr2=0x7f8a2c19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a2be00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c1098c0 0x7f8a2c19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a2be00640 1 -- 172.21.15.175:0/3186355188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c104260 msgr2=0x7f8a2c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a2be00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c104260 0x7f8a2c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a2be00640 1 -- 172.21.15.175:0/3186355188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c19f160 con 0x7f8a2c1038c0 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a30c00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c1098c0 0x7f8a2c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a2be00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c19a1b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e9f0 tx=0x7f8a1c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a29600640 1 -- 172.21.15.175:0/3186355188 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00ccc0 con 0x7f8a2c1038c0 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a29600640 1 -- 172.21.15.175:0/3186355188 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1c010910 con 0x7f8a2c1038c0 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.863+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c19f450 con 0x7f8a2c1038c0 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.864+0000 7f8a29600640 1 -- 172.21.15.175:0/3186355188 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0188a0 con 0x7f8a2c1038c0 2024-07-29T00:38:39.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.864+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c1a6d30 con 0x7f8a2c1038c0 2024-07-29T00:38:39.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.867+0000 7f8a29600640 1 -- 172.21.15.175:0/3186355188 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8a1c010ac0 con 0x7f8a2c1038c0 2024-07-29T00:38:39.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.867+0000 7f8a357ff740 1 Processor -- start 2024-07-29T00:38:39.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.867+0000 7f8a357ff740 1 -- start start 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b4745f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4745f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474606d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a357ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b47455fab0 con 0x55b4745f9b70 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a357ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b4744f7a80 con 0x55b474604910 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a357ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b47455ba70 con 0x55b4745fb0e0 2024-07-29T00:38:39.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4745f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a2be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4745f9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53888/0 (socket says 172.21.15.175:53888) 2024-07-29T00:38:39.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a2be00640 1 -- 172.21.15.175:0/1773643018 learned_addr learned my addr 172.21.15.175:0/1773643018 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:39.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a30c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b4745f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.868+0000 7f8a23e00640 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474606d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a30c00640 1 -- 172.21.15.175:0/1773643018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 msgr2=0x55b474606d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a30c00640 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474606d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a30c00640 1 -- 172.21.15.175:0/1773643018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 msgr2=0x55b4745f9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a30c00640 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4745f9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a30c00640 1 -- 172.21.15.175:0/1773643018 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4742d13c0 con 0x55b4745fb0e0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a30c00640 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b4745f9630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a24011760 tx=0x7f8a24011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9a4a3f389a6109 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a29600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8a100a2cf0 0x7f8a100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a21200640 1 -- 172.21.15.175:0/1773643018 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24004d20 con 0x55b4745fb0e0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a21200640 1 -- 172.21.15.175:0/1773643018 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a24003970 con 0x55b4745fb0e0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a21200640 1 -- 172.21.15.175:0/1773643018 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240044f0 con 0x55b4745fb0e0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a2be00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8a100a2cf0 0x7f8a100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.869+0000 7f8a29600640 1 -- 172.21.15.175:0/3186355188 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8a1c0cf8b0 con 0x7f8a2c1038c0 2024-07-29T00:38:39.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 -- 172.21.15.175:0/1773643018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 msgr2=0x55b4745f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a2be00640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8a100a2cf0 0x7f8a100a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e9c0 tx=0x7f8a1c0d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:39.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b4745f9630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a24011760 tx=0x7f8a24011c20 comp rx=0 tx=0).stop 2024-07-29T00:38:39.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 -- 172.21.15.175:0/1773643018 shutdown_connections 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474606d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4745f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 --2- 172.21.15.175:0/1773643018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b4745f9630 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 -- 172.21.15.175:0/1773643018 >> 172.21.15.175:0/1773643018 conn(0x55b4745c2950 msgr2=0x55b4745f6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 -- 172.21.15.175:0/1773643018 shutdown_connections 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.870+0000 7f8a357ff740 1 -- 172.21.15.175:0/1773643018 wait complete. 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.871+0000 7f8a357ff740 1 -- start start 2024-07-29T00:38:39.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.871+0000 7f8a357ff740 1 Processor -- start 2024-07-29T00:38:39.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 0x55b4746018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745fb0e0 0x55b474601e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474602350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b4746076b0 con 0x55b4745fb0e0 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b47451ac10 con 0x55b474604910 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b474607830 con 0x55b4745f9b70 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a357ff740 10 client.0 _ll_register_callbacks cb 0x55b4744e3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a30c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 0x55b4746018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a30c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 0x55b4746018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:37978/0 (socket says 172.21.15.175:37978) 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a30c00640 1 -- 172.21.15.175:0/2079021021 learned_addr learned my addr 172.21.15.175:0/2079021021 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a30c00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474602350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.872+0000 7f8a23e00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745fb0e0 0x55b474601e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a30c00640 1 -- 172.21.15.175:0/2079021021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 msgr2=0x55b474602350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a30c00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474602350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a30c00640 1 -- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745fb0e0 msgr2=0x55b474601e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a30c00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745fb0e0 0x55b474601e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a30c00640 1 -- 172.21.15.175:0/2079021021 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47451e1e0 con 0x55b4745f9b70 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a23e00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745fb0e0 0x55b474601e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a30c00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 0x55b4746018d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400f790 tx=0x7f8a24004130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f0eba154382bce server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a19600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a19600640 1 -- 172.21.15.175:0/2079021021 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2402e060 con 0x55b4745f9b70 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a24003970 con 0x55b4745f9b70 2024-07-29T00:38:39.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.873+0000 7f8a19600640 1 -- 172.21.15.175:0/2079021021 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8a2400fa60 con 0x55b4745f9b70 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a19600640 1 -- 172.21.15.175:0/2079021021 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a2401d070 con 0x55b4745f9b70 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 10 client.25837 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b474613460 con 0x55b4745f9b70 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a19600640 1 -- 172.21.15.175:0/2079021021 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a24003970 con 0x55b4745f9b70 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a19600640 1 -- 172.21.15.175:0/2079021021 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8a24005090 con 0x55b4745f9b70 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 10 client.25837 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:39.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 10 client.25837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 20 client.25837 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474614870 0x55b474616c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 4 client.25837 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.874+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b47456d020 con 0x55b474614870 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.875+0000 7f8a23e00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474614870 0x55b474616c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.875+0000 7f8a23e00640 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474614870 0x55b474616c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aba6af0a69c3b5cc server_cookie=8652a90a20675037 in_seq=0 out_seq=0 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.875+0000 7f8a19600640 10 client.25837 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a19600640 1 -- 172.21.15.175:0/2079021021 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b47456d020 con 0x55b474614870 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a19600640 10 client.25837 handle_command_reply: tid=0 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 client.25837 shutdown 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 msgr2=0x55b4746018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 0x55b4746018d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400f790 tx=0x7f8a24004130 comp rx=0 tx=0).stop 2024-07-29T00:38:39.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 shutdown_connections 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474614870 0x55b474616c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b474602350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745fb0e0 0x55b474601e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a19600640 0 client.25837 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 --2- 172.21.15.175:0/2079021021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745f9b70 0x55b4746018d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 >> 172.21.15.175:0/2079021021 conn(0x55b4745c2950 msgr2=0x55b4745fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.876+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 shutdown_connections 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.877+0000 7f8a357ff740 1 -- 172.21.15.175:0/2079021021 wait complete. 2024-07-29T00:38:39.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.877+0000 7f8a357ff740 20 client.25837 trim_cache size 0 max 16384 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.890+0000 7f8a357ff740 1 Processor -- start 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.890+0000 7f8a357ff740 1 -- start start 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4746021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b47461c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b47461c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a357ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1af7c0 con 0x55b4745f9b70 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a357ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b13c0 con 0x55b474604910 2024-07-29T00:38:39.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a357ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1afe90 con 0x55b4745fb0e0 2024-07-29T00:38:39.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b47461c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a30c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4746021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b47461c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:37990/0 (socket says 172.21.15.175:37990) 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.891+0000 7f8a23e00640 1 -- 172.21.15.175:0/335299461 learned_addr learned my addr 172.21.15.175:0/335299461 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a30c00640 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b47461c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a23e00640 1 -- 172.21.15.175:0/335299461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 msgr2=0x55b47461c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a23e00640 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b47461c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a23e00640 1 -- 172.21.15.175:0/335299461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 msgr2=0x55b4746021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a23e00640 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4746021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a23e00640 1 -- 172.21.15.175:0/335299461 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4745fa180 con 0x55b4745fb0e0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a30c00640 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b47461c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.892+0000 7f8a23e00640 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b47461c380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009230 tx=0x7f8a14005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2137c3d2a353f7b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a1be00640 1 -- 172.21.15.175:0/335299461 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14035040 con 0x55b4745fb0e0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a1be00640 1 -- 172.21.15.175:0/335299461 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1404a070 con 0x55b4745fb0e0 2024-07-29T00:38:39.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a1be00640 1 -- 172.21.15.175:0/335299461 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14004d30 con 0x55b4745fb0e0 2024-07-29T00:38:39.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 -- 172.21.15.175:0/335299461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 msgr2=0x55b47461c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b47461c380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009230 tx=0x7f8a14005b50 comp rx=0 tx=0).stop 2024-07-29T00:38:39.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 -- 172.21.15.175:0/335299461 shutdown_connections 2024-07-29T00:38:39.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b474604910 0x55b47461c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4745fb0e0 0x55b47461c380 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 --2- 172.21.15.175:0/335299461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b4746021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.893+0000 7f8a357ff740 1 -- 172.21.15.175:0/335299461 >> 172.21.15.175:0/335299461 conn(0x55b4745c2950 msgr2=0x55b474603950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.894+0000 7f8a357ff740 1 -- 172.21.15.175:0/335299461 shutdown_connections 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.894+0000 7f8a357ff740 1 -- 172.21.15.175:0/335299461 wait complete. 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.894+0000 7f8a357ff740 1 -- start start 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.894+0000 7f8a357ff740 1 Processor -- start 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b474608020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4745fb0e0 0x55b474608560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b474604910 0x55b474685b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b0c30 con 0x55b4745f9b70 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b0560 con 0x55b4745fb0e0 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1ad7a0 con 0x55b474604910 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a357ff740 10 client.0 _ll_register_callbacks cb 0x7f8a1c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b474608020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b474608020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53920/0 (socket says 172.21.15.175:53920) 2024-07-29T00:38:39.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 -- 172.21.15.175:0/988439648 learned_addr learned my addr 172.21.15.175:0/988439648 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:39.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a23e00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4745fb0e0 0x55b474608560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b474604910 0x55b474685b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 -- 172.21.15.175:0/988439648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b474604910 msgr2=0x55b474685b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b474604910 0x55b474685b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 -- 172.21.15.175:0/988439648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4745fb0e0 msgr2=0x55b474608560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4745fb0e0 0x55b474608560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.895+0000 7f8a30c00640 1 -- 172.21.15.175:0/988439648 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b474686090 con 0x55b4745f9b70 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a30c00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b474608020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8a240099a0 tx=0x7f8a24014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4290caef587b9e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:39.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a19600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a19600640 1 -- 172.21.15.175:0/988439648 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2402e040 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a19600640 1 -- 172.21.15.175:0/988439648 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f8a24004f10 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1404a070 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a19600640 1 -- 172.21.15.175:0/988439648 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a24005110 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a357ff740 10 client.15747 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.896+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b474687280 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a19600640 1 -- 172.21.15.175:0/988439648 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1404a070 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a19600640 1 -- 172.21.15.175:0/988439648 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8a240046c0 con 0x55b4745f9b70 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a357ff740 10 client.15747 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a357ff740 10 client.15747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a357ff740 20 client.15747 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a357ff740 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474687890 0x55b474689c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a357ff740 4 client.15747 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:39.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b474567530 con 0x55b474687890 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.897+0000 7f8a23e00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474687890 0x55b474689c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a23e00640 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474687890 0x55b474689c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc1c95da20f540df server_cookie=f3cb9cc5aa182dcc in_seq=0 out_seq=0 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a19600640 10 client.15747 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a19600640 1 -- 172.21.15.175:0/988439648 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b474567530 con 0x55b474687890 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a19600640 10 client.15747 handle_command_reply: tid=0 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 client.15747 shutdown 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 msgr2=0x55b474608020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b474608020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8a240099a0 tx=0x7f8a24014040 comp rx=0 tx=0).stop 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 shutdown_connections 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55b474687890 0x55b474689c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a19600640 0 client.15747 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b474604910 0x55b474685b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4745fb0e0 0x55b474608560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 --2- 172.21.15.175:0/988439648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4745f9b70 0x55b474608020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.898+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 >> 172.21.15.175:0/988439648 conn(0x55b4745c2950 msgr2=0x55b4745f4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:39.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 shutdown_connections 2024-07-29T00:38:39.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a357ff740 1 -- 172.21.15.175:0/988439648 wait complete. 2024-07-29T00:38:39.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a357ff740 20 client.15747 trim_cache size 0 max 16384 2024-07-29T00:38:39.902 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8a100a2cf0 msgr2=0x7f8a100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a31600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8a100a2cf0 0x7f8a100a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e9c0 tx=0x7f8a1c0d5530 comp rx=0 tx=0).stop 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 msgr2=0x7f8a2c19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.899+0000 7f8a31600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c19a1b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e9f0 tx=0x7f8a1c00edf0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 shutdown_connections 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a2c1098c0 0x7f8a2c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a2c104260 0x7f8a2c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f8a100a2cf0 0x7f8a100a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 --2- 172.21.15.175:0/3186355188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a2c1038c0 0x7f8a2c19a1b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 >> 172.21.15.175:0/3186355188 conn(0x7f8a2c0ff5c0 msgr2=0x7f8a2c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:39.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 shutdown_connections 2024-07-29T00:38:39.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:39.900+0000 7f8a31600640 1 -- 172.21.15.175:0/3186355188 wait complete. 2024-07-29T00:38:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:39 smithi181 ceph-mon[36706]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 250 KiB/s wr, 46 op/s 2024-07-29T00:38:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:39 smithi204 ceph-mon[37368]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 250 KiB/s wr, 46 op/s 2024-07-29T00:38:40.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39497 7.46 MB/sec warmup 31 sec latency 4960.342 ms 2024-07-29T00:38:41.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 39930 7.25 MB/sec warmup 32 sec latency 334.167 ms 2024-07-29T00:38:41.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:41 smithi181 ceph-mon[36706]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 250 KiB/s wr, 46 op/s 2024-07-29T00:38:42.032 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:41 smithi175 ceph-mon[32138]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 250 KiB/s wr, 46 op/s 2024-07-29T00:38:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:41 smithi204 ceph-mon[37368]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 250 KiB/s wr, 46 op/s 2024-07-29T00:38:42.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:38:42.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 41616 7.25 MB/sec warmup 33 sec latency 17.153 ms 2024-07-29T00:38:43.070 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:43.840 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:43 smithi175 ceph-mon[32138]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 262 KiB/s wr, 55 op/s 2024-07-29T00:38:43.849 INFO:tasks.workunit.client.0.smithi175.stdout: 1 42870 7.36 MB/sec warmup 34 sec latency 19.138 ms 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 -- 172.21.15.175:0/1640077302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 msgr2=0x7f001010caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 --2- 172.21.15.175:0/1640077302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f001010caf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efffc009860 tx=0x7efffc02a340 comp rx=0 tx=0).stop 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 -- 172.21.15.175:0/1640077302 shutdown_connections 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 --2- 172.21.15.175:0/1640077302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f001010d220 0x7f0010114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 --2- 172.21.15.175:0/1640077302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f001010caf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 --2- 172.21.15.175:0/1640077302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0010108140 0x7f0010108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.885+0000 7f0017800640 1 -- 172.21.15.175:0/1640077302 >> 172.21.15.175:0/1640077302 conn(0x7f0010077890 msgr2=0x7f0010079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.886+0000 7f0017800640 1 -- 172.21.15.175:0/1640077302 shutdown_connections 2024-07-29T00:38:43.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.886+0000 7f0017800640 1 -- 172.21.15.175:0/1640077302 wait complete. 2024-07-29T00:38:43.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 Processor -- start 2024-07-29T00:38:43.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 -- start start 2024-07-29T00:38:43.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0010108140 0x7f00101a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f00101a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001010d220 0x7f001019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0010118220 con 0x7f001010d220 2024-07-29T00:38:43.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00101180a0 con 0x7f0010108af0 2024-07-29T00:38:43.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.887+0000 7f0017800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00101183a0 con 0x7f0010108140 2024-07-29T00:38:43.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f00101a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f00101a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57912/0 (socket says 172.21.15.175:57912) 2024-07-29T00:38:43.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 -- 172.21.15.175:0/1484469923 learned_addr learned my addr 172.21.15.175:0/1484469923 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f0015800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001010d220 0x7f001019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f0014e00640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0010108140 0x7f00101a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 -- 172.21.15.175:0/1484469923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0010108140 msgr2=0x7f00101a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0010108140 0x7f00101a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 -- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001010d220 msgr2=0x7f001019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001010d220 0x7f001019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 -- 172.21.15.175:0/1484469923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001019ecc0 con 0x7f0010108af0 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f0014e00640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0010108140 0x7f00101a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f0015800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001010d220 0x7f001019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.888+0000 7f000fe00640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f00101a4910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efffc035040 tx=0x7efffc033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.889+0000 7f000d600640 1 -- 172.21.15.175:0/1484469923 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc037040 con 0x7f0010108af0 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.889+0000 7f000d600640 1 -- 172.21.15.175:0/1484469923 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efffc040070 con 0x7f0010108af0 2024-07-29T00:38:43.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.889+0000 7f000d600640 1 -- 172.21.15.175:0/1484469923 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc0039b0 con 0x7f0010108af0 2024-07-29T00:38:43.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.889+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001019ef50 con 0x7f0010108af0 2024-07-29T00:38:43.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.889+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101ab2a0 con 0x7f0010108af0 2024-07-29T00:38:43.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.893+0000 7f001a21c740 1 Processor -- start 2024-07-29T00:38:43.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.893+0000 7f000d600640 1 -- 172.21.15.175:0/1484469923 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efffc0305e0 con 0x7f0010108af0 2024-07-29T00:38:43.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.893+0000 7f001a21c740 1 -- start start 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f001a21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x55887364e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f001a21c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x55887364edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f001a21c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x55887365bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f001a21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588735b4ab0 con 0x55887364fef0 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f001a21c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55887354ca80 con 0x558873659720 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f001a21c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5588735b0a70 con 0x55887364e980 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x55887364edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x55887364edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38046/0 (socket says 172.21.15.175:38046) 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 -- 172.21.15.175:0/990076132 learned_addr learned my addr 172.21.15.175:0/990076132 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0015800640 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x55887364e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f000fe00640 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x55887365bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 -- 172.21.15.175:0/990076132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 msgr2=0x55887365bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x55887365bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 -- 172.21.15.175:0/990076132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 msgr2=0x55887364e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x55887364e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f0014e00640 1 -- 172.21.15.175:0/990076132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588733263c0 con 0x55887364e980 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.894+0000 7f000fe00640 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x55887365bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:43.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.895+0000 7f0014e00640 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x55887364edc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0004009c30 tx=0x7f0004010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f260abdf8a5b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:43.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.895+0000 7f0002000640 1 -- 172.21.15.175:0/990076132 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004004020 con 0x55887364e980 2024-07-29T00:38:43.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.895+0000 7f0002000640 1 -- 172.21.15.175:0/990076132 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00040041c0 con 0x55887364e980 2024-07-29T00:38:43.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.895+0000 7f0002000640 1 -- 172.21.15.175:0/990076132 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040126b0 con 0x55887364e980 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f000d600640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efff80a2d70 0x7efff80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f000d600640 1 -- 172.21.15.175:0/1484469923 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7efffc0ef410 con 0x7f0010108af0 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f0015800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efff80a2d70 0x7efff80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 -- 172.21.15.175:0/990076132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 msgr2=0x55887364edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x55887364edc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0004009c30 tx=0x7f0004010740 comp rx=0 tx=0).stop 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 -- 172.21.15.175:0/990076132 shutdown_connections 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x55887365bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x55887364edc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 --2- 172.21.15.175:0/990076132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x55887364e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 -- 172.21.15.175:0/990076132 >> 172.21.15.175:0/990076132 conn(0x558873617950 msgr2=0x55887364b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.896+0000 7f001a21c740 1 -- 172.21.15.175:0/990076132 shutdown_connections 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.897+0000 7f001a21c740 1 -- 172.21.15.175:0/990076132 wait complete. 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.897+0000 7f0015800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efff80a2d70 0x7efff80a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0008005fd0 tx=0x7f0008009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.897+0000 7f001a21c740 1 -- start start 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.897+0000 7f001a21c740 1 Processor -- start 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x5588736566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873656c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558873659720 0x558873657140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55887365c4c0 con 0x558873659720 2024-07-29T00:38:43.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55887356fc10 con 0x55887364fef0 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55887365c640 con 0x55887364e980 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f001a21c740 10 client.0 _ll_register_callbacks cb 0x558873538a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f000fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873656c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f0014e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x5588736566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f0014e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x5588736566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38056/0 (socket says 172.21.15.175:38056) 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f000fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873656c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57940/0 (socket says 172.21.15.175:57940) 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f0014e00640 1 -- 172.21.15.175:0/346644731 learned_addr learned my addr 172.21.15.175:0/346644731 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:43.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.898+0000 7f0015800640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558873659720 0x558873657140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0014e00640 1 -- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 msgr2=0x558873656c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0014e00640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873656c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0014e00640 1 -- 172.21.15.175:0/346644731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558873659720 msgr2=0x558873657140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0014e00640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558873659720 0x558873657140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0014e00640 1 -- 172.21.15.175:0/346644731 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588735731e0 con 0x55887364e980 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0015800640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558873659720 0x558873657140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f000fe00640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873656c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f0014e00640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x5588736566c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0004006010 tx=0x7f0004003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3aac1bbb0987a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7efff6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7efff6a00640 1 -- 172.21.15.175:0/346644731 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040035f0 con 0x55887364e980 2024-07-29T00:38:43.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7efff6a00640 1 -- 172.21.15.175:0/346644731 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0004003790 con 0x55887364e980 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.899+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00040041c0 con 0x55887364e980 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7efff6a00640 1 -- 172.21.15.175:0/346644731 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0004003990 con 0x55887364e980 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 10 client.25852 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558873668280 con 0x55887364e980 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7efff6a00640 1 -- 172.21.15.175:0/346644731 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00040041c0 con 0x55887364e980 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7efff6a00640 1 -- 172.21.15.175:0/346644731 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f000400f040 con 0x55887364e980 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 10 client.25852 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 20 client.25852 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558873669680 0x55887366ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 4 client.25852 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.900+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588735c2020 con 0x558873669680 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.901+0000 7f000fe00640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558873669680 0x55887366ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.901+0000 7f000fe00640 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558873669680 0x55887366ba70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0b28f43d2f508b server_cookie=749c078d8cf95742 in_seq=0 out_seq=0 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.901+0000 7efff6a00640 10 client.25852 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.902+0000 7efff6a00640 1 -- 172.21.15.175:0/346644731 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588735c2020 con 0x558873669680 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.902+0000 7efff6a00640 10 client.25852 handle_command_reply: tid=0 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.902+0000 7f001a21c740 1 client.25852 shutdown 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.902+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 msgr2=0x5588736566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.902+0000 7f001a21c740 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x5588736566c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0004006010 tx=0x7f0004003be0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 shutdown_connections 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558873669680 0x55887366ba70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558873659720 0x558873657140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873656c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 --2- 172.21.15.175:0/346644731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x5588736566c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7efff6a00640 0 client.25852 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 >> 172.21.15.175:0/346644731 conn(0x558873617950 msgr2=0x558873650ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 shutdown_connections 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 1 -- 172.21.15.175:0/346644731 wait complete. 2024-07-29T00:38:43.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.903+0000 7f001a21c740 20 client.25852 trim_cache size 0 max 16384 2024-07-29T00:38:43.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 Processor -- start 2024-07-29T00:38:43.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 -- start start 2024-07-29T00:38:43.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 0x5588736556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873671100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558873659720 0x558873671640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b3d70 con 0x55887364e980 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b5970 con 0x55887364fef0 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.918+0000 7f001a21c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b4440 con 0x558873659720 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 0x5588736556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 0x5588736556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53966/0 (socket says 172.21.15.175:53966) 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f000fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873671100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 -- 172.21.15.175:0/2310673332 learned_addr learned my addr 172.21.15.175:0/2310673332 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:43.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0015800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558873659720 0x558873671640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 -- 172.21.15.175:0/2310673332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558873659720 msgr2=0x558873671640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558873659720 0x558873671640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 -- 172.21.15.175:0/2310673332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 msgr2=0x558873671100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873671100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 -- 172.21.15.175:0/2310673332 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887364e680 con 0x55887364e980 2024-07-29T00:38:43.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.919+0000 7f0014e00640 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 0x5588736556c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f000402d760 tx=0x7f0004010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5e896793a44ed1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:43.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.920+0000 7f0001600640 1 -- 172.21.15.175:0/2310673332 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000401f030 con 0x55887364e980 2024-07-29T00:38:43.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.920+0000 7f0001600640 1 -- 172.21.15.175:0/2310673332 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0004005510 con 0x55887364e980 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.920+0000 7f0001600640 1 -- 172.21.15.175:0/2310673332 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004004580 con 0x55887364e980 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 -- 172.21.15.175:0/2310673332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 msgr2=0x5588736556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 0x5588736556c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f000402d760 tx=0x7f0004010d70 comp rx=0 tx=0).stop 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 -- 172.21.15.175:0/2310673332 shutdown_connections 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558873659720 0x558873671640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55887364fef0 0x558873671100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 --2- 172.21.15.175:0/2310673332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364e980 0x5588736556c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 -- 172.21.15.175:0/2310673332 >> 172.21.15.175:0/2310673332 conn(0x558873617950 msgr2=0x558873654fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 -- 172.21.15.175:0/2310673332 shutdown_connections 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 -- 172.21.15.175:0/2310673332 wait complete. 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.921+0000 7f001a21c740 1 -- start start 2024-07-29T00:38:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 Processor -- start 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x558873661900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x558873661e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x558873652990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b51e0 con 0x55887364fef0 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b4b10 con 0x558873659720 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b1d50 con 0x55887364e980 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.922+0000 7f001a21c740 10 client.0 _ll_register_callbacks cb 0x7f0004009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x558873661900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x558873661900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38064/0 (socket says 172.21.15.175:38064) 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 -- 172.21.15.175:0/2532422571 learned_addr learned my addr 172.21.15.175:0/2532422571 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f000fe00640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x558873661e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0015800640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x558873652990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 -- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 msgr2=0x558873652990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x558873652990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 -- 172.21.15.175:0/2532422571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 msgr2=0x558873661e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x558873661e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 -- 172.21.15.175:0/2532422571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588736624b0 con 0x55887364e980 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0015800640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x558873652990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7f0014e00640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x558873661900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0004005ec0 tx=0x7f0004005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97878999a899fb7e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.923+0000 7efff6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:38:43.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7efff6a00640 1 -- 172.21.15.175:0/2532422571 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000401f070 con 0x55887364e980 2024-07-29T00:38:43.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7efff6a00640 1 -- 172.21.15.175:0/2532422571 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0004003800 con 0x55887364e980 2024-07-29T00:38:43.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0004005510 con 0x55887364e980 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7efff6a00640 1 -- 172.21.15.175:0/2532422571 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0004047050 con 0x55887364e980 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7f001a21c740 10 client.25858 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588736d3640 con 0x55887364e980 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7efff6a00640 1 -- 172.21.15.175:0/2532422571 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0004005510 con 0x55887364e980 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.924+0000 7efff6a00640 1 -- 172.21.15.175:0/2532422571 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0004026cb0 con 0x55887364e980 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f001a21c740 10 client.25858 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f001a21c740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f001a21c740 20 client.25858 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f001a21c740 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588736d3df0 0x5588736d6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f001a21c740 4 client.25858 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588735bc530 con 0x5588736d3df0 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f000fe00640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588736d3df0 0x5588736d6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7f000fe00640 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588736d3df0 0x5588736d6210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa0cdc58dc94656 server_cookie=2d6383458af13472 in_seq=0 out_seq=0 2024-07-29T00:38:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.925+0000 7efff6a00640 10 client.25858 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7efff6a00640 1 -- 172.21.15.175:0/2532422571 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588735bc530 con 0x5588736d3df0 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7efff6a00640 10 client.25858 handle_command_reply: tid=0 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7f001a21c740 1 client.25858 shutdown 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 msgr2=0x558873661900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7f001a21c740 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x558873661900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0004005ec0 tx=0x7f0004005710 comp rx=0 tx=0).stop 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 shutdown_connections 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.926+0000 7f001a21c740 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588736d3df0 0x5588736d6210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558873659720 0x558873652990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55887364fef0 0x558873661e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7efff6a00640 0 client.25858 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 1 --2- 172.21.15.175:0/2532422571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55887364e980 0x558873661900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 >> 172.21.15.175:0/2532422571 conn(0x558873617950 msgr2=0x5588736557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 shutdown_connections 2024-07-29T00:38:43.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 1 -- 172.21.15.175:0/2532422571 wait complete. 2024-07-29T00:38:43.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.927+0000 7f001a21c740 20 client.25858 trim_cache size 0 max 16384 2024-07-29T00:38:43.931 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efff80a2d70 msgr2=0x7efff80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efff80a2d70 0x7efff80a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0008005fd0 tx=0x7f0008009500 comp rx=0 tx=0).stop 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 msgr2=0x7f00101a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f00101a4910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efffc035040 tx=0x7efffc033b60 comp rx=0 tx=0).stop 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 shutdown_connections 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efff80a2d70 0x7efff80a5230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f001010d220 0x7f001019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0010108af0 0x7f00101a4910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 --2- 172.21.15.175:0/1484469923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0010108140 0x7f00101a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 >> 172.21.15.175:0/1484469923 conn(0x7f0010077890 msgr2=0x7f0010078bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.928+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 shutdown_connections 2024-07-29T00:38:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:43.929+0000 7f0017800640 1 -- 172.21.15.175:0/1484469923 wait complete. 2024-07-29T00:38:43.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:43 smithi181 ceph-mon[36706]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 262 KiB/s wr, 55 op/s 2024-07-29T00:38:44.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:43 smithi204 ceph-mon[37368]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 262 KiB/s wr, 55 op/s 2024-07-29T00:38:44.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 43510 7.18 MB/sec warmup 35 sec latency 14.457 ms 2024-07-29T00:38:45.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 45475 7.27 MB/sec warmup 36 sec latency 12.593 ms 2024-07-29T00:38:45.854 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:38:45.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:38:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:45 smithi181 ceph-mon[36706]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-07-29T00:38:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:45 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:38:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:45 smithi175 ceph-mon[32138]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-07-29T00:38:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:45 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:38:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:45 smithi204 ceph-mon[37368]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-07-29T00:38:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:45 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:38:46.356 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:46.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 45962 7.16 MB/sec warmup 37 sec latency 668.958 ms 2024-07-29T00:38:46.951 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:46 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:38:46.951 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:46 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:38:46.951 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:46 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:38:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:46 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:38:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:46 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:38:46.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:46 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:38:47.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:46 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:38:47.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:46 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:38:47.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:46 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:38:47.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 -- 172.21.15.175:0/1538031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580103a80 msgr2=0x7f9580103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 --2- 172.21.15.175:0/1538031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580103a80 0x7f9580103ee0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f956c009930 tx=0x7f956c02a440 comp rx=0 tx=0).stop 2024-07-29T00:38:47.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 -- 172.21.15.175:0/1538031032 shutdown_connections 2024-07-29T00:38:47.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 --2- 172.21.15.175:0/1538031032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580104420 0x7f9580110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 --2- 172.21.15.175:0/1538031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580103a80 0x7f9580103ee0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 --2- 172.21.15.175:0/1538031032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580109a80 0x7f9580109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 -- 172.21.15.175:0/1538031032 >> 172.21.15.175:0/1538031032 conn(0x7f95800ff780 msgr2=0x7f9580101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:47.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 -- 172.21.15.175:0/1538031032 shutdown_connections 2024-07-29T00:38:47.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.127+0000 7f9586400640 1 -- 172.21.15.175:0/1538031032 wait complete. 2024-07-29T00:38:47.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.128+0000 7f9586400640 1 Processor -- start 2024-07-29T00:38:47.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 -- start start 2024-07-29T00:38:47.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580103a80 0x7f958019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580104420 0x7f958019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580109a80 0x7f958019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9580113f30 con 0x7f9580109a80 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9580113db0 con 0x7f9580103a80 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f9586400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95801140b0 con 0x7f9580104420 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f957ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580104420 0x7f958019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f957fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580109a80 0x7f958019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f957ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580104420 0x7f958019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38080/0 (socket says 172.21.15.175:38080) 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f957ea00640 1 -- 172.21.15.175:0/241351449 learned_addr learned my addr 172.21.15.175:0/241351449 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.129+0000 7f957f400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580103a80 0x7f958019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f957fe00640 1 -- 172.21.15.175:0/241351449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580104420 msgr2=0x7f958019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f957fe00640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580104420 0x7f958019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f957fe00640 1 -- 172.21.15.175:0/241351449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580103a80 msgr2=0x7f958019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f957fe00640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580103a80 0x7f958019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f957fe00640 1 -- 172.21.15.175:0/241351449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958019f380 con 0x7f9580109a80 2024-07-29T00:38:47.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f957fe00640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580109a80 0x7f958019eca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f957800be20 tx=0x7f957800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:47.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f9577e00640 1 -- 172.21.15.175:0/241351449 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578019070 con 0x7f9580109a80 2024-07-29T00:38:47.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f9577e00640 1 -- 172.21.15.175:0/241351449 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9578014920 con 0x7f9580109a80 2024-07-29T00:38:47.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.130+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958019f670 con 0x7f9580109a80 2024-07-29T00:38:47.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.131+0000 7f9577e00640 1 -- 172.21.15.175:0/241351449 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578012940 con 0x7f9580109a80 2024-07-29T00:38:47.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.131+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95801a6f50 con 0x7f9580109a80 2024-07-29T00:38:47.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9577e00640 1 -- 172.21.15.175:0/241351449 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f957801c070 con 0x7f9580109a80 2024-07-29T00:38:47.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 Processor -- start 2024-07-29T00:38:47.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 -- start start 2024-07-29T00:38:47.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c151df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c15287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c1528d20 0x5574c152b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574c14838a0 con 0x5574c151e460 2024-07-29T00:38:47.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574c141b710 con 0x5574c151db40 2024-07-29T00:38:47.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.134+0000 7f9589050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5574c147f860 con 0x5574c1528d20 2024-07-29T00:38:47.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c151df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c15287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c151df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57976/0 (socket says 172.21.15.175:57976) 2024-07-29T00:38:47.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957ea00640 1 -- 172.21.15.175:0/824562587 learned_addr learned my addr 172.21.15.175:0/824562587 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:47.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957ea00640 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c1528d20 0x5574c152b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957f400640 1 -- 172.21.15.175:0/824562587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c1528d20 msgr2=0x5574c152b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957f400640 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c1528d20 0x5574c152b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957f400640 1 -- 172.21.15.175:0/824562587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 msgr2=0x5574c151df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957f400640 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c151df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.135+0000 7f957f400640 1 -- 172.21.15.175:0/824562587 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c11f56a0 con 0x5574c151e460 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f9577e00640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95640a2cf0 0x7f95640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f9577e00640 1 -- 172.21.15.175:0/241351449 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f95780cede0 con 0x7f9580109a80 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f957f400640 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c15287e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9570009b70 tx=0x7f95700117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323fa2a3169f22f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:47.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f957ea00640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95640a2cf0 0x7f95640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f9574c00640 1 -- 172.21.15.175:0/824562587 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570009e60 con 0x5574c151e460 2024-07-29T00:38:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f9574c00640 1 -- 172.21.15.175:0/824562587 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570012930 con 0x5574c151e460 2024-07-29T00:38:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.136+0000 7f9574c00640 1 -- 172.21.15.175:0/824562587 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957000fbb0 con 0x5574c151e460 2024-07-29T00:38:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 -- 172.21.15.175:0/824562587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 msgr2=0x5574c15287e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f957ea00640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95640a2cf0 0x7f95640a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f956c0096c0 tx=0x7f956c031400 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c15287e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9570009b70 tx=0x7f95700117b0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 -- 172.21.15.175:0/824562587 shutdown_connections 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c1528d20 0x5574c152b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c15287e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 --2- 172.21.15.175:0/824562587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c151df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 -- 172.21.15.175:0/824562587 >> 172.21.15.175:0/824562587 conn(0x5574c151a0d0 msgr2=0x5574c151a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 -- 172.21.15.175:0/824562587 shutdown_connections 2024-07-29T00:38:47.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.137+0000 7f9589050740 1 -- 172.21.15.175:0/824562587 wait complete. 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.138+0000 7f9589050740 1 -- start start 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.138+0000 7f9589050740 1 Processor -- start 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c1525c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c1526180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c1528d20 0x5574c15266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574c152b8f0 con 0x5574c151e460 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574c143e8c0 con 0x5574c1528d20 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5574c152ba70 con 0x5574c151db40 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f9589050740 10 client.0 _ll_register_callbacks cb 0x5574c142aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f957ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c1526180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f957ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c1526180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54004/0 (socket says 172.21.15.175:54004) 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f957ea00640 1 -- 172.21.15.175:0/2905154572 learned_addr learned my addr 172.21.15.175:0/2905154572 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.139+0000 7f957fe00640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c1528d20 0x5574c15266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957ea00640 1 -- 172.21.15.175:0/2905154572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 msgr2=0x5574c1525c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957ea00640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c1525c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957ea00640 1 -- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c1528d20 msgr2=0x5574c15266c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957ea00640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c1528d20 0x5574c15266c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957ea00640 1 -- 172.21.15.175:0/2905154572 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c1421bc0 con 0x5574c151e460 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957fe00640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c1528d20 0x5574c15266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f957ea00640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c1526180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f956c003ad0 tx=0x7f956c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0a2738b904f19f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f9569600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:38:47.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f9569600640 1 -- 172.21.15.175:0/2905154572 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c04e070 con 0x5574c151e460 2024-07-29T00:38:47.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9570012930 con 0x5574c151e460 2024-07-29T00:38:47.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.140+0000 7f9569600640 1 -- 172.21.15.175:0/2905154572 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f956c0319c0 con 0x5574c151e460 2024-07-29T00:38:47.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9569600640 1 -- 172.21.15.175:0/2905154572 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f956c035040 con 0x5574c151e460 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 10 client.15774 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5574c1537960 con 0x5574c151e460 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9569600640 1 -- 172.21.15.175:0/2905154572 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570012930 con 0x5574c151e460 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9569600640 1 -- 172.21.15.175:0/2905154572 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f956c048040 con 0x5574c151e460 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 10 client.15774 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 20 client.15774 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c1538e80 0x5574c153b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 4 client.15774 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.141+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c1443c90 con 0x5574c1538e80 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.142+0000 7f957f400640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c1538e80 0x5574c153b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.142+0000 7f957f400640 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c1538e80 0x5574c153b2f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7151e58baa24794e server_cookie=ff3ee8c0794da119 in_seq=0 out_seq=0 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.142+0000 7f9569600640 10 client.15774 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:47.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9569600640 1 -- 172.21.15.175:0/2905154572 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5574c1443c90 con 0x5574c1538e80 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9569600640 10 client.15774 handle_command_reply: tid=0 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 client.15774 shutdown 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 msgr2=0x5574c1526180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c1526180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f956c003ad0 tx=0x7f956c005a20 comp rx=0 tx=0).stop 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 shutdown_connections 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c1538e80 0x5574c153b2f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c1528d20 0x5574c15266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c151e460 0x5574c1526180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 --2- 172.21.15.175:0/2905154572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c1525c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.143+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 >> 172.21.15.175:0/2905154572 conn(0x5574c151a0d0 msgr2=0x5574c1519230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.144+0000 7f9569600640 0 client.15774 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.144+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 shutdown_connections 2024-07-29T00:38:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.144+0000 7f9589050740 1 -- 172.21.15.175:0/2905154572 wait complete. 2024-07-29T00:38:47.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.144+0000 7f9589050740 20 client.15774 trim_cache size 0 max 16384 2024-07-29T00:38:47.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.162+0000 7f9589050740 1 Processor -- start 2024-07-29T00:38:47.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.162+0000 7f9589050740 1 -- start start 2024-07-29T00:38:47.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.162+0000 7f9589050740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c15205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.162+0000 7f9589050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151e460 0x5574c1527bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f9589050740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c1540bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f9589050740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95801af820 con 0x5574c1528d20 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f9589050740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95801b1420 con 0x5574c151db40 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f9589050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95801afef0 con 0x5574c151e460 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c1540bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c15205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c15205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57980/0 (socket says 172.21.15.175:57980) 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 -- 172.21.15.175:0/194938503 learned_addr learned my addr 172.21.15.175:0/194938503 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957ea00640 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151e460 0x5574c1527bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 -- 172.21.15.175:0/194938503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151e460 msgr2=0x5574c1527bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151e460 0x5574c1527bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 -- 172.21.15.175:0/194938503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 msgr2=0x5574c1540bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c1540bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.163+0000 7f957f400640 1 -- 172.21.15.175:0/194938503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c1515840 con 0x5574c151db40 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.164+0000 7f957f400640 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c15205a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f957001c040 tx=0x7f9570009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9478c780b92bd84 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.164+0000 7f956be00640 1 -- 172.21.15.175:0/194938503 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957000f8c0 con 0x5574c151db40 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.164+0000 7f956be00640 1 -- 172.21.15.175:0/194938503 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570028070 con 0x5574c151db40 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.164+0000 7f956be00640 1 -- 172.21.15.175:0/194938503 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700035f0 con 0x5574c151db40 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 -- 172.21.15.175:0/194938503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 msgr2=0x5574c15205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c15205a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f957001c040 tx=0x7f9570009700 comp rx=0 tx=0).stop 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 -- 172.21.15.175:0/194938503 shutdown_connections 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c1540bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151e460 0x5574c1527bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 --2- 172.21.15.175:0/194938503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151db40 0x5574c15205a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 -- 172.21.15.175:0/194938503 >> 172.21.15.175:0/194938503 conn(0x5574c151a0d0 msgr2=0x5574c15274a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 -- 172.21.15.175:0/194938503 shutdown_connections 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.165+0000 7f9589050740 1 -- 172.21.15.175:0/194938503 wait complete. 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 -- start start 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 Processor -- start 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c159be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151e460 0x5574c159c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c152c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95801b0c90 con 0x5574c1528d20 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95801b05c0 con 0x5574c151e460 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95801ad800 con 0x5574c151db40 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.166+0000 7f9589050740 10 client.0 _ll_register_callbacks cb 0x7f956c0039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c152c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c159be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151e460 0x5574c159c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c152c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54020/0 (socket says 172.21.15.175:54020) 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 -- 172.21.15.175:0/1790787729 learned_addr learned my addr 172.21.15.175:0/1790787729 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 -- 172.21.15.175:0/1790787729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 msgr2=0x5574c159be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c159be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 -- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151e460 msgr2=0x5574c159c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151e460 0x5574c159c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 -- 172.21.15.175:0/1790787729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c152c990 con 0x5574c1528d20 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957ea00640 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151e460 0x5574c159c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f957fe00640 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c152c2a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f95780977f0 tx=0x7f957800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e054757110867de8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f9569600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.167+0000 7f9569600640 1 -- 172.21.15.175:0/1790787729 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780a4000 con 0x5574c1528d20 2024-07-29T00:38:47.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9569600640 1 -- 172.21.15.175:0/1790787729 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f95780cc730 con 0x5574c1528d20 2024-07-29T00:38:47.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9570028070 con 0x5574c1528d20 2024-07-29T00:38:47.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9569600640 1 -- 172.21.15.175:0/1790787729 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95780a3b70 con 0x5574c1528d20 2024-07-29T00:38:47.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 10 client.15780 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:47.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5574c15a3780 con 0x5574c1528d20 2024-07-29T00:38:47.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9569600640 1 -- 172.21.15.175:0/1790787729 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570028070 con 0x5574c1528d20 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9569600640 1 -- 172.21.15.175:0/1790787729 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9578012b40 con 0x5574c1528d20 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 10 client.15780 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 20 client.15780 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c15a3f30 0x5574c15a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 4 client.15780 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.168+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5574c1490e10 con 0x5574c15a3f30 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.169+0000 7f957f400640 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c15a3f30 0x5574c15a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.169+0000 7f957f400640 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c15a3f30 0x5574c15a6350 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d80104b2f5bc8a6 server_cookie=871fcb05d416d63c in_seq=0 out_seq=0 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.169+0000 7f9569600640 10 client.15780 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.170+0000 7f9569600640 1 -- 172.21.15.175:0/1790787729 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5574c1490e10 con 0x5574c15a3f30 2024-07-29T00:38:47.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.170+0000 7f9569600640 10 client.15780 handle_command_reply: tid=0 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.170+0000 7f9589050740 1 client.15780 shutdown 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 msgr2=0x5574c152c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c152c2a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f95780977f0 tx=0x7f957800ce70 comp rx=0 tx=0).stop 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 shutdown_connections 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5574c15a3f30 0x5574c15a6350 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574c1528d20 0x5574c152c2a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c151e460 0x5574c159c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 --2- 172.21.15.175:0/1790787729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574c151db40 0x5574c159be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9569600640 0 client.15780 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 >> 172.21.15.175:0/1790787729 conn(0x5574c151a0d0 msgr2=0x5574c1527cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 shutdown_connections 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 1 -- 172.21.15.175:0/1790787729 wait complete. 2024-07-29T00:38:47.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.171+0000 7f9589050740 20 client.15780 trim_cache size 0 max 16384 2024-07-29T00:38:47.175 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:38:47.175 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:38:47.175 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "e54c95ae-b281-44ae-afbb-4ec3bc1bb4b1", 2024-07-29T00:38:47.175 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:38:47.175 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:38:47.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95640a2cf0 msgr2=0x7f95640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95640a2cf0 0x7f95640a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f956c0096c0 tx=0x7f956c031400 comp rx=0 tx=0).stop 2024-07-29T00:38:47.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580109a80 msgr2=0x7f958019eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:47.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580109a80 0x7f958019eca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f957800be20 tx=0x7f957800bf10 comp rx=0 tx=0).stop 2024-07-29T00:38:47.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 shutdown_connections 2024-07-29T00:38:47.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95640a2cf0 0x7f95640a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9580109a80 0x7f958019eca0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9580104420 0x7f958019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 --2- 172.21.15.175:0/241351449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580103a80 0x7f958019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:47.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 >> 172.21.15.175:0/241351449 conn(0x7f95800ff780 msgr2=0x7f95800ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:47.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 shutdown_connections 2024-07-29T00:38:47.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:47.172+0000 7f9586400640 1 -- 172.21.15.175:0/241351449 wait complete. 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: pgmap v71: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 93 op/s 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:47.779 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:47 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 45962 6.97 MB/sec warmup 38 sec latency 1052.110 ms 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: pgmap v71: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 93 op/s 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:47.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:47 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: pgmap v71: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 93 op/s 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:47 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:38:48.706 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:38:48.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:48.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:48.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:38:48.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:48 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 46540 6.91 MB/sec warmup 39 sec latency 1363.984 ms 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:48 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:49.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:49.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:49.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:49.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:49.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:38:49.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:49.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:48 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 -- 172.21.15.175:0/2649631143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590103aa0 msgr2=0x7f4590103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 --2- 172.21.15.175:0/2649631143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590103aa0 0x7f4590103f00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4580009900 tx=0x7f458002a410 comp rx=0 tx=0).stop 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 -- 172.21.15.175:0/2649631143 shutdown_connections 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 --2- 172.21.15.175:0/2649631143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f4590110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 --2- 172.21.15.175:0/2649631143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590103aa0 0x7f4590103f00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 --2- 172.21.15.175:0/2649631143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4590109aa0 0x7f4590109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 -- 172.21.15.175:0/2649631143 >> 172.21.15.175:0/2649631143 conn(0x7f45900ff740 msgr2=0x7f4590101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:49.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 -- 172.21.15.175:0/2649631143 shutdown_connections 2024-07-29T00:38:49.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.384+0000 7f4598000640 1 -- 172.21.15.175:0/2649631143 wait complete. 2024-07-29T00:38:49.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.385+0000 7f4598000640 1 Processor -- start 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 -- start start 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4590103aa0 0x7f459019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f459019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590109aa0 0x7f459019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4590113ef0 con 0x7f4590103aa0 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4590113d70 con 0x7f4590109aa0 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4598000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4590114070 con 0x7f4590104440 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4594c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f459019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4596000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590109aa0 0x7f459019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4595600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4590103aa0 0x7f459019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4594c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f459019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:59236/0 (socket says 172.21.15.175:59236) 2024-07-29T00:38:49.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.386+0000 7f4594c00640 1 -- 172.21.15.175:0/4280575970 learned_addr learned my addr 172.21.15.175:0/4280575970 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:49.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4594c00640 1 -- 172.21.15.175:0/4280575970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590109aa0 msgr2=0x7f459019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4594c00640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590109aa0 0x7f459019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4594c00640 1 -- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4590103aa0 msgr2=0x7f459019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4594c00640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4590103aa0 0x7f459019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4594c00640 1 -- 172.21.15.175:0/4280575970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459019f320 con 0x7f4590104440 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4596000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590109aa0 0x7f459019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.387+0000 7f4594c00640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f459019a8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f458002aa00 tx=0x7f4580003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.388+0000 7f458e000640 1 -- 172.21.15.175:0/4280575970 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458003f070 con 0x7f4590104440 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.388+0000 7f458e000640 1 -- 172.21.15.175:0/4280575970 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4580031c30 con 0x7f4590104440 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.388+0000 7f458e000640 1 -- 172.21.15.175:0/4280575970 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580035030 con 0x7f4590104440 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.388+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459019f5b0 con 0x7f4590104440 2024-07-29T00:38:49.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.388+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45901a6ef0 con 0x7f4590104440 2024-07-29T00:38:49.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.391+0000 7f458e000640 1 -- 172.21.15.175:0/4280575970 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4580042070 con 0x7f4590104440 2024-07-29T00:38:49.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 Processor -- start 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 -- start start 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f486d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f4915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f493f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c35f3ebad0 con 0x55c35f487250 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c35f383af0 con 0x55c35f486930 2024-07-29T00:38:49.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f459aa8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c35f3e7a90 con 0x55c35f491b10 2024-07-29T00:38:49.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f4596000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f486d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f4596000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f486d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56008/0 (socket says 172.21.15.175:56008) 2024-07-29T00:38:49.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.392+0000 7f4596000640 1 -- 172.21.15.175:0/3249519448 learned_addr learned my addr 172.21.15.175:0/3249519448 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:49.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.393+0000 7f4594c00640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f493f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.393+0000 7f4596000640 1 -- 172.21.15.175:0/3249519448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 msgr2=0x55c35f493f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:49.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.393+0000 7f4596000640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f493f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.393+0000 7f4596000640 1 -- 172.21.15.175:0/3249519448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 msgr2=0x55c35f4915d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:38:49.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.393+0000 7f458e000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45780a2d70 0x7f45780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f458e000640 1 -- 172.21.15.175:0/4280575970 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f45800efa30 con 0x7f4590104440 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4595600640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f4915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4596000640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f4915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4596000640 1 -- 172.21.15.175:0/3249519448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c35f15d3c0 con 0x55c35f486930 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4594c00640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f493f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4595600640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f4915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4596000640 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f486d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f45880117e0 tx=0x7f4588011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba954e2b562ba7a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f4594c00640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45780a2d70 0x7f45780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.394+0000 7f457e600640 1 -- 172.21.15.175:0/3249519448 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588011310 con 0x55c35f486930 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f457e600640 1 -- 172.21.15.175:0/3249519448 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4588003e50 con 0x55c35f486930 2024-07-29T00:38:49.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f457e600640 1 -- 172.21.15.175:0/3249519448 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588004620 con 0x55c35f486930 2024-07-29T00:38:49.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 -- 172.21.15.175:0/3249519448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 msgr2=0x55c35f486d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f4594c00640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45780a2d70 0x7f45780a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55c35f484240 tx=0x7f4580009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:49.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f486d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f45880117e0 tx=0x7f4588011ca0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 -- 172.21.15.175:0/3249519448 shutdown_connections 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f493f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f4915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 --2- 172.21.15.175:0/3249519448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f486d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.395+0000 7f459aa8d740 1 -- 172.21.15.175:0/3249519448 >> 172.21.15.175:0/3249519448 conn(0x55c35f482ec0 msgr2=0x55c35f4832a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.396+0000 7f459aa8d740 1 -- 172.21.15.175:0/3249519448 shutdown_connections 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.396+0000 7f459aa8d740 1 -- 172.21.15.175:0/3249519448 wait complete. 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.396+0000 7f459aa8d740 1 -- start start 2024-07-29T00:38:49.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.396+0000 7f459aa8d740 1 Processor -- start 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f486930 0x55c35f48e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 0x55c35f48ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f491b10 0x55c35f48f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c35f4946e0 con 0x55c35f491b10 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c35f3a6bc0 con 0x55c35f487250 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c35f494860 con 0x55c35f486930 2024-07-29T00:38:49.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f459aa8d740 10 client.0 _ll_register_callbacks cb 0x55c35f392d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f4596000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f491b10 0x55c35f48f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.397+0000 7f4595600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 0x55c35f48ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 0x55c35f48ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56022/0 (socket says 172.21.15.175:56022) 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 -- 172.21.15.175:0/4142524526 learned_addr learned my addr 172.21.15.175:0/4142524526 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 -- 172.21.15.175:0/4142524526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f486930 msgr2=0x55c35f48e9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f486930 0x55c35f48e9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 -- 172.21.15.175:0/4142524526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f491b10 msgr2=0x55c35f48f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f491b10 0x55c35f48f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4595600640 1 -- 172.21.15.175:0/4142524526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c35f49dbd0 con 0x55c35f487250 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.398+0000 7f4596000640 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f491b10 0x55c35f48f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.399+0000 7f4595600640 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 0x55c35f48ef30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4584013710 tx=0x7f4584009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a1a513c2334ca6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.399+0000 7f4572a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.399+0000 7f4572a00640 1 -- 172.21.15.175:0/4142524526 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584004460 con 0x55c35f487250 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.400+0000 7f4572a00640 1 -- 172.21.15.175:0/4142524526 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f45840119c0 con 0x55c35f487250 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.400+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4588003e50 con 0x55c35f487250 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.400+0000 7f4572a00640 1 -- 172.21.15.175:0/4142524526 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4584011bc0 con 0x55c35f487250 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.400+0000 7f459aa8d740 10 client.25901 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:49.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.400+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c35f4a0530 con 0x55c35f487250 2024-07-29T00:38:49.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.400+0000 7f4572a00640 1 -- 172.21.15.175:0/4142524526 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4588003e50 con 0x55c35f487250 2024-07-29T00:38:49.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f4572a00640 1 -- 172.21.15.175:0/4142524526 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f458401a070 con 0x55c35f487250 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f459aa8d740 10 client.25901 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f459aa8d740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f459aa8d740 20 client.25901 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f459aa8d740 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f4a1ab0 0x55c35f4a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f459aa8d740 4 client.25901 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c35f3f9040 con 0x55c35f4a1ab0 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.401+0000 7f4596000640 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f4a1ab0 0x55c35f4a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.402+0000 7f4572a00640 10 client.25901 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:49.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.402+0000 7f4596000640 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f4a1ab0 0x55c35f4a3f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40aecd2425b4448 server_cookie=fe5e4d43c8cf33d in_seq=0 out_seq=0 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f4572a00640 1 -- 172.21.15.175:0/4142524526 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c35f3f9040 con 0x55c35f4a1ab0 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f4572a00640 10 client.25901 handle_command_reply: tid=0 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 client.25901 shutdown 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 msgr2=0x55c35f48ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 0x55c35f48ef30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4584013710 tx=0x7f4584009f90 comp rx=0 tx=0).stop 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 shutdown_connections 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f4a1ab0 0x55c35f4a3f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f491b10 0x55c35f48f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f487250 0x55c35f48ef30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.403+0000 7f459aa8d740 1 --2- 172.21.15.175:0/4142524526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f486930 0x55c35f48e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.404+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 >> 172.21.15.175:0/4142524526 conn(0x55c35f482ec0 msgr2=0x55c35f488bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:49.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.404+0000 7f4572a00640 0 client.25901 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:49.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.404+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 shutdown_connections 2024-07-29T00:38:49.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.404+0000 7f459aa8d740 1 -- 172.21.15.175:0/4142524526 wait complete. 2024-07-29T00:38:49.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.404+0000 7f459aa8d740 20 client.25901 trim_cache size 0 max 16384 2024-07-29T00:38:49.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 Processor -- start 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 -- start start 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f4893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f48f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f4a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45901af8c0 con 0x55c35f487250 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45901b14c0 con 0x55c35f486930 2024-07-29T00:38:49.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.424+0000 7f459aa8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45901aff90 con 0x55c35f491b10 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f48f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f48f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37502/0 (socket says 172.21.15.175:37502) 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 -- 172.21.15.175:0/2654223054 learned_addr learned my addr 172.21.15.175:0/2654223054 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4596000640 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f4a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4596000640 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f4893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 -- 172.21.15.175:0/2654223054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 msgr2=0x55c35f4a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:49.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f4a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 -- 172.21.15.175:0/2654223054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 msgr2=0x55c35f4893e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:49.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f4893e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.425+0000 7f4595600640 1 -- 172.21.15.175:0/2654223054 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c35f48cf40 con 0x55c35f487250 2024-07-29T00:38:49.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.426+0000 7f4595600640 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f48f2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f458401c730 tx=0x7f4584011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710bee2c48555a91 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:49.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.426+0000 7f457dc00640 1 -- 172.21.15.175:0/2654223054 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458401c920 con 0x55c35f487250 2024-07-29T00:38:49.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.426+0000 7f457dc00640 1 -- 172.21.15.175:0/2654223054 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458401cac0 con 0x55c35f487250 2024-07-29T00:38:49.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.426+0000 7f457dc00640 1 -- 172.21.15.175:0/2654223054 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584024420 con 0x55c35f487250 2024-07-29T00:38:49.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 -- 172.21.15.175:0/2654223054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 msgr2=0x55c35f48f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f48f2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f458401c730 tx=0x7f4584011b50 comp rx=0 tx=0).stop 2024-07-29T00:38:49.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 -- 172.21.15.175:0/2654223054 shutdown_connections 2024-07-29T00:38:49.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f491b10 0x55c35f4a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f487250 0x55c35f48f2c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 --2- 172.21.15.175:0/2654223054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f486930 0x55c35f4893e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 -- 172.21.15.175:0/2654223054 >> 172.21.15.175:0/2654223054 conn(0x55c35f482ec0 msgr2=0x55c35f48dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 -- 172.21.15.175:0/2654223054 shutdown_connections 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.427+0000 7f459aa8d740 1 -- 172.21.15.175:0/2654223054 wait complete. 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.428+0000 7f459aa8d740 1 -- start start 2024-07-29T00:38:49.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.428+0000 7f459aa8d740 1 Processor -- start 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 0x55c35f495060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f487250 0x55c35f4955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f491b10 0x55c35f50a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45901b0d30 con 0x55c35f486930 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45901b0660 con 0x55c35f491b10 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45901ad8a0 con 0x55c35f487250 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f459aa8d740 10 client.0 _ll_register_callbacks cb 0x7f4584009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 0x55c35f495060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4595600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f487250 0x55c35f4955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 0x55c35f495060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37512/0 (socket says 172.21.15.175:37512) 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 -- 172.21.15.175:0/680066345 learned_addr learned my addr 172.21.15.175:0/680066345 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4595600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f487250 0x55c35f4955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:59280/0 (socket says 172.21.15.175:59280) 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f491b10 0x55c35f50a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 -- 172.21.15.175:0/680066345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f487250 msgr2=0x55c35f4955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f487250 0x55c35f4955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 -- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f491b10 msgr2=0x55c35f50a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f491b10 0x55c35f50a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.429+0000 7f4596000640 1 -- 172.21.15.175:0/680066345 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c35f50ad40 con 0x55c35f486930 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.430+0000 7f4596000640 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 0x55c35f495060 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4588004130 tx=0x7f4588011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552212d38b7d098f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.430+0000 7f4572a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.430+0000 7f4572a00640 1 -- 172.21.15.175:0/680066345 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458802d040 con 0x55c35f486930 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.430+0000 7f4572a00640 1 -- 172.21.15.175:0/680066345 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f45880052f0 con 0x55c35f486930 2024-07-29T00:38:49.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.430+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458401cac0 con 0x55c35f486930 2024-07-29T00:38:49.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.430+0000 7f4572a00640 1 -- 172.21.15.175:0/680066345 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f458800fe70 con 0x55c35f486930 2024-07-29T00:38:49.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 10 client.15795 fetch_fsmap learned FSMap version 31 2024-07-29T00:38:49.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c35f50c030 con 0x55c35f486930 2024-07-29T00:38:49.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f4572a00640 1 -- 172.21.15.175:0/680066345 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458401cac0 con 0x55c35f486930 2024-07-29T00:38:49.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f4572a00640 1 -- 172.21.15.175:0/680066345 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4588033040 con 0x55c35f486930 2024-07-29T00:38:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 10 client.15795 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:38:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 20 client.15795 populate_metadata read hostname 'smithi175' 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f50c5d0 0x55c35f50e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 4 client.15795 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.431+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c35f3f3550 con 0x55c35f50c5d0 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.432+0000 7f4595600640 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f50c5d0 0x55c35f50e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.432+0000 7f4595600640 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f50c5d0 0x55c35f50e9f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bff45d3f8866c1f server_cookie=bf8879df33d36ac3 in_seq=0 out_seq=0 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.432+0000 7f4572a00640 10 client.15795 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f4572a00640 1 -- 172.21.15.175:0/680066345 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c35f3f3550 con 0x55c35f50c5d0 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f4572a00640 10 client.15795 handle_command_reply: tid=0 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 client.15795 shutdown 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 msgr2=0x55c35f495060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 0x55c35f495060 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4588004130 tx=0x7f4588011ca0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 shutdown_connections 2024-07-29T00:38:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c35f50c5d0 0x55c35f50e9f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c35f491b10 0x55c35f50a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35f487250 0x55c35f4955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 --2- 172.21.15.175:0/680066345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c35f486930 0x55c35f495060 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 >> 172.21.15.175:0/680066345 conn(0x55c35f482ec0 msgr2=0x55c35f481670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f4572a00640 0 client.15795 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 shutdown_connections 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 1 -- 172.21.15.175:0/680066345 wait complete. 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.433+0000 7f459aa8d740 20 client.15795 trim_cache size 0 max 16384 2024-07-29T00:38:49.436 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: "e54c95ae-b281-44ae-afbb-4ec3bc1bb4b1": { 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "e54c95ae-b281-44ae-afbb-4ec3bc1bb4b1", 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:38:49.437 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45780a2d70 msgr2=0x7f45780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45780a2d70 0x7f45780a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55c35f484240 tx=0x7f4580009260 comp rx=0 tx=0).stop 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 msgr2=0x7f459019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f459019a8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f458002aa00 tx=0x7f4580003c10 comp rx=0 tx=0).stop 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 shutdown_connections 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4590109aa0 0x7f459019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4590104440 0x7f459019a8b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45780a2d70 0x7f45780a5230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 --2- 172.21.15.175:0/4280575970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4590103aa0 0x7f459019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:38:49.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.434+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 >> 172.21.15.175:0/4280575970 conn(0x7f45900ff740 msgr2=0x7f45900ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:38:49.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.435+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 shutdown_connections 2024-07-29T00:38:49.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:38:49.435+0000 7f4598000640 1 -- 172.21.15.175:0/4280575970 wait complete. 2024-07-29T00:38:49.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 47677 6.87 MB/sec warmup 40 sec latency 18.308 ms 2024-07-29T00:38:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:49 smithi181 ceph-mon[36706]: pgmap v72: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.1 MiB/s wr, 85 op/s 2024-07-29T00:38:50.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:49 smithi175 ceph-mon[32138]: pgmap v72: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.1 MiB/s wr, 85 op/s 2024-07-29T00:38:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:49 smithi204 ceph-mon[37368]: pgmap v72: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.1 MiB/s wr, 85 op/s 2024-07-29T00:38:50.518 INFO:tasks.cephfs.filesystem:scrub status for tag:e54c95ae-b281-44ae-afbb-4ec3bc1bb4b1 - {'path': '/', 'tag': 'e54c95ae-b281-44ae-afbb-4ec3bc1bb4b1', 'options': 'recursive,force'} 2024-07-29T00:38:50.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 49493 6.93 MB/sec warmup 41 sec latency 14.091 ms 2024-07-29T00:38:51.799 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:51 smithi175 ceph-mon[32138]: pgmap v73: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.1 MiB/s wr, 85 op/s 2024-07-29T00:38:51.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 49891 6.87 MB/sec warmup 42 sec latency 546.308 ms 2024-07-29T00:38:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:51 smithi181 ceph-mon[36706]: pgmap v73: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.1 MiB/s wr, 85 op/s 2024-07-29T00:38:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:51 smithi204 ceph-mon[37368]: pgmap v73: 97 pgs: 97 active+clean; 43 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.1 MiB/s wr, 85 op/s 2024-07-29T00:38:52.850 INFO:tasks.workunit.client.0.smithi175.stdout: 1 49891 6.71 MB/sec warmup 43 sec latency 1546.406 ms 2024-07-29T00:38:52.981 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:52 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:38:52.981 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:52 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:38:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:52 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:38:52.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:52 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:38:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:52 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:38:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:52 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:38:53.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 49932 6.55 MB/sec warmup 44 sec latency 2365.780 ms 2024-07-29T00:38:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:53 smithi181 ceph-mon[36706]: pgmap v74: 97 pgs: 97 active+clean; 43 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-07-29T00:38:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:53 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:38:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:53 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:38:54.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:53 smithi175 ceph-mon[32138]: pgmap v74: 97 pgs: 97 active+clean; 43 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-07-29T00:38:54.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:53 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:38:54.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:53 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:38:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:53 smithi204 ceph-mon[37368]: pgmap v74: 97 pgs: 97 active+clean; 43 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-07-29T00:38:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:53 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:38:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:53 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:38:54.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 50365 6.42 MB/sec warmup 45 sec latency 25.479 ms 2024-07-29T00:38:55.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 52121 6.43 MB/sec warmup 46 sec latency 14.705 ms 2024-07-29T00:38:55.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:55 smithi181 ceph-mon[36706]: pgmap v75: 97 pgs: 97 active+clean; 43 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-07-29T00:38:56.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:55 smithi175 ceph-mon[32138]: pgmap v75: 97 pgs: 97 active+clean; 43 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-07-29T00:38:56.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:55 smithi204 ceph-mon[37368]: pgmap v75: 97 pgs: 97 active+clean; 43 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-07-29T00:38:56.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 52950 6.45 MB/sec warmup 47 sec latency 474.747 ms 2024-07-29T00:38:57.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:57 smithi175 ceph-mon[32138]: pgmap v76: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.0 MiB/s wr, 211 op/s 2024-07-29T00:38:57.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 52962 6.31 MB/sec warmup 48 sec latency 945.324 ms 2024-07-29T00:38:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:57 smithi181 ceph-mon[36706]: pgmap v76: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.0 MiB/s wr, 211 op/s 2024-07-29T00:38:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:57 smithi204 ceph-mon[37368]: pgmap v76: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.0 MiB/s wr, 211 op/s 2024-07-29T00:38:58.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 53452 6.27 MB/sec warmup 49 sec latency 1432.357 ms 2024-07-29T00:38:59.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 53774 6.16 MB/sec warmup 50 sec latency 24.292 ms 2024-07-29T00:38:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:38:59 smithi181 ceph-mon[36706]: pgmap v77: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-07-29T00:39:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:38:59 smithi175 ceph-mon[32138]: pgmap v77: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-07-29T00:39:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:38:59 smithi204 ceph-mon[37368]: pgmap v77: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-07-29T00:39:00.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 55280 6.15 MB/sec warmup 51 sec latency 14.480 ms 2024-07-29T00:39:01.767 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:01 smithi175 ceph-mon[32138]: pgmap v78: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-07-29T00:39:01.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 55614 6.05 MB/sec warmup 52 sec latency 285.030 ms 2024-07-29T00:39:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:01 smithi181 ceph-mon[36706]: pgmap v78: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-07-29T00:39:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:01 smithi204 ceph-mon[37368]: pgmap v78: 97 pgs: 97 active+clean; 44 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-07-29T00:39:02.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 56746 6.14 MB/sec warmup 53 sec latency 136.437 ms 2024-07-29T00:39:03.701 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.5 2024-07-29T00:39:03.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:39:03.851 INFO:tasks.workunit.client.0.smithi175.stdout: 1 57187 6.05 MB/sec warmup 54 sec latency 66.566 ms 2024-07-29T00:39:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:03 smithi181 ceph-mon[36706]: pgmap v79: 97 pgs: 97 active+clean; 44 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.8 MiB/s wr, 217 op/s 2024-07-29T00:39:04.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:03 smithi175 ceph-mon[32138]: pgmap v79: 97 pgs: 97 active+clean; 44 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.8 MiB/s wr, 217 op/s 2024-07-29T00:39:04.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:03 smithi204 ceph-mon[37368]: pgmap v79: 97 pgs: 97 active+clean; 44 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.8 MiB/s wr, 217 op/s 2024-07-29T00:39:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:04.852 INFO:tasks.workunit.client.0.smithi175.stdout: 1 58001 5.98 MB/sec warmup 55 sec latency 97.843 ms 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 -- 172.21.15.175:0/2467890839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe70100ff0 msgr2=0x7fbe7010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 --2- 172.21.15.175:0/2467890839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe70100ff0 0x7fbe7010dcb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009900 tx=0x7fbe5c02a410 comp rx=0 tx=0).stop 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 -- 172.21.15.175:0/2467890839 shutdown_connections 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 --2- 172.21.15.175:0/2467890839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7010e1f0 0x7fbe701105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 --2- 172.21.15.175:0/2467890839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe70100ff0 0x7fbe7010dcb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 --2- 172.21.15.175:0/2467890839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe70100640 0x7fbe70100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:04.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 -- 172.21.15.175:0/2467890839 >> 172.21.15.175:0/2467890839 conn(0x7fbe700fc490 msgr2=0x7fbe700fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:04.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 -- 172.21.15.175:0/2467890839 shutdown_connections 2024-07-29T00:39:04.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.866+0000 7fbe78000640 1 -- 172.21.15.175:0/2467890839 wait complete. 2024-07-29T00:39:04.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 Processor -- start 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 -- start start 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 0x7fbe7019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe70100ff0 0x7fbe7019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe7010e1f0 0x7fbe7019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70113ee0 con 0x7fbe70100ff0 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70113d60 con 0x7fbe7010e1f0 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.868+0000 7fbe78000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70114060 con 0x7fbe70100640 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 0x7fbe7019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:04.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 0x7fbe7019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53646/0 (socket says 172.21.15.175:53646) 2024-07-29T00:39:04.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 -- 172.21.15.175:0/3582842136 learned_addr learned my addr 172.21.15.175:0/3582842136 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:04.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe76000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe7010e1f0 0x7fbe7019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:04.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe74c00640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe70100ff0 0x7fbe7019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:04.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 -- 172.21.15.175:0/3582842136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe7010e1f0 msgr2=0x7fbe7019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe7010e1f0 0x7fbe7019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 -- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe70100ff0 msgr2=0x7fbe7019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe70100ff0 0x7fbe7019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 -- 172.21.15.175:0/3582842136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7019f350 con 0x7fbe70100640 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe76000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe7010e1f0 0x7fbe7019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.869+0000 7fbe75600640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 0x7fbe7019a3a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400ebc0 tx=0x7fbe6400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.870+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64010630 con 0x7fbe70100640 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.870+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe64010cb0 con 0x7fbe70100640 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.870+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64018bd0 con 0x7fbe70100640 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.870+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7019f640 con 0x7fbe70100640 2024-07-29T00:39:04.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.870+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a6f20 con 0x7fbe70100640 2024-07-29T00:39:04.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.871+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7006ace0 con 0x7fbe70100640 2024-07-29T00:39:04.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.873+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fbe640107d0 con 0x7fbe70100640 2024-07-29T00:39:04.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.874+0000 7fbe6e000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbe580a2d70 0x7fbe580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:04.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.875+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fbe640cfa70 con 0x7fbe70100640 2024-07-29T00:39:04.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.875+0000 7fbe74c00640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbe580a2d70 0x7fbe580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:04.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.876+0000 7fbe74c00640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbe580a2d70 0x7fbe580a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c000c00 tx=0x7fbe5c005b50 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:04.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:04.878+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbe6409bfc0 con 0x7fbe70100640 2024-07-29T00:39:05.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.148+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe70109380 con 0x7fbe70100640 2024-07-29T00:39:05.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.149+0000 7fbe6e000640 1 -- 172.21.15.175:0/3582842136 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7fbe640a02c0 con 0x7fbe70100640 2024-07-29T00:39:05.149 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:39:05.149 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:39:05.150 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:39:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbe580a2d70 msgr2=0x7fbe580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbe580a2d70 0x7fbe580a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c000c00 tx=0x7fbe5c005b50 comp rx=0 tx=0).stop 2024-07-29T00:39:05.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 msgr2=0x7fbe7019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 0x7fbe7019a3a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400ebc0 tx=0x7fbe6400c6a0 comp rx=0 tx=0).stop 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 shutdown_connections 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe7010e1f0 0x7fbe7019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbe580a2d70 0x7fbe580a5230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe70100ff0 0x7fbe7019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 --2- 172.21.15.175:0/3582842136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe70100640 0x7fbe7019a3a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 >> 172.21.15.175:0/3582842136 conn(0x7fbe700fc490 msgr2=0x7fbe700fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 shutdown_connections 2024-07-29T00:39:05.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:05.153+0000 7fbe78000640 1 -- 172.21.15.175:0/3582842136 wait complete. 2024-07-29T00:39:05.852 INFO:tasks.workunit.client.0.smithi175.stdout: 1 59952 6.07 MB/sec warmup 56 sec latency 21.437 ms 2024-07-29T00:39:06.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:05 smithi175 ceph-mon[32138]: pgmap v80: 97 pgs: 97 active+clean; 44 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-07-29T00:39:06.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:05 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3582842136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:05 smithi204 ceph-mon[37368]: pgmap v80: 97 pgs: 97 active+clean; 44 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-07-29T00:39:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:05 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3582842136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:06.145 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 90 --expiration 138.5 --await-for 45.0' 2024-07-29T00:39:06.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db / --timeout 90 --expiration 138.5 --await-for 45.0 2024-07-29T00:39:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:05 smithi181 ceph-mon[36706]: pgmap v80: 97 pgs: 97 active+clean; 44 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-07-29T00:39:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:05 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3582842136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:06.625 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:06.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 60755 6.07 MB/sec warmup 57 sec latency 18.679 ms 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 -- 172.21.15.175:0/801142556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18102220 msgr2=0x7f9f1810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 --2- 172.21.15.175:0/801142556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18102220 0x7f9f1810e750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009c10 tx=0x7f9f1002a6f0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 -- 172.21.15.175:0/801142556 shutdown_connections 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 --2- 172.21.15.175:0/801142556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18102220 0x7f9f1810e750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 --2- 172.21.15.175:0/801142556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f18101880 0x7f9f18101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 --2- 172.21.15.175:0/801142556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18107880 0x7f9f18107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 -- 172.21.15.175:0/801142556 >> 172.21.15.175:0/801142556 conn(0x7f9f180fd5a0 msgr2=0x7f9f180ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:07.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.296+0000 7f9f1f600640 1 -- 172.21.15.175:0/801142556 shutdown_connections 2024-07-29T00:39:07.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.297+0000 7f9f1f600640 1 -- 172.21.15.175:0/801142556 wait complete. 2024-07-29T00:39:07.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.298+0000 7f9f1f600640 1 Processor -- start 2024-07-29T00:39:07.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.298+0000 7f9f1f600640 1 -- start start 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.298+0000 7f9f1f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18101880 0x7f9f18198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.298+0000 7f9f1f600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 0x7f9f18198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f1f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f18107880 0x7f9f1819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f1f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18111c80 con 0x7f9f18107880 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f1f600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18111b00 con 0x7f9f18102220 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f1f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18111e00 con 0x7f9f18101880 2024-07-29T00:39:07.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 0x7f9f18198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 0x7f9f18198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48758/0 (socket says 172.21.15.175:48758) 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 -- 172.21.15.175:0/3520416786 learned_addr learned my addr 172.21.15.175:0/3520416786 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f1d600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f18107880 0x7f9f1819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 -- 172.21.15.175:0/3520416786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18101880 msgr2=0x7f9f18198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18101880 0x7f9f18198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 -- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f18107880 msgr2=0x7f9f1819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f18107880 0x7f9f1819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.299+0000 7f9f17e00640 1 -- 172.21.15.175:0/3520416786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1819d0b0 con 0x7f9f18102220 2024-07-29T00:39:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.300+0000 7f9f17e00640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 0x7f9f18198640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e9c0 tx=0x7f9f0800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.300+0000 7f9f15600640 1 -- 172.21.15.175:0/3520416786 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800cde0 con 0x7f9f18102220 2024-07-29T00:39:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.300+0000 7f9f15600640 1 -- 172.21.15.175:0/3520416786 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f080035f0 con 0x7f9f18102220 2024-07-29T00:39:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.300+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1819d3a0 con 0x7f9f18102220 2024-07-29T00:39:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.301+0000 7f9f15600640 1 -- 172.21.15.175:0/3520416786 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08010690 con 0x7f9f18102220 2024-07-29T00:39:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.301+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f181a4c80 con 0x7f9f18102220 2024-07-29T00:39:07.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.304+0000 7f9f15600640 1 -- 172.21.15.175:0/3520416786 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9f080037a0 con 0x7f9f18102220 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.304+0000 7f9f220c9740 1 Processor -- start 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.304+0000 7f9f220c9740 1 -- start start 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.305+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc4e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.305+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4e630 0x55c0ccc4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.305+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc63ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.305+0000 7f9f220c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c0ccbb3a80 con 0x55c0ccc61ad0 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.305+0000 7f9f220c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0ccb4b080 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.305+0000 7f9f220c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0ccbafa40 con 0x55c0ccc4e630 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.306+0000 7f9f1d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc4e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.306+0000 7f9f1d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc4e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48768/0 (socket says 172.21.15.175:48768) 2024-07-29T00:39:07.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.306+0000 7f9f1d600640 1 -- 172.21.15.175:0/3575845893 learned_addr learned my addr 172.21.15.175:0/3575845893 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:07.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.306+0000 7f9f17e00640 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc63ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f1d600640 1 -- 172.21.15.175:0/3575845893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4e630 msgr2=0x55c0ccc4ea90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:39:07.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f15600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9f000a2d70 0x7f9f000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f1d600640 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4e630 0x55c0ccc4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f1d600640 1 -- 172.21.15.175:0/3575845893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 msgr2=0x55c0ccc63ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f1d600640 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc63ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f1d600640 1 -- 172.21.15.175:0/3575845893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0cc925b90 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f15600640 1 -- 172.21.15.175:0/3520416786 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f9f080cf800 con 0x7f9f18102220 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.307+0000 7f9f17e00640 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc63ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.308+0000 7f9f1d600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9f000a2d70 0x7f9f000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.308+0000 7f9f1d600640 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc4e0f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009be0 tx=0x7f9f10030a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c6726755e97a7d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.308+0000 7f9f05c00640 1 -- 172.21.15.175:0/3575845893 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10041040 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.308+0000 7f9f05c00640 1 -- 172.21.15.175:0/3575845893 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f10003e50 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.308+0000 7f9f05c00640 1 -- 172.21.15.175:0/3575845893 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10036040 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f220c9740 1 -- 172.21.15.175:0/3575845893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 msgr2=0x55c0ccc4e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f220c9740 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc4e0f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009be0 tx=0x7f9f10030a30 comp rx=0 tx=0).stop 2024-07-29T00:39:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f1d600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9f000a2d70 0x7f9f000a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f10005f00 tx=0x7f9f1003d460 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f220c9740 1 -- 172.21.15.175:0/3575845893 shutdown_connections 2024-07-29T00:39:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f220c9740 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc63ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f220c9740 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4e630 0x55c0ccc4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.309+0000 7f9f220c9740 1 --2- 172.21.15.175:0/3575845893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc4e0f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.310+0000 7f9f220c9740 1 -- 172.21.15.175:0/3575845893 >> 172.21.15.175:0/3575845893 conn(0x55c0ccc4a800 msgr2=0x55c0ccc4abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.310+0000 7f9f220c9740 1 -- 172.21.15.175:0/3575845893 shutdown_connections 2024-07-29T00:39:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.310+0000 7f9f220c9740 1 -- 172.21.15.175:0/3575845893 wait complete. 2024-07-29T00:39:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.311+0000 7f9f220c9740 1 -- start start 2024-07-29T00:39:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.311+0000 7f9f220c9740 1 Processor -- start 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.311+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc51f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f220c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c0ccc647d0 con 0x55c0ccc61ad0 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f220c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0ccb6e750 con 0x55c0ccc4e630 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f220c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0ccc64950 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f220c9740 10 client.0 _ll_register_callbacks cb 0x55c0cc8c3090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc51f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc51f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48774/0 (socket says 172.21.15.175:48774) 2024-07-29T00:39:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f1cc00640 1 -- 172.21.15.175:0/2266043504 learned_addr learned my addr 172.21.15.175:0/2266043504 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f1d600640 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.312+0000 7f9f1cc00640 1 -- 172.21.15.175:0/2266043504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 msgr2=0x55c0ccc52490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:39:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.313+0000 7f9f1cc00640 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.313+0000 7f9f1cc00640 1 -- 172.21.15.175:0/2266043504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 msgr2=0x55c0ccc529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.313+0000 7f9f1cc00640 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.313+0000 7f9f1cc00640 1 -- 172.21.15.175:0/2266043504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ccc6ddb0 con 0x55c0ccc4e630 2024-07-29T00:39:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.313+0000 7f9f1cc00640 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc51f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e600 tx=0x7f9f0c00eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a25c0e600bc9a0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.313+0000 7f9efa000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9efa000640 1 -- 172.21.15.175:0/2266043504 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c005220 con 0x55c0ccc4e630 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9efa000640 1 -- 172.21.15.175:0/2266043504 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f9f0c0053c0 con 0x55c0ccc4e630 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f10003e50 con 0x55c0ccc4e630 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9efa000640 1 -- 172.21.15.175:0/2266043504 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9f0c0055e0 con 0x55c0ccc4e630 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9f220c9740 10 client.25925 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0ccc70740 con 0x55c0ccc4e630 2024-07-29T00:39:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.314+0000 7f9efa000640 1 -- 172.21.15.175:0/2266043504 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f10003e50 con 0x55c0ccc4e630 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9efa000640 1 -- 172.21.15.175:0/2266043504 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9f0c018430 con 0x55c0ccc4e630 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f220c9740 10 client.25925 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f220c9740 10 client.25925 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f220c9740 20 client.25925 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f220c9740 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0ccc71cd0 0x55c0ccc74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f220c9740 4 client.25925 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0ccbc11b0 con 0x55c0ccc71cd0 2024-07-29T00:39:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.315+0000 7f9f17e00640 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0ccc71cd0 0x55c0ccc74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.316+0000 7f9f17e00640 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0ccc71cd0 0x55c0ccc74130 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d3da3485e9b9d7 server_cookie=ceeddeff09e25775 in_seq=0 out_seq=0 2024-07-29T00:39:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.316+0000 7f9efa000640 10 client.25925 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.316+0000 7f9efa000640 1 -- 172.21.15.175:0/2266043504 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0ccbc11b0 con 0x55c0ccc71cd0 2024-07-29T00:39:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.316+0000 7f9efa000640 10 client.25925 handle_command_reply: tid=0 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 client.25925 shutdown 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 msgr2=0x55c0ccc51f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc51f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e600 tx=0x7f9f0c00eac0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 shutdown_connections 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0ccc71cd0 0x55c0ccc74130 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9efa000640 0 client.25925 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 --2- 172.21.15.175:0/2266043504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc51f50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 >> 172.21.15.175:0/2266043504 conn(0x55c0ccc4a800 msgr2=0x55c0ccc50520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 shutdown_connections 2024-07-29T00:39:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.317+0000 7f9f220c9740 1 -- 172.21.15.175:0/2266043504 wait complete. 2024-07-29T00:39:07.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.318+0000 7f9f220c9740 20 client.25925 trim_cache size 0 max 16384 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.337+0000 7f9f220c9740 1 Processor -- start 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 -- start start 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc4e630 0x55c0ccc546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181ab4d0 con 0x55c0ccc4e630 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181ad0d0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f220c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181abba0 con 0x55c0ccc61ad0 2024-07-29T00:39:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f1d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc4e630 0x55c0ccc546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48788/0 (socket says 172.21.15.175:48788) 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.338+0000 7f9f17e00640 1 -- 172.21.15.175:0/889595395 learned_addr learned my addr 172.21.15.175:0/889595395 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f17e00640 1 -- 172.21.15.175:0/889595395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc61ad0 msgr2=0x55c0ccc62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f17e00640 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f17e00640 1 -- 172.21.15.175:0/889595395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc4e630 msgr2=0x55c0ccc546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f17e00640 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc4e630 0x55c0ccc546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f17e00640 1 -- 172.21.15.175:0/889595395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ccc618c0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f17e00640 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f0809bc30 tx=0x7f9f080d54b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76f1e94e2c8e603 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f05200640 1 -- 172.21.15.175:0/889595395 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0809b7e0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f05200640 1 -- 172.21.15.175:0/889595395 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f0809c3a0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.339+0000 7f9f05200640 1 -- 172.21.15.175:0/889595395 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0809b600 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.340+0000 7f9f220c9740 1 -- 172.21.15.175:0/889595395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 msgr2=0x55c0ccc52820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.340+0000 7f9f220c9740 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f0809bc30 tx=0x7f9f080d54b0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.340+0000 7f9f220c9740 1 -- 172.21.15.175:0/889595395 shutdown_connections 2024-07-29T00:39:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.340+0000 7f9f220c9740 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.340+0000 7f9f220c9740 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc52820 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.341+0000 7f9f220c9740 1 --2- 172.21.15.175:0/889595395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc4e630 0x55c0ccc546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.341+0000 7f9f220c9740 1 -- 172.21.15.175:0/889595395 >> 172.21.15.175:0/889595395 conn(0x55c0ccc4a800 msgr2=0x55c0ccc6d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.341+0000 7f9f220c9740 1 -- 172.21.15.175:0/889595395 shutdown_connections 2024-07-29T00:39:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.341+0000 7f9f220c9740 1 -- 172.21.15.175:0/889595395 wait complete. 2024-07-29T00:39:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.341+0000 7f9f220c9740 1 -- start start 2024-07-29T00:39:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.341+0000 7f9f220c9740 1 Processor -- start 2024-07-29T00:39:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc69f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc6a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc64d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181ac940 con 0x55c0ccc61ad0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181ac270 con 0x55c0ccc4e630 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a94b0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f220c9740 10 client.0 _ll_register_callbacks cb 0x7f9f0c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc69f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc69f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48792/0 (socket says 172.21.15.175:48792) 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f1cc00640 1 -- 172.21.15.175:0/1086926578 learned_addr learned my addr 172.21.15.175:0/1086926578 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f17e00640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc6a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.342+0000 7f9f1d600640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc64d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f17e00640 1 -- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 msgr2=0x55c0ccc69f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f17e00640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc69f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f17e00640 1 -- 172.21.15.175:0/1086926578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 msgr2=0x55c0ccc64d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f17e00640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc64d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f17e00640 1 -- 172.21.15.175:0/1086926578 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ccc65560 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f1d600640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc64d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f17e00640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc6a440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f0809bc30 tx=0x7f9f080ccef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d781b472aad86fca server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9efa000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9efa000640 1 -- 172.21.15.175:0/1086926578 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0809b7e0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.343+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f0809c3a0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.344+0000 7f9efa000640 1 -- 172.21.15.175:0/1086926578 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f9f08003b10 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.344+0000 7f9efa000640 1 -- 172.21.15.175:0/1086926578 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9f0809bca0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.344+0000 7f9f220c9740 10 client.25900 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.344+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0cccdd5f0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9efa000640 1 -- 172.21.15.175:0/1086926578 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f0809c3a0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9efa000640 1 -- 172.21.15.175:0/1086926578 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9f0809b0f0 con 0x55c0ccc4f7f0 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9f220c9740 10 client.25900 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9f220c9740 10 client.25900 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9f220c9740 20 client.25900 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9f220c9740 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0cccdddf0 0x55c0ccce0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9f220c9740 4 client.25900 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.5, "await_for": 45.0}] 2024-07-29T00:39:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.345+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.5, "await_for": 45.0}) -- 0x55c0ccbbb500 con 0x55c0cccdddf0 2024-07-29T00:39:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.346+0000 7f9f1cc00640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0cccdddf0 0x55c0ccce0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.347+0000 7f9f1cc00640 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0cccdddf0 0x55c0ccce0210 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61eb156f99f1014 server_cookie=4d39da39ed661894 in_seq=0 out_seq=0 2024-07-29T00:39:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.347+0000 7f9efa000640 10 client.25900 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.561+0000 7f9efa000640 1 -- 172.21.15.175:0/1086926578 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c0ccbbb500 con 0x55c0cccdddf0 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.561+0000 7f9efa000640 10 client.25900 handle_command_reply: tid=0 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.561+0000 7f9f220c9740 1 client.25900 shutdown 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 msgr2=0x55c0ccc6a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc6a440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f0809bc30 tx=0x7f9f080ccef0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 shutdown_connections 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c0cccdddf0 0x55c0ccce0210 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0ccc61ad0 0x55c0ccc64d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0ccc4f7f0 0x55c0ccc6a440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 --2- 172.21.15.175:0/1086926578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0ccc4e630 0x55c0ccc69f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9efa000640 0 client.25900 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 >> 172.21.15.175:0/1086926578 conn(0x55c0ccc4a800 msgr2=0x55c0ccc53630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 shutdown_connections 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 1 -- 172.21.15.175:0/1086926578 wait complete. 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.562+0000 7f9f220c9740 20 client.25900 trim_cache size 0 max 16384 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:39:07.563 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 31, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 6, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "c698167e": { 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 6, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 138.5, 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:39:07.564 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:07.565 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.563+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9f000a2d70 msgr2=0x7f9f000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.563+0000 7f9f1f600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9f000a2d70 0x7f9f000a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f10005f00 tx=0x7f9f1003d460 comp rx=0 tx=0).stop 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.563+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 msgr2=0x7f9f18198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.563+0000 7f9f1f600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 0x7f9f18198640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e9c0 tx=0x7f9f0800ee80 comp rx=0 tx=0).stop 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 shutdown_connections 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f9f000a2d70 0x7f9f000a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f18107880 0x7f9f1819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f18102220 0x7f9f18198640 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 --2- 172.21.15.175:0/3520416786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f18101880 0x7f9f18198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:07.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 >> 172.21.15.175:0/3520416786 conn(0x7f9f180fd5a0 msgr2=0x7f9f180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:07.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 shutdown_connections 2024-07-29T00:39:07.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:07.564+0000 7f9f1f600640 1 -- 172.21.15.175:0/3520416786 wait complete. 2024-07-29T00:39:07.852 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.98 MB/sec warmup 58 sec latency 456.161 ms 2024-07-29T00:39:07.955 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:07 smithi175 ceph-mon[32138]: pgmap v81: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-07-29T00:39:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:07 smithi204 ceph-mon[37368]: pgmap v81: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-07-29T00:39:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:07 smithi181 ceph-mon[36706]: pgmap v81: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-07-29T00:39:08.508 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c698167e', quiesce took 4.8 seconds. Will release after: 13.7 2024-07-29T00:39:08.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.88 MB/sec warmup 59 sec latency 1456.670 ms 2024-07-29T00:39:09.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.78 MB/sec warmup 60 sec latency 2456.775 ms 2024-07-29T00:39:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:09 smithi175 ceph-mon[32138]: pgmap v82: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-07-29T00:39:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:09 smithi204 ceph-mon[37368]: pgmap v82: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-07-29T00:39:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:09 smithi181 ceph-mon[36706]: pgmap v82: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-07-29T00:39:10.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.68 MB/sec warmup 61 sec latency 3456.921 ms 2024-07-29T00:39:11.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.59 MB/sec warmup 62 sec latency 4457.067 ms 2024-07-29T00:39:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:11 smithi175 ceph-mon[32138]: pgmap v83: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-07-29T00:39:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:11 smithi204 ceph-mon[37368]: pgmap v83: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-07-29T00:39:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:11 smithi181 ceph-mon[36706]: pgmap v83: 97 pgs: 97 active+clean; 46 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-07-29T00:39:12.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.50 MB/sec warmup 63 sec latency 5457.209 ms 2024-07-29T00:39:13.853 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.42 MB/sec warmup 64 sec latency 6457.304 ms 2024-07-29T00:39:14.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:13 smithi175 ceph-mon[32138]: pgmap v84: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-07-29T00:39:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:13 smithi204 ceph-mon[37368]: pgmap v84: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-07-29T00:39:14.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:13 smithi181 ceph-mon[36706]: pgmap v84: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-07-29T00:39:14.854 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.33 MB/sec warmup 65 sec latency 7457.448 ms 2024-07-29T00:39:15.854 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.25 MB/sec warmup 66 sec latency 8457.592 ms 2024-07-29T00:39:16.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:15 smithi175 ceph-mon[32138]: pgmap v85: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 762 KiB/s wr, 109 op/s 2024-07-29T00:39:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:15 smithi204 ceph-mon[37368]: pgmap v85: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 762 KiB/s wr, 109 op/s 2024-07-29T00:39:16.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:15 smithi181 ceph-mon[36706]: pgmap v85: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 762 KiB/s wr, 109 op/s 2024-07-29T00:39:16.855 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.17 MB/sec warmup 67 sec latency 9458.221 ms 2024-07-29T00:39:17.861 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.10 MB/sec warmup 68 sec latency 10465.062 ms 2024-07-29T00:39:17.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:17 smithi175 ceph-mon[32138]: pgmap v86: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 773 KiB/s wr, 117 op/s 2024-07-29T00:39:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:17 smithi204 ceph-mon[37368]: pgmap v86: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 773 KiB/s wr, 117 op/s 2024-07-29T00:39:18.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:17 smithi181 ceph-mon[36706]: pgmap v86: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 773 KiB/s wr, 117 op/s 2024-07-29T00:39:18.861 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 5.02 MB/sec warmup 69 sec latency 11465.215 ms 2024-07-29T00:39:19.861 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.95 MB/sec warmup 70 sec latency 12465.306 ms 2024-07-29T00:39:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:19 smithi204 ceph-mon[37368]: pgmap v87: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-07-29T00:39:20.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:19 smithi181 ceph-mon[36706]: pgmap v87: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-07-29T00:39:20.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:19 smithi175 ceph-mon[32138]: pgmap v87: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-07-29T00:39:20.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:39:20.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.88 MB/sec warmup 71 sec latency 13465.394 ms 2024-07-29T00:39:20.975 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 -- 172.21.15.175:0/996783411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77810d220 msgr2=0x7fa778114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 --2- 172.21.15.175:0/996783411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77810d220 0x7fa778114da0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa770009ca0 tx=0x7fa77002a8b0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 -- 172.21.15.175:0/996783411 shutdown_connections 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 --2- 172.21.15.175:0/996783411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77810d220 0x7fa778114da0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 --2- 172.21.15.175:0/996783411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 --2- 172.21.15.175:0/996783411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa778108140 0x7fa778108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 -- 172.21.15.175:0/996783411 >> 172.21.15.175:0/996783411 conn(0x7fa778077890 msgr2=0x7fa778079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 -- 172.21.15.175:0/996783411 shutdown_connections 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.631+0000 7fa780200640 1 -- 172.21.15.175:0/996783411 wait complete. 2024-07-29T00:39:21.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.632+0000 7fa780200640 1 Processor -- start 2024-07-29T00:39:21.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 -- start start 2024-07-29T00:39:21.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa778108140 0x7fa77819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810d220 0x7fa7781a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa778118280 con 0x7fa778108af0 2024-07-29T00:39:21.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa778118100 con 0x7fa778108140 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa780200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa778118400 con 0x7fa77810d220 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa77ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa77d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa778108140 0x7fa77819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa77e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810d220 0x7fa7781a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa77ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39810/0 (socket says 172.21.15.175:39810) 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.633+0000 7fa77ce00640 1 -- 172.21.15.175:0/3970363806 learned_addr learned my addr 172.21.15.175:0/3970363806 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77d800640 1 -- 172.21.15.175:0/3970363806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810d220 msgr2=0x7fa7781a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77d800640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810d220 0x7fa7781a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77d800640 1 -- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 msgr2=0x7fa77819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77d800640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77d800640 1 -- 172.21.15.175:0/3970363806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7781a3650 con 0x7fa778108140 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77ce00640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.634+0000 7fa77d800640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa778108140 0x7fa77819e6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e9f0 tx=0x7fa76c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:21.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.635+0000 7fa776000640 1 -- 172.21.15.175:0/3970363806 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00ccc0 con 0x7fa778108140 2024-07-29T00:39:21.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.635+0000 7fa776000640 1 -- 172.21.15.175:0/3970363806 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76c010910 con 0x7fa778108140 2024-07-29T00:39:21.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.635+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7781a3940 con 0x7fa778108140 2024-07-29T00:39:21.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.635+0000 7fa776000640 1 -- 172.21.15.175:0/3970363806 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c018960 con 0x7fa778108140 2024-07-29T00:39:21.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.635+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781ab220 con 0x7fa778108140 2024-07-29T00:39:21.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.638+0000 7fa776000640 1 -- 172.21.15.175:0/3970363806 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa76c010ac0 con 0x7fa778108140 2024-07-29T00:39:21.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.638+0000 7fa782e80740 1 Processor -- start 2024-07-29T00:39:21.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.638+0000 7fa782e80740 1 -- start start 2024-07-29T00:39:21.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa782e80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bc930 0x564e4b2bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa782e80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa782e80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa782e80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564e4b221ad0 con 0x564e4b2c7b10 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa782e80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564e4b1b9af0 con 0x564e4b2bd250 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa782e80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564e4b21da90 con 0x564e4b2bc930 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa77d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa77d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48204/0 (socket says 172.21.15.175:48204) 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa77d800640 1 -- 172.21.15.175:0/4032438798 learned_addr learned my addr 172.21.15.175:0/4032438798 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa77e200640 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bc930 0x564e4b2bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.639+0000 7fa77ce00640 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77d800640 1 -- 172.21.15.175:0/4032438798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bc930 msgr2=0x564e4b2bcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77d800640 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bc930 0x564e4b2bcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77d800640 1 -- 172.21.15.175:0/4032438798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 msgr2=0x564e4b2c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77d800640 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77d800640 1 -- 172.21.15.175:0/4032438798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e4af933c0 con 0x564e4b2bd250 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa776000640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa7600a2d70 0x7fa7600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77d800640 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c75d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0ccdd0 tx=0x7fa76c0d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6028885e8c5e59 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa77e200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa7600a2d70 0x7fa7600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa776000640 1 -- 172.21.15.175:0/3970363806 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa76c0cf7c0 con 0x7fa778108140 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa76aa00640 1 -- 172.21.15.175:0/4032438798 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c026030 con 0x564e4b2bd250 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa76aa00640 1 -- 172.21.15.175:0/4032438798 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76c09bd10 con 0x564e4b2bd250 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.640+0000 7fa76aa00640 1 -- 172.21.15.175:0/4032438798 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c09b850 con 0x564e4b2bd250 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 -- 172.21.15.175:0/4032438798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 msgr2=0x564e4b2c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa77e200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa7600a2d70 0x7fa7600a5230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa770006010 tx=0x7fa770005d80 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c75d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0ccdd0 tx=0x7fa76c0d7670 comp rx=0 tx=0).stop 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 -- 172.21.15.175:0/4032438798 shutdown_connections 2024-07-29T00:39:21.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c75d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 --2- 172.21.15.175:0/4032438798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bc930 0x564e4b2bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.641+0000 7fa782e80740 1 -- 172.21.15.175:0/4032438798 >> 172.21.15.175:0/4032438798 conn(0x564e4b2b8ec0 msgr2=0x564e4b2b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.642+0000 7fa782e80740 1 -- 172.21.15.175:0/4032438798 shutdown_connections 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.642+0000 7fa782e80740 1 -- 172.21.15.175:0/4032438798 wait complete. 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.642+0000 7fa782e80740 1 -- start start 2024-07-29T00:39:21.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.642+0000 7fa782e80740 1 Processor -- start 2024-07-29T00:39:21.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bc930 0x564e4b2c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564e4b2ca6e0 con 0x564e4b2c7b10 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564e4b1dcbc0 con 0x564e4b2bc930 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564e4b2ca860 con 0x564e4b2bd250 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa782e80740 10 client.0 _ll_register_callbacks cb 0x564e4b1c8d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:51316/0 (socket says 172.21.15.175:51316) 2024-07-29T00:39:21.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 -- 172.21.15.175:0/1992277591 learned_addr learned my addr 172.21.15.175:0/1992277591 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77d800640 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bc930 0x564e4b2c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 -- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bc930 msgr2=0x564e4b2c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bc930 0x564e4b2c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 -- 172.21.15.175:0/1992277591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 msgr2=0x564e4b2c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.643+0000 7fa77ce00640 1 -- 172.21.15.175:0/1992277591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4b2d3bd0 con 0x564e4b2bd250 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.644+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2c4f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa764012d50 tx=0x7fa76400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c4516b42698e6b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.644+0000 7fa75f400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.644+0000 7fa75f400640 1 -- 172.21.15.175:0/1992277591 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764012110 con 0x564e4b2bd250 2024-07-29T00:39:21.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.644+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76c09bd10 con 0x564e4b2bd250 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.644+0000 7fa75f400640 1 -- 172.21.15.175:0/1992277591 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa764004050 con 0x564e4b2bd250 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.644+0000 7fa75f400640 1 -- 172.21.15.175:0/1992277591 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa7640122b0 con 0x564e4b2bd250 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 10 client.25912 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564e4b2d65f0 con 0x564e4b2bd250 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa75f400640 1 -- 172.21.15.175:0/1992277591 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76c09bd10 con 0x564e4b2bd250 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa75f400640 1 -- 172.21.15.175:0/1992277591 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa764010430 con 0x564e4b2bd250 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 10 client.25912 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 20 client.25912 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b2d7a80 0x564e4b2d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 4 client.25912 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.645+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e4b22f040 con 0x564e4b2d7a80 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.646+0000 7fa77d800640 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b2d7a80 0x564e4b2d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.646+0000 7fa77d800640 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b2d7a80 0x564e4b2d9ea0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b78fcd37a6c34a server_cookie=fe88187937e19a3 in_seq=0 out_seq=0 2024-07-29T00:39:21.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.646+0000 7fa75f400640 10 client.25912 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa75f400640 1 -- 172.21.15.175:0/1992277591 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564e4b22f040 con 0x564e4b2d7a80 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa75f400640 10 client.25912 handle_command_reply: tid=0 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 client.25912 shutdown 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 msgr2=0x564e4b2c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2c4f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa764012d50 tx=0x7fa76400f790 comp rx=0 tx=0).stop 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 shutdown_connections 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b2d7a80 0x564e4b2d9ea0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2c7b10 0x564e4b2c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2c4f20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 --2- 172.21.15.175:0/1992277591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bc930 0x564e4b2c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa75f400640 0 client.25912 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:21.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.647+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 >> 172.21.15.175:0/1992277591 conn(0x564e4b2b8ec0 msgr2=0x564e4b2beba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:21.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.648+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 shutdown_connections 2024-07-29T00:39:21.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.648+0000 7fa782e80740 1 -- 172.21.15.175:0/1992277591 wait complete. 2024-07-29T00:39:21.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.648+0000 7fa782e80740 20 client.25912 trim_cache size 0 max 16384 2024-07-29T00:39:21.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.666+0000 7fa782e80740 1 Processor -- start 2024-07-29T00:39:21.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 -- start start 2024-07-29T00:39:21.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2c7b10 0x564e4b2df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b3cc0 con 0x564e4b2bc930 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b58c0 con 0x564e4b2c7b10 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa782e80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b4390 con 0x564e4b2bd250 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa77d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa77d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39844/0 (socket says 172.21.15.175:39844) 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa77d800640 1 -- 172.21.15.175:0/1011518752 learned_addr learned my addr 172.21.15.175:0/1011518752 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.667+0000 7fa77e200640 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2c7b10 0x564e4b2df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77ce00640 1 -- 172.21.15.175:0/1011518752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2c7b10 msgr2=0x564e4b2df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2c7b10 0x564e4b2df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77ce00640 1 -- 172.21.15.175:0/1011518752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 msgr2=0x564e4b2c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77ce00640 1 -- 172.21.15.175:0/1011518752 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e4b2bd080 con 0x564e4b2bd250 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77d800640 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa77ce00640 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2b7de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa764010a40 tx=0x7fa76400fee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d53eb5a6bd51af server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa76a000640 1 -- 172.21.15.175:0/1011518752 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764029070 con 0x564e4b2bd250 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa76a000640 1 -- 172.21.15.175:0/1011518752 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7640055c0 con 0x564e4b2bd250 2024-07-29T00:39:21.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.668+0000 7fa76a000640 1 -- 172.21.15.175:0/1011518752 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764003870 con 0x564e4b2bd250 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 -- 172.21.15.175:0/1011518752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 msgr2=0x564e4b2b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2b7de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa764010a40 tx=0x7fa76400fee0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 -- 172.21.15.175:0/1011518752 shutdown_connections 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2c7b10 0x564e4b2df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2bd250 0x564e4b2b7de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 --2- 172.21.15.175:0/1011518752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 -- 172.21.15.175:0/1011518752 >> 172.21.15.175:0/1011518752 conn(0x564e4b2b8ec0 msgr2=0x564e4b2b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 -- 172.21.15.175:0/1011518752 shutdown_connections 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 -- 172.21.15.175:0/1011518752 wait complete. 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.669+0000 7fa782e80740 1 -- start start 2024-07-29T00:39:21.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 Processor -- start 2024-07-29T00:39:21.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2c7b10 0x564e4b2c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b5130 con 0x564e4b2bc930 2024-07-29T00:39:21.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b4a60 con 0x564e4b2bd250 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b1ca0 con 0x564e4b2c7b10 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.670+0000 7fa782e80740 10 client.0 _ll_register_callbacks cb 0x7fa764012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48228/0 (socket says 172.21.15.175:48228) 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 -- 172.21.15.175:0/2576162986 learned_addr learned my addr 172.21.15.175:0/2576162986 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77e200640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2c7b10 0x564e4b2c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 -- 172.21.15.175:0/2576162986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2c7b10 msgr2=0x564e4b2c1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2c7b10 0x564e4b2c1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 -- 172.21.15.175:0/2576162986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 msgr2=0x564e4b2d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 -- 172.21.15.175:0/2576162986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4b2c2490 con 0x564e4b2bd250 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77e200640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2c7b10 0x564e4b2c1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa77ce00640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c1720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7640109c0 tx=0x7fa764010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c62428f9c7c7dd server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.671+0000 7fa75f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa75f400640 1 -- 172.21.15.175:0/2576162986 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764032070 con 0x564e4b2bd250 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa75f400640 1 -- 172.21.15.175:0/2576162986 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa7640052c0 con 0x564e4b2bd250 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7640055c0 con 0x564e4b2bd250 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa75f400640 1 -- 172.21.15.175:0/2576162986 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa76403e050 con 0x564e4b2bd250 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa782e80740 10 client.25949 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564e4b3426c0 con 0x564e4b2bd250 2024-07-29T00:39:21.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa75f400640 1 -- 172.21.15.175:0/2576162986 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7640055c0 con 0x564e4b2bd250 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa75f400640 1 -- 172.21.15.175:0/2576162986 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa76402e460 con 0x564e4b2bd250 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.672+0000 7fa782e80740 10 client.25949 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.673+0000 7fa782e80740 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.673+0000 7fa782e80740 20 client.25949 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.673+0000 7fa782e80740 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b342de0 0x564e4b345200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.673+0000 7fa782e80740 4 client.25949 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:39:21.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.673+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e4b229550 con 0x564e4b342de0 2024-07-29T00:39:21.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.673+0000 7fa77d800640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b342de0 0x564e4b345200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:21.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa77d800640 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b342de0 0x564e4b345200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec05709f023fa25 server_cookie=d75a614bffed715b in_seq=0 out_seq=0 2024-07-29T00:39:21.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa75f400640 10 client.25949 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:21.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa75f400640 1 -- 172.21.15.175:0/2576162986 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564e4b229550 con 0x564e4b342de0 2024-07-29T00:39:21.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa75f400640 10 client.25949 handle_command_reply: tid=0 2024-07-29T00:39:21.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 client.25949 shutdown 2024-07-29T00:39:21.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 msgr2=0x564e4b2c1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c1720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7640109c0 tx=0x7fa764010ca0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 shutdown_connections 2024-07-29T00:39:21.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564e4b342de0 0x564e4b345200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e4b2c7b10 0x564e4b2c1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e4b2bd250 0x564e4b2c1720 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa75f400640 0 client.25949 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 --2- 172.21.15.175:0/2576162986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e4b2bc930 0x564e4b2d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 >> 172.21.15.175:0/2576162986 conn(0x564e4b2b8ec0 msgr2=0x564e4b2b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 shutdown_connections 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.674+0000 7fa782e80740 1 -- 172.21.15.175:0/2576162986 wait complete. 2024-07-29T00:39:21.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.675+0000 7fa782e80740 20 client.25949 trim_cache size 0 max 16384 2024-07-29T00:39:21.682 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:39:21.682 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:39:21.682 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:39:21.683 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:39:21.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.675+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa7600a2d70 msgr2=0x7fa7600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.675+0000 7fa780200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa7600a2d70 0x7fa7600a5230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa770006010 tx=0x7fa770005d80 comp rx=0 tx=0).stop 2024-07-29T00:39:21.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.675+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa778108140 msgr2=0x7fa77819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:21.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.675+0000 7fa780200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa778108140 0x7fa77819e6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e9f0 tx=0x7fa76c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 shutdown_connections 2024-07-29T00:39:21.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810d220 0x7fa7781a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa7600a2d70 0x7fa7600a5230 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa778108af0 0x7fa77819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 --2- 172.21.15.175:0/3970363806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa778108140 0x7fa77819e6a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:21.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 >> 172.21.15.175:0/3970363806 conn(0x7fa778077890 msgr2=0x7fa778078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:21.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 shutdown_connections 2024-07-29T00:39:21.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:21.676+0000 7fa780200640 1 -- 172.21.15.175:0/3970363806 wait complete. 2024-07-29T00:39:21.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.82 MB/sec warmup 72 sec latency 14465.529 ms 2024-07-29T00:39:22.049 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:21 smithi175 ceph-mon[32138]: pgmap v88: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-07-29T00:39:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:21 smithi204 ceph-mon[37368]: pgmap v88: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-07-29T00:39:22.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:39:22.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:21 smithi181 ceph-mon[36706]: pgmap v88: 97 pgs: 97 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-07-29T00:39:22.785 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:39:22.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:39:22.832 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:22.861 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.75 MB/sec warmup 73 sec latency 15465.625 ms 2024-07-29T00:39:23.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.504+0000 7f2d77e00640 1 -- 172.21.15.175:0/582937299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 msgr2=0x7f2d70102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:23.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.504+0000 7f2d77e00640 1 --2- 172.21.15.175:0/582937299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d70102e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d6402a480 comp rx=0 tx=0).stop 2024-07-29T00:39:23.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 -- 172.21.15.175:0/582937299 shutdown_connections 2024-07-29T00:39:23.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 --2- 172.21.15.175:0/582937299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 0x7f2d7010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 --2- 172.21.15.175:0/582937299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 0x7f2d7010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 --2- 172.21.15.175:0/582937299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d70102e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 -- 172.21.15.175:0/582937299 >> 172.21.15.175:0/582937299 conn(0x7f2d700fc490 msgr2=0x7f2d700fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 -- 172.21.15.175:0/582937299 shutdown_connections 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.505+0000 7f2d77e00640 1 -- 172.21.15.175:0/582937299 wait complete. 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 Processor -- start 2024-07-29T00:39:23.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 -- start start 2024-07-29T00:39:23.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d7019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:23.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 0x7f2d7019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 0x7f2d7019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111cb0 con 0x7f2d7010be60 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111b30 con 0x7f2d70102a30 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.506+0000 7f2d77e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111e30 con 0x7f2d70103350 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d75e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 0x7f2d7019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d75400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d7019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d74a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 0x7f2d7019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d75e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 0x7f2d7019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39872/0 (socket says 172.21.15.175:39872) 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d75400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d7019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48242/0 (socket says 172.21.15.175:48242) 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d75e00640 1 -- 172.21.15.175:0/2307136609 learned_addr learned my addr 172.21.15.175:0/2307136609 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d74a00640 1 -- 172.21.15.175:0/2307136609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 msgr2=0x7f2d7019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:23.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d74a00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d7019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d74a00640 1 -- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 msgr2=0x7f2d7019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d74a00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 0x7f2d7019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.507+0000 7f2d74a00640 1 -- 172.21.15.175:0/2307136609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7019f2e0 con 0x7f2d70103350 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.508+0000 7f2d74a00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 0x7f2d7019a870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00e7c0 tx=0x7f2d5c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.508+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c009530 con 0x7f2d70103350 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.508+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7019f5d0 con 0x7f2d70103350 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.508+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5c00ccc0 con 0x7f2d70103350 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.508+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a6eb0 con 0x7f2d70103350 2024-07-29T00:39:23.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.508+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c01c070 con 0x7f2d70103350 2024-07-29T00:39:23.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.509+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d70102e10 con 0x7f2d70103350 2024-07-29T00:39:23.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.513+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2d5c005020 con 0x7f2d70103350 2024-07-29T00:39:23.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.514+0000 7f2d6e000640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2d580a2d70 0x7f2d580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:23.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.515+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f2d5c0cf3d0 con 0x7f2d70103350 2024-07-29T00:39:23.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.515+0000 7f2d75400640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2d580a2d70 0x7f2d580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:23.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.516+0000 7f2d75400640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2d580a2d70 0x7f2d580a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000c00 tx=0x7f2d64005b50 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:23.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.519+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2d5c09bf70 con 0x7f2d70103350 2024-07-29T00:39:23.587 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:23.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.768+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d701a71b0 con 0x7f2d70103350 2024-07-29T00:39:23.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.769+0000 7f2d6e000640 1 -- 172.21.15.175:0/2307136609 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7f2d5c09b870 con 0x7f2d70103350 2024-07-29T00:39:23.770 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:39:23.771 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:39:23.771 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2d580a2d70 msgr2=0x7f2d580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2d580a2d70 0x7f2d580a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000c00 tx=0x7f2d64005b50 comp rx=0 tx=0).stop 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 msgr2=0x7f2d7019a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 0x7f2d7019a870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00e7c0 tx=0x7f2d5c00ec80 comp rx=0 tx=0).stop 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 shutdown_connections 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2d580a2d70 0x7f2d580a5230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7010be60 0x7f2d7019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70103350 0x7f2d7019a870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 --2- 172.21.15.175:0/2307136609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d70102a30 0x7f2d7019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:23.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.773+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 >> 172.21.15.175:0/2307136609 conn(0x7f2d700fc490 msgr2=0x7f2d7010d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:23.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.774+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 shutdown_connections 2024-07-29T00:39:23.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:23.774+0000 7f2d77e00640 1 -- 172.21.15.175:0/2307136609 wait complete. 2024-07-29T00:39:23.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.68 MB/sec warmup 74 sec latency 16465.758 ms 2024-07-29T00:39:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:23 smithi175 ceph-mon[32138]: pgmap v89: 97 pgs: 97 active+clean; 50 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 669 KiB/s wr, 57 op/s 2024-07-29T00:39:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:23 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2307136609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:23 smithi204 ceph-mon[37368]: pgmap v89: 97 pgs: 97 active+clean; 50 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 669 KiB/s wr, 57 op/s 2024-07-29T00:39:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:23 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2307136609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:24.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:23 smithi181 ceph-mon[36706]: pgmap v89: 97 pgs: 97 active+clean; 50 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 669 KiB/s wr, 57 op/s 2024-07-29T00:39:24.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:23 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2307136609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:24.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 -- 172.21.15.175:0/498060112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570102a20 msgr2=0x7f9570102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:24.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 --2- 172.21.15.175:0/498060112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570102a20 0x7f9570102e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9560009cd0 tx=0x7f956002a8e0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 -- 172.21.15.175:0/498060112 shutdown_connections 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 --2- 172.21.15.175:0/498060112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9570107f90 0x7f957010e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 --2- 172.21.15.175:0/498060112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95701033d0 0x7f9570107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 --2- 172.21.15.175:0/498060112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570102a20 0x7f9570102e00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 -- 172.21.15.175:0/498060112 >> 172.21.15.175:0/498060112 conn(0x7f95700fc4c0 msgr2=0x7f95700fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 -- 172.21.15.175:0/498060112 shutdown_connections 2024-07-29T00:39:24.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.397+0000 7f9577a00640 1 -- 172.21.15.175:0/498060112 wait complete. 2024-07-29T00:39:24.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.398+0000 7f9577a00640 1 Processor -- start 2024-07-29T00:39:24.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 -- start start 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 0x7f95701982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95701033d0 0x7f9570198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570107f90 0x7f957019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9570111de0 con 0x7f95701033d0 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9570111c60 con 0x7f9570107f90 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9577a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9570111f60 con 0x7f9570102a20 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9575000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 0x7f95701982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9567e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95701033d0 0x7f9570198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9575000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 0x7f95701982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:51348/0 (socket says 172.21.15.175:51348) 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.399+0000 7f9575000640 1 -- 172.21.15.175:0/3960391740 learned_addr learned my addr 172.21.15.175:0/3960391740 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:24.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9575a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570107f90 0x7f957019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:24.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9567e00640 1 -- 172.21.15.175:0/3960391740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 msgr2=0x7f95701982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:24.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9567e00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 0x7f95701982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9567e00640 1 -- 172.21.15.175:0/3960391740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570107f90 msgr2=0x7f957019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:24.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9567e00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570107f90 0x7f957019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9567e00640 1 -- 172.21.15.175:0/3960391740 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957019d290 con 0x7f95701033d0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9575000640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 0x7f95701982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9575a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570107f90 0x7f957019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.400+0000 7f9567e00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95701033d0 0x7f9570198820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f95700694c0 tx=0x7f955800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.401+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955800cc40 con 0x7f95701033d0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.401+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955800cde0 con 0x7f95701033d0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.401+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558004a00 con 0x7f95701033d0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.401+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957019d580 con 0x7f95701033d0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.401+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701a4e60 con 0x7f95701033d0 2024-07-29T00:39:24.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.402+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957006a540 con 0x7f95701033d0 2024-07-29T00:39:24.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.404+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9558004ba0 con 0x7f95701033d0 2024-07-29T00:39:24.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.406+0000 7f956e000640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95540a2d70 0x7f95540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:24.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.406+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f95580cf390 con 0x7f95701033d0 2024-07-29T00:39:24.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.406+0000 7f9575000640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95540a2d70 0x7f95540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:24.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.407+0000 7f9575000640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95540a2d70 0x7f95540a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9560000f80 tx=0x7f9560032fd0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:24.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.411+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f955809bf30 con 0x7f95701033d0 2024-07-29T00:39:24.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.643+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9570109f20 con 0x7f95701033d0 2024-07-29T00:39:24.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.645+0000 7f956e000640 1 -- 172.21.15.175:0/3960391740 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7f955809b830 con 0x7f95701033d0 2024-07-29T00:39:24.646 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:39:24.646 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:39:24.646 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:39:24.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95540a2d70 msgr2=0x7f95540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:24.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95540a2d70 0x7f95540a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9560000f80 tx=0x7f9560032fd0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95701033d0 msgr2=0x7f9570198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95701033d0 0x7f9570198820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f95700694c0 tx=0x7f955800ee50 comp rx=0 tx=0).stop 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 shutdown_connections 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9570107f90 0x7f957019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f95540a2d70 0x7f95540a5230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95701033d0 0x7f9570198820 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 --2- 172.21.15.175:0/3960391740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9570102a20 0x7f95701982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 >> 172.21.15.175:0/3960391740 conn(0x7f95700fc4c0 msgr2=0x7f9570105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 shutdown_connections 2024-07-29T00:39:24.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:24.648+0000 7f9577a00640 1 -- 172.21.15.175:0/3960391740 wait complete. 2024-07-29T00:39:24.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.62 MB/sec warmup 75 sec latency 17465.846 ms 2024-07-29T00:39:24.938 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:24 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3960391740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:25.002 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id c698167e --release --await' 2024-07-29T00:39:25.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db --set-id c698167e --release --await 2024-07-29T00:39:25.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:24 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3960391740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:25.150 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:24 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3960391740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:39:25.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61213 4.56 MB/sec warmup 76 sec latency 18465.952 ms 2024-07-29T00:39:25.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:39:25.973 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:25 smithi175 ceph-mon[32138]: pgmap v90: 97 pgs: 97 active+clean; 50 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 546 KiB/s wr, 29 op/s 2024-07-29T00:39:26.000 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:25 smithi204 ceph-mon[37368]: pgmap v90: 97 pgs: 97 active+clean; 50 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 546 KiB/s wr, 29 op/s 2024-07-29T00:39:26.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:25 smithi181 ceph-mon[36706]: pgmap v90: 97 pgs: 97 active+clean; 50 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 546 KiB/s wr, 29 op/s 2024-07-29T00:39:26.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 -- 172.21.15.175:0/1467981421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 msgr2=0x7fd4e810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/1467981421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 0x7fd4e810e130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009930 tx=0x7fd4d802a440 comp rx=0 tx=0).stop 2024-07-29T00:39:26.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 -- 172.21.15.175:0/1467981421 shutdown_connections 2024-07-29T00:39:26.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/1467981421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4e8108670 0x7fd4e8114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/1467981421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/1467981421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 0x7fd4e810e130 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 -- 172.21.15.175:0/1467981421 >> 172.21.15.175:0/1467981421 conn(0x7fd4e8077670 msgr2=0x7fd4e8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:26.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 -- 172.21.15.175:0/1467981421 shutdown_connections 2024-07-29T00:39:26.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.751+0000 7fd4e7e00640 1 -- 172.21.15.175:0/1467981421 wait complete. 2024-07-29T00:39:26.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.752+0000 7fd4e7e00640 1 Processor -- start 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.752+0000 7fd4e7e00640 1 -- start start 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4e8108670 0x7fd4e819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 0x7fd4e81a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8118060 con 0x7fd4e810dd50 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8117ee0 con 0x7fd4e8108670 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81181e0 con 0x7fd4e8107d10 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:51360/0 (socket says 172.21.15.175:51360) 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e6a00640 1 -- 172.21.15.175:0/75107159 learned_addr learned my addr 172.21.15.175:0/75107159 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e7400640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 0x7fd4e81a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.753+0000 7fd4e6000640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4e8108670 0x7fd4e819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e6a00640 1 -- 172.21.15.175:0/75107159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4e8108670 msgr2=0x7fd4e819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4e8108670 0x7fd4e819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e6a00640 1 -- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 msgr2=0x7fd4e81a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 0x7fd4e81a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e6a00640 1 -- 172.21.15.175:0/75107159 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e81a34f0 con 0x7fd4e8107d10 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e819e540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009230 tx=0x7fd4d8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4df400640 1 -- 172.21.15.175:0/75107159 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d803f030 con 0x7fd4e8107d10 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4df400640 1 -- 172.21.15.175:0/75107159 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d8003ad0 con 0x7fd4e8107d10 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.754+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81a3780 con 0x7fd4e8107d10 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.755+0000 7fd4df400640 1 -- 172.21.15.175:0/75107159 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8035070 con 0x7fd4e8107d10 2024-07-29T00:39:26.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.755+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e81ab0c0 con 0x7fd4e8107d10 2024-07-29T00:39:26.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 Processor -- start 2024-07-29T00:39:26.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 -- start start 2024-07-29T00:39:26.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4df400640 1 -- 172.21.15.175:0/75107159 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd4d8042030 con 0x7fd4e8107d10 2024-07-29T00:39:26.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b6d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b77a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b7a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565118ad2cc0 con 0x565118b77fb0 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x565118a6abc0 con 0x565118b6d6f0 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.758+0000 7fd4f046a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565118acec80 con 0x565118b6cdd0 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b77a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b7a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b77a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48282/0 (socket says 172.21.15.175:48282) 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b7a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39924/0 (socket says 172.21.15.175:39924) 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6a00640 1 -- 172.21.15.175:0/3901413620 learned_addr learned my addr 172.21.15.175:0/3901413620 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:26.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e7400640 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b6d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 -- 172.21.15.175:0/3901413620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 msgr2=0x565118b6d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b6d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 -- 172.21.15.175:0/3901413620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 msgr2=0x565118b77a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b77a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.759+0000 7fd4e6000640 1 -- 172.21.15.175:0/3901413620 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651187e2580 con 0x565118b77fb0 2024-07-29T00:39:26.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4e6000640 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b7a3a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400bbc0 tx=0x7fd4d40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c7a143073c4d99 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:26.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4df400640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd4cc0a2d70 0x7fd4cc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4df400640 1 -- 172.21.15.175:0/75107159 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd4d80ef8b0 con 0x7fd4e8107d10 2024-07-29T00:39:26.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4d3e00640 1 -- 172.21.15.175:0/3901413620 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4004030 con 0x565118b77fb0 2024-07-29T00:39:26.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4d3e00640 1 -- 172.21.15.175:0/3901413620 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d40041d0 con 0x565118b77fb0 2024-07-29T00:39:26.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4d3e00640 1 -- 172.21.15.175:0/3901413620 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40116a0 con 0x565118b77fb0 2024-07-29T00:39:26.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.760+0000 7fd4e7400640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd4cc0a2d70 0x7fd4cc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 -- 172.21.15.175:0/3901413620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 msgr2=0x565118b7a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b7a3a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400bbc0 tx=0x7fd4d40107a0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4e7400640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd4cc0a2d70 0x7fd4cc0a5230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005fd0 tx=0x7fd4e0009550 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 -- 172.21.15.175:0/3901413620 shutdown_connections 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b7a3a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b77a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 --2- 172.21.15.175:0/3901413620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b6d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.761+0000 7fd4f046a740 1 -- 172.21.15.175:0/3901413620 >> 172.21.15.175:0/3901413620 conn(0x565118b69360 msgr2=0x565118b69740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.762+0000 7fd4f046a740 1 -- 172.21.15.175:0/3901413620 shutdown_connections 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.762+0000 7fd4f046a740 1 -- 172.21.15.175:0/3901413620 wait complete. 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.762+0000 7fd4f046a740 1 -- start start 2024-07-29T00:39:26.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.762+0000 7fd4f046a740 1 Processor -- start 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b74ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b75970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565118b7ab80 con 0x565118b6cdd0 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x565118a8dca0 con 0x565118b77fb0 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565118b7ad00 con 0x565118b6d6f0 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4f046a740 10 client.0 _ll_register_callbacks cb 0x565118a79e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:26.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b74ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b74ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39938/0 (socket says 172.21.15.175:39938) 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4e6a00640 1 -- 172.21.15.175:0/668615507 learned_addr learned my addr 172.21.15.175:0/668615507 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.763+0000 7fd4e7400640 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b75970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4e6a00640 1 -- 172.21.15.175:0/668615507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 msgr2=0x565118b75430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b75430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4e6a00640 1 -- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 msgr2=0x565118b75970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b75970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4e6a00640 1 -- 172.21.15.175:0/668615507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651188ac960 con 0x565118b6cdd0 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b74ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80c4490 tx=0x7fd4d80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9b99e682bd7cea server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4d0c00640 1 -- 172.21.15.175:0/668615507 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8049070 con 0x565118b6cdd0 2024-07-29T00:39:26.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4d0c00640 1 -- 172.21.15.175:0/668615507 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd4d80b8470 con 0x565118b6cdd0 2024-07-29T00:39:26.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d40041d0 con 0x565118b6cdd0 2024-07-29T00:39:26.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4d0c00640 1 -- 172.21.15.175:0/668615507 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4d80bb760 con 0x565118b6cdd0 2024-07-29T00:39:26.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4f046a740 10 client.15852 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:26.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.764+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565118b86880 con 0x565118b6cdd0 2024-07-29T00:39:26.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4d0c00640 1 -- 172.21.15.175:0/668615507 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d40041d0 con 0x565118b6cdd0 2024-07-29T00:39:26.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4d0c00640 1 -- 172.21.15.175:0/668615507 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd4d80ed200 con 0x565118b6cdd0 2024-07-29T00:39:26.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4f046a740 10 client.15852 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4f046a740 10 client.15852 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4f046a740 20 client.15852 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4f046a740 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118b87dc0 0x565118b8a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4f046a740 4 client.15852 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565118ae0230 con 0x565118b87dc0 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.765+0000 7fd4e6000640 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118b87dc0 0x565118b8a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.766+0000 7fd4e6000640 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118b87dc0 0x565118b8a260 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a8180ebcea6e6e server_cookie=8eb2fb285644b120 in_seq=0 out_seq=0 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.766+0000 7fd4d0c00640 10 client.15852 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4d0c00640 1 -- 172.21.15.175:0/668615507 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565118ae0230 con 0x565118b87dc0 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4d0c00640 10 client.15852 handle_command_reply: tid=0 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 client.15852 shutdown 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 msgr2=0x565118b74ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b74ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80c4490 tx=0x7fd4d80095e0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 shutdown_connections 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118b87dc0 0x565118b8a260 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b75970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4d0c00640 0 client.15852 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 --2- 172.21.15.175:0/668615507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b74ef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.767+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 >> 172.21.15.175:0/668615507 conn(0x565118b69360 msgr2=0x565118b6f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.768+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 shutdown_connections 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.768+0000 7fd4f046a740 1 -- 172.21.15.175:0/668615507 wait complete. 2024-07-29T00:39:26.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.768+0000 7fd4f046a740 20 client.15852 trim_cache size 0 max 16384 2024-07-29T00:39:26.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.786+0000 7fd4f046a740 1 Processor -- start 2024-07-29T00:39:26.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 -- start start 2024-07-29T00:39:26.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b76750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b39f0 con 0x565118b6cdd0 2024-07-29T00:39:26.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b55f0 con 0x565118b77fb0 2024-07-29T00:39:26.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4f046a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b40c0 con 0x565118b6d6f0 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b76750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4e6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b8fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39948/0 (socket says 172.21.15.175:39948) 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.787+0000 7fd4e6a00640 1 -- 172.21.15.175:0/2712796781 learned_addr learned my addr 172.21.15.175:0/2712796781 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4e6000640 1 -- 172.21.15.175:0/2712796781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 msgr2=0x565118b76750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4e6000640 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b76750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4e6000640 1 -- 172.21.15.175:0/2712796781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 msgr2=0x565118b8fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4e6000640 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b8fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4e6000640 1 -- 172.21.15.175:0/2712796781 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565118b67920 con 0x565118b6d6f0 2024-07-29T00:39:26.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4e6000640 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b900b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x565118b76530 tx=0x7fd4d4003d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0dd4d2a5eb51a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:26.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4d3400640 1 -- 172.21.15.175:0/2712796781 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400bd70 con 0x565118b6d6f0 2024-07-29T00:39:26.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4d3400640 1 -- 172.21.15.175:0/2712796781 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d4003ad0 con 0x565118b6d6f0 2024-07-29T00:39:26.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.788+0000 7fd4d3400640 1 -- 172.21.15.175:0/2712796781 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4005590 con 0x565118b6d6f0 2024-07-29T00:39:26.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 -- 172.21.15.175:0/2712796781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 msgr2=0x565118b900b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b900b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x565118b76530 tx=0x7fd4d4003d30 comp rx=0 tx=0).stop 2024-07-29T00:39:26.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 -- 172.21.15.175:0/2712796781 shutdown_connections 2024-07-29T00:39:26.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b77fb0 0x565118b76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6d6f0 0x565118b900b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 --2- 172.21.15.175:0/2712796781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b6cdd0 0x565118b8fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 -- 172.21.15.175:0/2712796781 >> 172.21.15.175:0/2712796781 conn(0x565118b69360 msgr2=0x565118b737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.789+0000 7fd4f046a740 1 -- 172.21.15.175:0/2712796781 shutdown_connections 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.790+0000 7fd4f046a740 1 -- 172.21.15.175:0/2712796781 wait complete. 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.790+0000 7fd4f046a740 1 -- start start 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.790+0000 7fd4f046a740 1 Processor -- start 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b80520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b71ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b4e60 con 0x565118b77fb0 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b4790 con 0x565118b6d6f0 2024-07-29T00:39:26.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b19d0 con 0x565118b6cdd0 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4f046a740 10 client.0 _ll_register_callbacks cb 0x7fd4d8009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b71ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b71ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39960/0 (socket says 172.21.15.175:39960) 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 -- 172.21.15.175:0/908544842 learned_addr learned my addr 172.21.15.175:0/908544842 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b80520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e6000640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 -- 172.21.15.175:0/908544842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 msgr2=0x565118b80520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b80520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 -- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 msgr2=0x565118b71ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b71ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e7400640 1 -- 172.21.15.175:0/908544842 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565118b72690 con 0x565118b77fb0 2024-07-29T00:39:26.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.791+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4e7400640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b71ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0004490 tx=0x7fd4e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff69d69afda9755 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4d0c00640 1 -- 172.21.15.175:0/908544842 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0011cd0 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4d0c00640 1 -- 172.21.15.175:0/908544842 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd4e0020420 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d4003ad0 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4d0c00640 1 -- 172.21.15.175:0/908544842 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4e0011e70 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4f046a740 10 client.15858 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565118bf2a70 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.792+0000 7fd4d0c00640 1 -- 172.21.15.175:0/908544842 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d4003ad0 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4d0c00640 1 -- 172.21.15.175:0/908544842 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd4e000f040 con 0x565118b77fb0 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4f046a740 10 client.15858 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4f046a740 10 client.15858 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:39:26.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4f046a740 20 client.15858 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:26.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4f046a740 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118bf3330 0x565118bf5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:26.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4f046a740 4 client.15858 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c698167e", "release": true, "await": true}] 2024-07-29T00:39:26.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c698167e", "release": true, "await": true}) -- 0x565118ada740 con 0x565118bf3330 2024-07-29T00:39:26.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.793+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118bf3330 0x565118bf5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:26.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.794+0000 7fd4e6a00640 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118bf3330 0x565118bf5750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc7bbafc6c9191d server_cookie=eebf793bfe668660 in_seq=0 out_seq=0 2024-07-29T00:39:26.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.794+0000 7fd4d0c00640 10 client.15858 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:26.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.802+0000 7fd4d0c00640 1 -- 172.21.15.175:0/908544842 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x565118ada740 con 0x565118bf3330 2024-07-29T00:39:26.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.802+0000 7fd4d0c00640 10 client.15858 handle_command_reply: tid=0 2024-07-29T00:39:26.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.802+0000 7fd4f046a740 1 client.15858 shutdown 2024-07-29T00:39:26.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.802+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 msgr2=0x565118b71ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.802+0000 7fd4f046a740 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b71ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0004490 tx=0x7fd4e0004a40 comp rx=0 tx=0).stop 2024-07-29T00:39:26.804 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:39:26.804 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 31, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 8, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "c698167e": { 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 8, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 138.5, 2024-07-29T00:39:26.805 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:26.806 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 shutdown_connections 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x565118bf3330 0x565118bf5750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565118b77fb0 0x565118b71ff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565118b6d6f0 0x565118b71ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 --2- 172.21.15.175:0/908544842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565118b6cdd0 0x565118b80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 >> 172.21.15.175:0/908544842 conn(0x565118b69360 msgr2=0x565118b76f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4d0c00640 0 client.15858 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 shutdown_connections 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 1 -- 172.21.15.175:0/908544842 wait complete. 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.803+0000 7fd4f046a740 20 client.15858 trim_cache size 0 max 16384 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd4cc0a2d70 msgr2=0x7fd4cc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd4cc0a2d70 0x7fd4cc0a5230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005fd0 tx=0x7fd4e0009550 comp rx=0 tx=0).stop 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 msgr2=0x7fd4e819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e819e540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009230 tx=0x7fd4d8004270 comp rx=0 tx=0).stop 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 shutdown_connections 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd4cc0a2d70 0x7fd4cc0a5230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4e810dd50 0x7fd4e81a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4e8108670 0x7fd4e819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 --2- 172.21.15.175:0/75107159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4e8107d10 0x7fd4e819e540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 >> 172.21.15.175:0/75107159 conn(0x7fd4e8077670 msgr2=0x7fd4e8078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.804+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 shutdown_connections 2024-07-29T00:39:26.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:26.805+0000 7fd4e7e00640 1 -- 172.21.15.175:0/75107159 wait complete. 2024-07-29T00:39:26.813 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:26.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61272 4.50 MB/sec warmup 77 sec latency 19401.625 ms 2024-07-29T00:39:27.790 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c698167e', total seconds elapsed: 24.1 2024-07-29T00:39:27.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 61938 4.50 MB/sec warmup 78 sec latency 436.513 ms 2024-07-29T00:39:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:27 smithi204 ceph-mon[37368]: pgmap v91: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 821 KiB/s wr, 48 op/s 2024-07-29T00:39:28.136 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:27 smithi175 ceph-mon[32138]: pgmap v91: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 821 KiB/s wr, 48 op/s 2024-07-29T00:39:28.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:27 smithi181 ceph-mon[36706]: pgmap v91: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 821 KiB/s wr, 48 op/s 2024-07-29T00:39:28.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 -- 172.21.15.175:0/165809326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4102a30 msgr2=0x7f51a4102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 --2- 172.21.15.175:0/165809326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4102a30 0x7f51a4102e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f519c009ca0 tx=0x7f519c02a950 comp rx=0 tx=0).stop 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 -- 172.21.15.175:0/165809326 shutdown_connections 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 --2- 172.21.15.175:0/165809326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a410be60 0x7f51a410e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 --2- 172.21.15.175:0/165809326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4103350 0x7f51a410b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 --2- 172.21.15.175:0/165809326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4102a30 0x7f51a4102e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 -- 172.21.15.175:0/165809326 >> 172.21.15.175:0/165809326 conn(0x7f51a40fc490 msgr2=0x7f51a40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 -- 172.21.15.175:0/165809326 shutdown_connections 2024-07-29T00:39:28.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.461+0000 7f51aae00640 1 -- 172.21.15.175:0/165809326 wait complete. 2024-07-29T00:39:28.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.462+0000 7f51aae00640 1 Processor -- start 2024-07-29T00:39:28.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 -- start start 2024-07-29T00:39:28.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4102a30 0x7f51a4100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 0x7f51a4100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a410be60 0x7f51a4101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111cb0 con 0x7f51a4102a30 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111b30 con 0x7f51a4103350 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51aae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111e30 con 0x7f51a410be60 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 0x7f51a4100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 0x7f51a4100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48320/0 (socket says 172.21.15.175:48320) 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.463+0000 7f51a3400640 1 -- 172.21.15.175:0/3471829932 learned_addr learned my addr 172.21.15.175:0/3471829932 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a8e00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a410be60 0x7f51a4101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4102a30 0x7f51a4100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3400640 1 -- 172.21.15.175:0/3471829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a410be60 msgr2=0x7f51a4101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3400640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a410be60 0x7f51a4101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3400640 1 -- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4102a30 msgr2=0x7f51a4100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3400640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4102a30 0x7f51a4100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3400640 1 -- 172.21.15.175:0/3471829932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41a6610 con 0x7f51a4103350 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.464+0000 7f51a3400640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 0x7f51a4100bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f519000d950 tx=0x7f519000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.465+0000 7f51a0c00640 1 -- 172.21.15.175:0/3471829932 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190017030 con 0x7f51a4103350 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.465+0000 7f51a0c00640 1 -- 172.21.15.175:0/3471829932 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5190004020 con 0x7f51a4103350 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.465+0000 7f51a0c00640 1 -- 172.21.15.175:0/3471829932 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190003c70 con 0x7f51a4103350 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.465+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41a6900 con 0x7f51a4103350 2024-07-29T00:39:28.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.465+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a6e90 con 0x7f51a4103350 2024-07-29T00:39:28.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.468+0000 7f51a0c00640 1 -- 172.21.15.175:0/3471829932 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f519001a070 con 0x7f51a4103350 2024-07-29T00:39:28.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.468+0000 7f51ad882740 1 Processor -- start 2024-07-29T00:39:28.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 -- start start 2024-07-29T00:39:28.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b8100b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5609b8059ab0 con 0x5609b80fe720 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5609b7ff1a80 con 0x5609b80f4ef0 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51ad882740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5609b8055a70 con 0x5609b80f3980 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51a8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:51452/0 (socket says 172.21.15.175:51452) 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.469+0000 7f51a3e00640 1 -- 172.21.15.175:0/2507101904 learned_addr learned my addr 172.21.15.175:0/2507101904 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:28.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a0c00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51880a2d70 0x7f51880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3e00640 1 -- 172.21.15.175:0/2507101904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 msgr2=0x5609b80f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3e00640 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3e00640 1 -- 172.21.15.175:0/2507101904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 msgr2=0x5609b8100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a0c00640 1 -- 172.21.15.175:0/3471829932 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f51900cec30 con 0x7f51a4103350 2024-07-29T00:39:28.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3400640 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b8100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3e00640 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b8100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3e00640 1 -- 172.21.15.175:0/2507101904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609b7dcb3c0 con 0x5609b80f3980 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a8e00640 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a3400640 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b8100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.470+0000 7f51a8e00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51880a2d70 0x7f51880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.471+0000 7f51a3e00640 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80f3dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f519c035000 tx=0x7f519c02f6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8bee2c5b5503e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.471+0000 7f5199600640 1 -- 172.21.15.175:0/2507101904 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c034040 con 0x5609b80f3980 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.471+0000 7f5199600640 1 -- 172.21.15.175:0/2507101904 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c02fe20 con 0x5609b80f3980 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.471+0000 7f5199600640 1 -- 172.21.15.175:0/2507101904 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c004740 con 0x5609b80f3980 2024-07-29T00:39:28.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.471+0000 7f51a8e00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51880a2d70 0x7f51880a5230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5194005fd0 tx=0x7f51940046c0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:28.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 -- 172.21.15.175:0/2507101904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 msgr2=0x5609b80f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80f3dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f519c035000 tx=0x7f519c02f6e0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 -- 172.21.15.175:0/2507101904 shutdown_connections 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b8100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80f3dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 --2- 172.21.15.175:0/2507101904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 -- 172.21.15.175:0/2507101904 >> 172.21.15.175:0/2507101904 conn(0x5609b80bc950 msgr2=0x5609b80f0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 -- 172.21.15.175:0/2507101904 shutdown_connections 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 -- 172.21.15.175:0/2507101904 wait complete. 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.472+0000 7f51ad882740 1 -- start start 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 Processor -- start 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b80fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5609b8101460 con 0x5609b80fe720 2024-07-29T00:39:28.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5609b8014c10 con 0x5609b80f4ef0 2024-07-29T00:39:28.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5609b81015e0 con 0x5609b80f3980 2024-07-29T00:39:28.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51ad882740 10 client.0 _ll_register_callbacks cb 0x5609b7fdda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:28.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48346/0 (socket says 172.21.15.175:48346) 2024-07-29T00:39:28.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.473+0000 7f51a3400640 1 -- 172.21.15.175:0/3561524339 learned_addr learned my addr 172.21.15.175:0/3561524339 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a8e00640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b80fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3e00640 1 -- 172.21.15.175:0/3561524339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 msgr2=0x5609b80fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80fbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3e00640 1 -- 172.21.15.175:0/3561524339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 msgr2=0x5609b80fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b80fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3e00640 1 -- 172.21.15.175:0/3561524339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b80181e0 con 0x5609b80f3980 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3400640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80fb640 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f519c02af40 tx=0x7f519c02ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5803e2516e2b3a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.474+0000 7f518e000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f518e000640 1 -- 172.21.15.175:0/3561524339 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c051060 con 0x5609b80f3980 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519c02fe20 con 0x5609b80f3980 2024-07-29T00:39:28.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f518e000640 1 -- 172.21.15.175:0/3561524339 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f519c003d80 con 0x5609b80f3980 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f518e000640 1 -- 172.21.15.175:0/3561524339 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f519c034070 con 0x5609b80f3980 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f51ad882740 10 client.25954 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609b8101460 con 0x5609b80f3980 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f518e000640 1 -- 172.21.15.175:0/3561524339 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c02fe20 con 0x5609b80f3980 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.475+0000 7f518e000640 1 -- 172.21.15.175:0/3561524339 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f519c042070 con 0x5609b80f3980 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51ad882740 10 client.25954 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51ad882740 10 client.25954 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51ad882740 20 client.25954 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51ad882740 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b810e2a0 0x5609b8110690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51ad882740 4 client.25954 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609b8067020 con 0x5609b810e2a0 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51a3400640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b810e2a0 0x5609b8110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f51a3400640 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b810e2a0 0x5609b8110690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2470e32a4ce0d0d1 server_cookie=7040650c66545680 in_seq=0 out_seq=0 2024-07-29T00:39:28.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.476+0000 7f518e000640 10 client.25954 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:39:28.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.477+0000 7f518e000640 1 -- 172.21.15.175:0/3561524339 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5609b8067020 con 0x5609b810e2a0 2024-07-29T00:39:28.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.477+0000 7f518e000640 10 client.25954 handle_command_reply: tid=0 2024-07-29T00:39:28.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.477+0000 7f51ad882740 1 client.25954 shutdown 2024-07-29T00:39:28.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.477+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 msgr2=0x5609b80fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.477+0000 7f51ad882740 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80fb640 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f519c02af40 tx=0x7f519c02ffd0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 shutdown_connections 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80fe720 0x5609b80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b810e2a0 0x5609b8110690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 --2- 172.21.15.175:0/3561524339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80f3980 0x5609b80fb640 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f518e000640 0 client.25954 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 >> 172.21.15.175:0/3561524339 conn(0x5609b80bc950 msgr2=0x5609b80f5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 shutdown_connections 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 1 -- 172.21.15.175:0/3561524339 wait complete. 2024-07-29T00:39:28.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.478+0000 7f51ad882740 20 client.25954 trim_cache size 0 max 16384 2024-07-29T00:39:28.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.494+0000 7f51ad882740 1 Processor -- start 2024-07-29T00:39:28.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.494+0000 7f51ad882740 1 -- start start 2024-07-29T00:39:28.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51ad882740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 0x5609b80fa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51ad882740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51ad882740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b8100050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51ad882740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41af8b0 con 0x5609b80f3980 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51ad882740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41b14b0 con 0x5609b80f4ef0 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51ad882740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41aff80 con 0x5609b80fe720 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 0x5609b80fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 0x5609b80fa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40020/0 (socket says 172.21.15.175:40020) 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51a3e00640 1 -- 172.21.15.175:0/3025228470 learned_addr learned my addr 172.21.15.175:0/3025228470 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51a8e00640 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b8100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.495+0000 7f51a3400640 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a3e00640 1 -- 172.21.15.175:0/3025228470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 msgr2=0x5609b8100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b8100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a3e00640 1 -- 172.21.15.175:0/3025228470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 msgr2=0x5609b80ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a3e00640 1 -- 172.21.15.175:0/3025228470 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609b80f9b90 con 0x5609b80f3980 2024-07-29T00:39:28.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a8e00640 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:28.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f51a3e00640 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 0x5609b80fa620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f519c002230 tx=0x7f519c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e0f56dc89e8fca server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:28.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f5198c00640 1 -- 172.21.15.175:0/3025228470 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c051030 con 0x5609b80f3980 2024-07-29T00:39:28.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.496+0000 7f5198c00640 1 -- 172.21.15.175:0/3025228470 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c02f5c0 con 0x5609b80f3980 2024-07-29T00:39:28.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f5198c00640 1 -- 172.21.15.175:0/3025228470 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c003de0 con 0x5609b80f3980 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 -- 172.21.15.175:0/3025228470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 msgr2=0x5609b80fa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 0x5609b80fa620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f519c002230 tx=0x7f519c009b20 comp rx=0 tx=0).stop 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 -- 172.21.15.175:0/3025228470 shutdown_connections 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f4ef0 0x5609b80ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 --2- 172.21.15.175:0/3025228470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f3980 0x5609b80fa620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.497+0000 7f51ad882740 1 -- 172.21.15.175:0/3025228470 >> 172.21.15.175:0/3025228470 conn(0x5609b80bc950 msgr2=0x5609b80f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.498+0000 7f51ad882740 1 -- 172.21.15.175:0/3025228470 shutdown_connections 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.498+0000 7f51ad882740 1 -- 172.21.15.175:0/3025228470 wait complete. 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.498+0000 7f51ad882740 1 -- start start 2024-07-29T00:39:28.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.498+0000 7f51ad882740 1 Processor -- start 2024-07-29T00:39:28.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 0x5609b81068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f4ef0 0x5609b8106de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b80f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41b0d20 con 0x5609b80f4ef0 2024-07-29T00:39:28.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41b0650 con 0x5609b80f3980 2024-07-29T00:39:28.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41ad890 con 0x5609b80fe720 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51ad882740 10 client.0 _ll_register_callbacks cb 0x7f519400a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 0x5609b81068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f4ef0 0x5609b8106de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 0x5609b81068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48358/0 (socket says 172.21.15.175:48358) 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 -- 172.21.15.175:0/2100813788 learned_addr learned my addr 172.21.15.175:0/2100813788 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a8e00640 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b80f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 -- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 msgr2=0x5609b80f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b80f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 -- 172.21.15.175:0/2100813788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f4ef0 msgr2=0x5609b8106de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f4ef0 0x5609b8106de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.499+0000 7f51a3e00640 1 -- 172.21.15.175:0/2100813788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b80f8e00 con 0x5609b80f3980 2024-07-29T00:39:28.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f51a3e00640 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 0x5609b81068a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f519c009bf0 tx=0x7f519c004000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2341021d74e05c7d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f518e000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f518e000640 1 -- 172.21.15.175:0/2100813788 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c051040 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609b8061530 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f518e000640 1 -- 172.21.15.175:0/2100813788 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f519c02fe50 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f518e000640 1 -- 172.21.15.175:0/2100813788 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f519c06d050 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f51ad882740 10 client.25997 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609b81156f0 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.500+0000 7f518e000640 1 -- 172.21.15.175:0/2100813788 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5609b8061530 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f518e000640 1 -- 172.21.15.175:0/2100813788 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f519c042040 con 0x5609b80f3980 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51ad882740 10 client.25997 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51ad882740 10 client.25997 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51ad882740 20 client.25997 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51ad882740 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b8178da0 0x5609b817b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:28.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51ad882740 4 client.25997 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:39:28.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609b7fde010 con 0x5609b8178da0 2024-07-29T00:39:28.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51a3400640 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b8178da0 0x5609b817b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f51a3400640 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b8178da0 0x5609b817b190 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8940f511ea5ec95 server_cookie=86f67b292447ead5 in_seq=0 out_seq=0 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.501+0000 7f518e000640 10 client.25997 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f518e000640 1 -- 172.21.15.175:0/2100813788 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609b7fde010 con 0x5609b8178da0 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f518e000640 10 client.25997 handle_command_reply: tid=0 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 client.25997 shutdown 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 msgr2=0x5609b81068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 0x5609b81068a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f519c009bf0 tx=0x7f519c004000 comp rx=0 tx=0).stop 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 shutdown_connections 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609b80fe720 0x5609b80f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609b80f4ef0 0x5609b8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5609b8178da0 0x5609b817b190 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 --2- 172.21.15.175:0/2100813788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609b80f3980 0x5609b81068a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f518e000640 0 client.25997 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:39:28.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 >> 172.21.15.175:0/2100813788 conn(0x5609b80bc950 msgr2=0x5609b80fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:28.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 shutdown_connections 2024-07-29T00:39:28.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 1 -- 172.21.15.175:0/2100813788 wait complete. 2024-07-29T00:39:28.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.502+0000 7f51ad882740 20 client.25997 trim_cache size 0 max 16384 2024-07-29T00:39:28.506 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51880a2d70 msgr2=0x7f51880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51880a2d70 0x7f51880a5230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5194005fd0 tx=0x7f51940046c0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 msgr2=0x7f51a4100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 0x7f51a4100bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f519000d950 tx=0x7f519000de10 comp rx=0 tx=0).stop 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 shutdown_connections 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a410be60 0x7f51a4101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51a4103350 0x7f51a4100bd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51880a2d70 0x7f51880a5230 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 --2- 172.21.15.175:0/3471829932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4102a30 0x7f51a4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:28.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 >> 172.21.15.175:0/3471829932 conn(0x7f51a40fc490 msgr2=0x7f51a40fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:28.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 shutdown_connections 2024-07-29T00:39:28.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:28.503+0000 7f51aae00640 1 -- 172.21.15.175:0/3471829932 wait complete. 2024-07-29T00:39:28.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 63703 4.56 MB/sec warmup 79 sec latency 16.631 ms 2024-07-29T00:39:29.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:39:29.863 INFO:tasks.workunit.client.0.smithi175.stdout: 1 64281 4.56 MB/sec warmup 80 sec latency 23.138 ms 2024-07-29T00:39:30.043 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:29 smithi204 ceph-mon[37368]: pgmap v92: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 40 op/s 2024-07-29T00:39:30.193 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:29 smithi175 ceph-mon[32138]: pgmap v92: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 40 op/s 2024-07-29T00:39:30.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:29 smithi181 ceph-mon[36706]: pgmap v92: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 40 op/s 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 -- 172.21.15.175:0/3480824704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 msgr2=0x7f1960107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 --2- 172.21.15.175:0/3480824704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f1960107aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1950009ca0 tx=0x7f195002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 -- 172.21.15.175:0/3480824704 shutdown_connections 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 --2- 172.21.15.175:0/3480824704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1960102060 0x7f196010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 --2- 172.21.15.175:0/3480824704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19601016c0 0x7f1960101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 --2- 172.21.15.175:0/3480824704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f1960107aa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 -- 172.21.15.175:0/3480824704 >> 172.21.15.175:0/3480824704 conn(0x7f19600fd400 msgr2=0x7f19600ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:30.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.749+0000 7f1965e00640 1 -- 172.21.15.175:0/3480824704 shutdown_connections 2024-07-29T00:39:30.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.750+0000 7f1965e00640 1 -- 172.21.15.175:0/3480824704 wait complete. 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 Processor -- start 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 -- start start 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19601016c0 0x7f196019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1960102060 0x7f196019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f196019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1960111ad0 con 0x7f1960102060 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1960111950 con 0x7f19601076c0 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1960111c50 con 0x7f19601016c0 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.751+0000 7f1965400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f196019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f196019ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59580/0 (socket says 172.21.15.175:59580) 2024-07-29T00:39:30.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 -- 172.21.15.175:0/719498736 learned_addr learned my addr 172.21.15.175:0/719498736 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:30.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1964a00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19601016c0 0x7f196019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f195fe00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1960102060 0x7f196019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 -- 172.21.15.175:0/719498736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19601016c0 msgr2=0x7f196019a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19601016c0 0x7f196019a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 -- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1960102060 msgr2=0x7f196019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1960102060 0x7f196019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 -- 172.21.15.175:0/719498736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196019f0f0 con 0x7f19601076c0 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f1965400640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f196019ea10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f195800e970 tx=0x7f195800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.752+0000 7f195d600640 1 -- 172.21.15.175:0/719498736 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195800ccc0 con 0x7f19601076c0 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.753+0000 7f195d600640 1 -- 172.21.15.175:0/719498736 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f195800ce60 con 0x7f19601076c0 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.753+0000 7f195d600640 1 -- 172.21.15.175:0/719498736 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958010490 con 0x7f19601076c0 2024-07-29T00:39:30.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.753+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196019f3e0 con 0x7f19601076c0 2024-07-29T00:39:30.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.753+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a6cc0 con 0x7f19601076c0 2024-07-29T00:39:30.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.756+0000 7f195d600640 1 -- 172.21.15.175:0/719498736 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f1958005020 con 0x7f19601076c0 2024-07-29T00:39:30.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.756+0000 7f1969e95740 1 Processor -- start 2024-07-29T00:39:30.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 -- start start 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a7822b0e0 0x557a78229630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78229fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78234910 0x557a78236d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557a7818fab0 con 0x557a78234910 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a78127a80 con 0x557a78229b70 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1969e95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a7818ba70 con 0x557a7822b0e0 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1964a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78229fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1964a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78229fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59584/0 (socket says 172.21.15.175:59584) 2024-07-29T00:39:30.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f1964a00640 1 -- 172.21.15.175:0/2185321721 learned_addr learned my addr 172.21.15.175:0/2185321721 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.757+0000 7f195fe00640 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a7822b0e0 0x557a78229630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f195fe00640 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78234910 0x557a78236d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f1964a00640 1 -- 172.21.15.175:0/2185321721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a7822b0e0 msgr2=0x557a78229630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f1964a00640 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a7822b0e0 0x557a78229630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f1964a00640 1 -- 172.21.15.175:0/2185321721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78234910 msgr2=0x557a78236d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f1964a00640 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78234910 0x557a78236d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f1964a00640 1 -- 172.21.15.175:0/2185321721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a77f013c0 con 0x557a78229b70 2024-07-29T00:39:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f195fe00640 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78234910 0x557a78236d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:39:30.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f1964a00640 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78229fb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f195002ade0 tx=0x7f1950005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7c9aaf1f2cf484 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:30.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f195d600640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19440a2d70 0x7f19440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.758+0000 7f195d600640 1 -- 172.21.15.175:0/719498736 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f19580cf870 con 0x7f19601076c0 2024-07-29T00:39:30.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.759+0000 7f195fe00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19440a2d70 0x7f19440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.759+0000 7f1955c00640 1 -- 172.21.15.175:0/2185321721 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950031da0 con 0x557a78229b70 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.759+0000 7f1955c00640 1 -- 172.21.15.175:0/2185321721 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1950036030 con 0x557a78229b70 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.759+0000 7f1955c00640 1 -- 172.21.15.175:0/2185321721 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950004240 con 0x557a78229b70 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 -- 172.21.15.175:0/2185321721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 msgr2=0x557a78229fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78229fb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f195002ade0 tx=0x7f1950005fa0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f195fe00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19440a2d70 0x7f19440a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x557a7822a610 tx=0x7f1948005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 -- 172.21.15.175:0/2185321721 shutdown_connections 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78234910 0x557a78236d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78229fb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 --2- 172.21.15.175:0/2185321721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a7822b0e0 0x557a78229630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 -- 172.21.15.175:0/2185321721 >> 172.21.15.175:0/2185321721 conn(0x557a781f2950 msgr2=0x557a78226200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 -- 172.21.15.175:0/2185321721 shutdown_connections 2024-07-29T00:39:30.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.760+0000 7f1969e95740 1 -- 172.21.15.175:0/2185321721 wait complete. 2024-07-29T00:39:30.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 -- start start 2024-07-29T00:39:30.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 Processor -- start 2024-07-29T00:39:30.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78231830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a78231d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a782322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557a782374d0 con 0x557a7822b0e0 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a7814ac10 con 0x557a78229b70 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a78237650 con 0x557a78234910 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.761+0000 7f1969e95740 10 client.0 _ll_register_callbacks cb 0x557a78113a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78231830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78231830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59592/0 (socket says 172.21.15.175:59592) 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 -- 172.21.15.175:0/2337413026 learned_addr learned my addr 172.21.15.175:0/2337413026 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1965400640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a782322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f195fe00640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a78231d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 -- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 msgr2=0x557a782322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a782322b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 -- 172.21.15.175:0/2337413026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 msgr2=0x557a78231d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a78231d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 -- 172.21.15.175:0/2337413026 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7814e1e0 con 0x557a78229b70 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1965400640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a782322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f1964a00640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78231830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1950009890 tx=0x7f1950032de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f05d33012e5fb3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.762+0000 7f194e000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f194e000640 1 -- 172.21.15.175:0/2337413026 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195004a040 con 0x557a78229b70 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f194e000640 1 -- 172.21.15.175:0/2337413026 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1950003b70 con 0x557a78229b70 2024-07-29T00:39:30.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1950036030 con 0x557a78229b70 2024-07-29T00:39:30.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f194e000640 1 -- 172.21.15.175:0/2337413026 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f195005e050 con 0x557a78229b70 2024-07-29T00:39:30.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f1969e95740 10 client.26009 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:30.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557a782374d0 con 0x557a78229b70 2024-07-29T00:39:30.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.763+0000 7f194e000640 1 -- 172.21.15.175:0/2337413026 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1950036030 con 0x557a78229b70 2024-07-29T00:39:30.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f194e000640 1 -- 172.21.15.175:0/2337413026 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f195004d030 con 0x557a78229b70 2024-07-29T00:39:30.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f1969e95740 10 client.26009 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f1969e95740 10 client.26009 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f1969e95740 20 client.26009 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f1969e95740 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a78244330 0x557a78246720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f1969e95740 4 client.26009 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a7819d020 con 0x557a78244330 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.764+0000 7f195fe00640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a78244330 0x557a78246720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.765+0000 7f195fe00640 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a78244330 0x557a78246720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a15a35844d78b6da server_cookie=f119c82e62e77263 in_seq=0 out_seq=0 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.765+0000 7f194e000640 10 client.26009 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.765+0000 7f194e000640 1 -- 172.21.15.175:0/2337413026 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a7819d020 con 0x557a78244330 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.765+0000 7f194e000640 10 client.26009 handle_command_reply: tid=0 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 client.26009 shutdown 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 msgr2=0x557a78231830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78231830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1950009890 tx=0x7f1950032de0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 shutdown_connections 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a78244330 0x557a78246720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a782322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a78231d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 --2- 172.21.15.175:0/2337413026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a78231830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f194e000640 0 client.26009 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 >> 172.21.15.175:0/2337413026 conn(0x557a781f2950 msgr2=0x557a7822bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 shutdown_connections 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.766+0000 7f1969e95740 1 -- 172.21.15.175:0/2337413026 wait complete. 2024-07-29T00:39:30.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.767+0000 7f1969e95740 20 client.26009 trim_cache size 0 max 16384 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.783+0000 7f1969e95740 1 Processor -- start 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.783+0000 7f1969e95740 1 -- start start 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1969e95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78229b70 0x557a782310f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1969e95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 0x557a78235d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1969e95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a78236240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1969e95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19601af5f0 con 0x557a78229b70 2024-07-29T00:39:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1969e95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19601b11f0 con 0x557a7822b0e0 2024-07-29T00:39:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1969e95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19601afcc0 con 0x557a78234910 2024-07-29T00:39:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 0x557a78235d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 0x557a78235d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59604/0 (socket says 172.21.15.175:59604) 2024-07-29T00:39:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 -- 172.21.15.175:0/2350415578 learned_addr learned my addr 172.21.15.175:0/2350415578 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1964a00640 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78229b70 0x557a782310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f1965400640 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a78236240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 -- 172.21.15.175:0/2350415578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 msgr2=0x557a78236240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a78236240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 -- 172.21.15.175:0/2350415578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78229b70 msgr2=0x557a782310f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78229b70 0x557a782310f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.784+0000 7f195fe00640 1 -- 172.21.15.175:0/2350415578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7822fd80 con 0x557a7822b0e0 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.785+0000 7f1964a00640 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78229b70 0x557a782310f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.785+0000 7f195fe00640 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 0x557a78235d00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x557a782310c0 tx=0x7f1948013050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49401f54fcba5d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.785+0000 7f1955200640 1 -- 172.21.15.175:0/2350415578 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194802a630 con 0x557a7822b0e0 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.785+0000 7f1955200640 1 -- 172.21.15.175:0/2350415578 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f194802acb0 con 0x557a7822b0e0 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.785+0000 7f1955200640 1 -- 172.21.15.175:0/2350415578 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480293b0 con 0x557a7822b0e0 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 -- 172.21.15.175:0/2350415578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 msgr2=0x557a78235d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 0x557a78235d00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x557a782310c0 tx=0x7f1948013050 comp rx=0 tx=0).stop 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 -- 172.21.15.175:0/2350415578 shutdown_connections 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a78236240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a7822b0e0 0x557a78235d00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 --2- 172.21.15.175:0/2350415578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a78229b70 0x557a782310f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 -- 172.21.15.175:0/2350415578 >> 172.21.15.175:0/2350415578 conn(0x557a781f2950 msgr2=0x557a78236910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:30.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 -- 172.21.15.175:0/2350415578 shutdown_connections 2024-07-29T00:39:30.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.786+0000 7f1969e95740 1 -- 172.21.15.175:0/2350415578 wait complete. 2024-07-29T00:39:30.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 -- start start 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 Processor -- start 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a7823c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a7823ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a7822eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19601b0a60 con 0x557a7822b0e0 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19601b0390 con 0x557a78229b70 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ad5d0 con 0x557a78234910 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.787+0000 7f1969e95740 10 client.0 _ll_register_callbacks cb 0x7f1950009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a7823c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a7823c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59608/0 (socket says 172.21.15.175:59608) 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 -- 172.21.15.175:0/2568503128 learned_addr learned my addr 172.21.15.175:0/2568503128 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f195fe00640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a7823ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 -- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 msgr2=0x557a7822eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1965400640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a7822eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a7822eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 -- 172.21.15.175:0/2568503128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 msgr2=0x557a7823ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a7823ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 -- 172.21.15.175:0/2568503128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7824cac0 con 0x557a78229b70 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1965400640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a7822eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f1964a00640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a7823c8e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1950004af0 tx=0x7f195002f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddfc09d9a3e8db35 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.788+0000 7f194e000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f194e000640 1 -- 172.21.15.175:0/2568503128 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195002fb60 con 0x557a78229b70 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f194e000640 1 -- 172.21.15.175:0/2568503128 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1950031760 con 0x557a78229b70 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f194802acb0 con 0x557a78229b70 2024-07-29T00:39:30.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f194e000640 1 -- 172.21.15.175:0/2568503128 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1950031980 con 0x557a78229b70 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f1969e95740 10 client.26015 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557a7824b4a0 con 0x557a78229b70 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f194e000640 1 -- 172.21.15.175:0/2568503128 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f194802acb0 con 0x557a78229b70 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f194e000640 1 -- 172.21.15.175:0/2568503128 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f195004d040 con 0x557a78229b70 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f1969e95740 10 client.26015 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f1969e95740 10 client.26015 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.789+0000 7f1969e95740 20 client.26015 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.790+0000 7f1969e95740 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a7824bba0 0x557a782b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.790+0000 7f1969e95740 4 client.26015 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.790+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a78197530 con 0x557a7824bba0 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.790+0000 7f195fe00640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a7824bba0 0x557a782b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:30.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.790+0000 7f195fe00640 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a7824bba0 0x557a782b0fd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38232363b54523ca server_cookie=4b6f2ab7b102e90 in_seq=0 out_seq=0 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.790+0000 7f194e000640 10 client.26015 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f194e000640 1 -- 172.21.15.175:0/2568503128 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a78197530 con 0x557a7824bba0 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f194e000640 10 client.26015 handle_command_reply: tid=0 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 client.26015 shutdown 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 msgr2=0x557a7823c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a7823c8e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1950004af0 tx=0x7f195002f420 comp rx=0 tx=0).stop 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 shutdown_connections 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x557a7824bba0 0x557a782b0fd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a78234910 0x557a7822eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f194e000640 0 client.26015 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a7822b0e0 0x557a7823ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 --2- 172.21.15.175:0/2568503128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a78229b70 0x557a7823c8e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 >> 172.21.15.175:0/2568503128 conn(0x557a781f2950 msgr2=0x557a7822ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:30.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 shutdown_connections 2024-07-29T00:39:30.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 1 -- 172.21.15.175:0/2568503128 wait complete. 2024-07-29T00:39:30.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.791+0000 7f1969e95740 20 client.26015 trim_cache size 0 max 16384 2024-07-29T00:39:30.796 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.792+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19440a2d70 msgr2=0x7f19440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.792+0000 7f1965e00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19440a2d70 0x7f19440a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x557a7822a610 tx=0x7f1948005f90 comp rx=0 tx=0).stop 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.792+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 msgr2=0x7f196019ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.792+0000 7f1965e00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f196019ea10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f195800e970 tx=0x7f195800ed70 comp rx=0 tx=0).stop 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 shutdown_connections 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19601076c0 0x7f196019ea10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19440a2d70 0x7f19440a5230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1960102060 0x7f196019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 --2- 172.21.15.175:0/719498736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19601016c0 0x7f196019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:30.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 >> 172.21.15.175:0/719498736 conn(0x7f19600fd400 msgr2=0x7f196010ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:30.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 shutdown_connections 2024-07-29T00:39:30.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:30.793+0000 7f1965e00640 1 -- 172.21.15.175:0/719498736 wait complete. 2024-07-29T00:39:30.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 65529 4.57 MB/sec warmup 81 sec latency 15.267 ms 2024-07-29T00:39:31.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:39:31.862 INFO:tasks.workunit.client.0.smithi175.stdout: 1 67362 4.66 MB/sec warmup 82 sec latency 13.212 ms 2024-07-29T00:39:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:31 smithi175 ceph-mon[32138]: pgmap v93: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 40 op/s 2024-07-29T00:39:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:31 smithi204 ceph-mon[37368]: pgmap v93: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 40 op/s 2024-07-29T00:39:32.153 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:32.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:31 smithi181 ceph-mon[36706]: pgmap v93: 97 pgs: 97 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 40 op/s 2024-07-29T00:39:32.863 INFO:tasks.workunit.client.0.smithi175.stdout: 1 67781 4.63 MB/sec warmup 83 sec latency 430.402 ms 2024-07-29T00:39:32.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 -- 172.21.15.175:0/2386364034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe034100410 msgr2=0x7fe0341007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 --2- 172.21.15.175:0/2386364034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe034100410 0x7fe0341007f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe02402a3f0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 -- 172.21.15.175:0/2386364034 shutdown_connections 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 --2- 172.21.15.175:0/2386364034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe03410dfc0 0x7fe0341103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 --2- 172.21.15.175:0/2386364034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe034100dc0 0x7fe03410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 --2- 172.21.15.175:0/2386364034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe034100410 0x7fe0341007f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 -- 172.21.15.175:0/2386364034 >> 172.21.15.175:0/2386364034 conn(0x7fe0340fc300 msgr2=0x7fe0340fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.983+0000 7fe038c00640 1 -- 172.21.15.175:0/2386364034 shutdown_connections 2024-07-29T00:39:32.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.984+0000 7fe038c00640 1 -- 172.21.15.175:0/2386364034 wait complete. 2024-07-29T00:39:32.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.984+0000 7fe038c00640 1 Processor -- start 2024-07-29T00:39:32.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 -- start start 2024-07-29T00:39:32.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe034100410 0x7fe03419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 0x7fe03419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03410dfc0 0x7fe03419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe034113d30 con 0x7fe034100dc0 2024-07-29T00:39:32.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe034113bb0 con 0x7fe03410dfc0 2024-07-29T00:39:32.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.985+0000 7fe038c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe034113eb0 con 0x7fe034100410 2024-07-29T00:39:32.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 0x7fe03419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 0x7fe03419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38874/0 (socket says 172.21.15.175:38874) 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe033e00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03410dfc0 0x7fe03419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 -- 172.21.15.175:0/1693087685 learned_addr learned my addr 172.21.15.175:0/1693087685 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe033400640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe034100410 0x7fe03419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 -- 172.21.15.175:0/1693087685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe034100410 msgr2=0x7fe03419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe034100410 0x7fe03419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 -- 172.21.15.175:0/1693087685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03410dfc0 msgr2=0x7fe03419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03410dfc0 0x7fe03419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe032a00640 1 -- 172.21.15.175:0/1693087685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03419f1e0 con 0x7fe034100dc0 2024-07-29T00:39:32.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.986+0000 7fe033e00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03410dfc0 0x7fe03419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:32.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.987+0000 7fe032a00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 0x7fe03419a770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9f0 tx=0x7fe02000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:32.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.987+0000 7fe02be00640 1 -- 172.21.15.175:0/1693087685 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02000cd80 con 0x7fe034100dc0 2024-07-29T00:39:32.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.987+0000 7fe02be00640 1 -- 172.21.15.175:0/1693087685 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe020005500 con 0x7fe034100dc0 2024-07-29T00:39:32.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.987+0000 7fe02be00640 1 -- 172.21.15.175:0/1693087685 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020010690 con 0x7fe034100dc0 2024-07-29T00:39:32.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.987+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03419f4d0 con 0x7fe034100dc0 2024-07-29T00:39:32.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.987+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0341a6db0 con 0x7fe034100dc0 2024-07-29T00:39:32.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.991+0000 7fe03cdc1740 1 Processor -- start 2024-07-29T00:39:32.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 -- start start 2024-07-29T00:39:32.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe02be00640 1 -- 172.21.15.175:0/1693087685 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe0200046c0 con 0x7fe034100dc0 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c501630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 0x562a4c501fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c50c910 0x562a4c50ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a4c467ab0 con 0x562a4c5030e0 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562a4c3ffa80 con 0x562a4c501b70 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.992+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a4c463a70 con 0x562a4c50c910 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe033400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 0x562a4c501fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe033400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 0x562a4c501fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59616/0 (socket says 172.21.15.175:59616) 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe033400640 1 -- 172.21.15.175:0/839467658 learned_addr learned my addr 172.21.15.175:0/839467658 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe032a00640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c50c910 0x562a4c50ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe033400640 1 -- 172.21.15.175:0/839467658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c50c910 msgr2=0x562a4c50ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:32.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe033400640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c50c910 0x562a4c50ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.993+0000 7fe033400640 1 -- 172.21.15.175:0/839467658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 msgr2=0x562a4c501630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:39:32.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe033e00640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c501630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe033400640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c501630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe033400640 1 -- 172.21.15.175:0/839467658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4c1d93c0 con 0x562a4c501b70 2024-07-29T00:39:32.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe02be00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0180a2cf0 0x7fe0180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe032a00640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c50c910 0x562a4c50ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:32.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe033e00640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c501630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:39:32.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe02be00640 1 -- 172.21.15.175:0/1693087685 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe0200cf8b0 con 0x7fe034100dc0 2024-07-29T00:39:32.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.994+0000 7fe033400640 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 0x562a4c501fb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe024005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b26bd48ec0f3628b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:32.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.995+0000 7fe033e00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0180a2cf0 0x7fe0180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:32.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.995+0000 7fe01fe00640 1 -- 172.21.15.175:0/839467658 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024035040 con 0x562a4c501b70 2024-07-29T00:39:32.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.995+0000 7fe01fe00640 1 -- 172.21.15.175:0/839467658 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe024004400 con 0x562a4c501b70 2024-07-29T00:39:32.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.995+0000 7fe01fe00640 1 -- 172.21.15.175:0/839467658 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240035f0 con 0x562a4c501b70 2024-07-29T00:39:32.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 -- 172.21.15.175:0/839467658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 msgr2=0x562a4c501fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe033e00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0180a2cf0 0x7fe0180a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe034069440 tx=0x7fe02c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 0x562a4c501fb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe024005de0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 -- 172.21.15.175:0/839467658 shutdown_connections 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c50c910 0x562a4c50ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c501b70 0x562a4c501fb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/839467658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c501630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 -- 172.21.15.175:0/839467658 >> 172.21.15.175:0/839467658 conn(0x562a4c4ca950 msgr2=0x562a4c4fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 -- 172.21.15.175:0/839467658 shutdown_connections 2024-07-29T00:39:32.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.996+0000 7fe03cdc1740 1 -- 172.21.15.175:0/839467658 wait complete. 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.997+0000 7fe03cdc1740 1 -- start start 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.997+0000 7fe03cdc1740 1 Processor -- start 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c5098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c509df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c50c910 0x562a4c50a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a4c50f650 con 0x562a4c5030e0 2024-07-29T00:39:32.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562a4c422c10 con 0x562a4c50c910 2024-07-29T00:39:32.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a4c50f7d0 con 0x562a4c501b70 2024-07-29T00:39:32.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe03cdc1740 10 client.0 _ll_register_callbacks cb 0x562a4c3eba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:32.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe032a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c509df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe032a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c509df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38888/0 (socket says 172.21.15.175:38888) 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe032a00640 1 -- 172.21.15.175:0/848153110 learned_addr learned my addr 172.21.15.175:0/848153110 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe033e00640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c50c910 0x562a4c50a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.998+0000 7fe033400640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c5098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.999+0000 7fe033e00640 1 -- 172.21.15.175:0/848153110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 msgr2=0x562a4c5098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.999+0000 7fe033e00640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c5098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.999+0000 7fe033e00640 1 -- 172.21.15.175:0/848153110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 msgr2=0x562a4c509df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.999+0000 7fe033e00640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c509df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.999+0000 7fe033e00640 1 -- 172.21.15.175:0/848153110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4c4261e0 con 0x562a4c50c910 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:32.999+0000 7fe032a00640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c509df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe033e00640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c50c910 0x562a4c50a330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0341025f0 tx=0x7fe02c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640a0a298e1bdb5b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:33.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe01cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe01cc00640 1 -- 172.21.15.175:0/848153110 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c028600 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe024004400 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe01cc00640 1 -- 172.21.15.175:0/848153110 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe02c0287a0 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe01cc00640 1 -- 172.21.15.175:0/848153110 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe02c004850 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe03cdc1740 10 client.26027 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.000+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a4c51b400 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.001+0000 7fe01cc00640 1 -- 172.21.15.175:0/848153110 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe024004400 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe01cc00640 1 -- 172.21.15.175:0/848153110 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe02c0203f0 con 0x562a4c50c910 2024-07-29T00:39:33.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe03cdc1740 10 client.26027 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe03cdc1740 10 client.26027 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe03cdc1740 20 client.26027 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c51c810 0x562a4c51ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe03cdc1740 4 client.26027 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a4c475020 con 0x562a4c51c810 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.002+0000 7fe033400640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c51c810 0x562a4c51ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.003+0000 7fe033400640 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c51c810 0x562a4c51ec20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c1fdb1f4e588f3c server_cookie=80461068eeaa941 in_seq=0 out_seq=0 2024-07-29T00:39:33.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.003+0000 7fe01cc00640 10 client.26027 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:39:33.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.003+0000 7fe01cc00640 1 -- 172.21.15.175:0/848153110 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a4c475020 con 0x562a4c51c810 2024-07-29T00:39:33.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.003+0000 7fe01cc00640 10 client.26027 handle_command_reply: tid=0 2024-07-29T00:39:33.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.003+0000 7fe03cdc1740 1 client.26027 shutdown 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c50c910 msgr2=0x562a4c50a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c50c910 0x562a4c50a330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0341025f0 tx=0x7fe02c004a40 comp rx=0 tx=0).stop 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 shutdown_connections 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c51c810 0x562a4c51ec20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c50c910 0x562a4c50a330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c5030e0 0x562a4c509df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/848153110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c5098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe01cc00640 0 client.26027 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 >> 172.21.15.175:0/848153110 conn(0x562a4c4ca950 msgr2=0x562a4c503ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 shutdown_connections 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.004+0000 7fe03cdc1740 1 -- 172.21.15.175:0/848153110 wait complete. 2024-07-29T00:39:33.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.005+0000 7fe03cdc1740 20 client.26027 trim_cache size 0 max 16384 2024-07-29T00:39:33.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 Processor -- start 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 -- start start 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c4fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c5244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c524a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0341af740 con 0x562a4c50c910 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0341b1340 con 0x562a4c5030e0 2024-07-29T00:39:33.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.022+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe0341afe10 con 0x562a4c501b70 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c5244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe033e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c524a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c5244c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59640/0 (socket says 172.21.15.175:59640) 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 -- 172.21.15.175:0/802718539 learned_addr learned my addr 172.21.15.175:0/802718539 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe033400640 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c4fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 -- 172.21.15.175:0/802718539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 msgr2=0x562a4c4fe3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c4fe3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 -- 172.21.15.175:0/802718539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 msgr2=0x562a4c524a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:33.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c524a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.023+0000 7fe032a00640 1 -- 172.21.15.175:0/802718539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4c507e00 con 0x562a4c5030e0 2024-07-29T00:39:33.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.024+0000 7fe032a00640 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c5244c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe020098720 tx=0x7fe0200980b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1413a16bd9474377 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:33.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.024+0000 7fe01f400640 1 -- 172.21.15.175:0/802718539 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0200d5020 con 0x562a4c5030e0 2024-07-29T00:39:33.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.024+0000 7fe01f400640 1 -- 172.21.15.175:0/802718539 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02009c450 con 0x562a4c5030e0 2024-07-29T00:39:33.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.024+0000 7fe01f400640 1 -- 172.21.15.175:0/802718539 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0200a4790 con 0x562a4c5030e0 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 -- 172.21.15.175:0/802718539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 msgr2=0x562a4c5244c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c5244c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe020098720 tx=0x7fe0200980b0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 -- 172.21.15.175:0/802718539 shutdown_connections 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c524a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c5244c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/802718539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c4fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 -- 172.21.15.175:0/802718539 >> 172.21.15.175:0/802718539 conn(0x562a4c4ca950 msgr2=0x562a4c4fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 -- 172.21.15.175:0/802718539 shutdown_connections 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.025+0000 7fe03cdc1740 1 -- 172.21.15.175:0/802718539 wait complete. 2024-07-29T00:39:33.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.026+0000 7fe03cdc1740 1 -- start start 2024-07-29T00:39:33.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.026+0000 7fe03cdc1740 1 Processor -- start 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.026+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c514cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c515230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe03cdc1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c5061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0341b0bb0 con 0x562a4c50c910 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0341b04e0 con 0x562a4c5030e0 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe03cdc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe0341ad720 con 0x562a4c501b70 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe03cdc1740 10 client.0 _ll_register_callbacks cb 0x7fe02c011de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:33.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c515230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe033400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c514cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c515230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59654/0 (socket says 172.21.15.175:59654) 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 -- 172.21.15.175:0/177218057 learned_addr learned my addr 172.21.15.175:0/177218057 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe033e00640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c5061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 -- 172.21.15.175:0/177218057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 msgr2=0x562a4c514cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c514cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 -- 172.21.15.175:0/177218057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 msgr2=0x562a4c5061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c5061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe032a00640 1 -- 172.21.15.175:0/177218057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4c5069f0 con 0x562a4c5030e0 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe033400640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c514cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.027+0000 7fe033e00640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c5061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe032a00640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c515230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe0200ccf20 tx=0x7fe020005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d6974ba173a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe01cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:33.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe01cc00640 1 -- 172.21.15.175:0/177218057 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0200b8020 con 0x562a4c5030e0 2024-07-29T00:39:33.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe01cc00640 1 -- 172.21.15.175:0/177218057 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe02009b1a0 con 0x562a4c5030e0 2024-07-29T00:39:33.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02009c450 con 0x562a4c5030e0 2024-07-29T00:39:33.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe01cc00640 1 -- 172.21.15.175:0/177218057 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0200d5030 con 0x562a4c5030e0 2024-07-29T00:39:33.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe03cdc1740 10 client.26033 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:33.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a4c587010 con 0x562a4c5030e0 2024-07-29T00:39:33.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.028+0000 7fe01cc00640 1 -- 172.21.15.175:0/177218057 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02009c450 con 0x562a4c5030e0 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe01cc00640 1 -- 172.21.15.175:0/177218057 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe02009ae10 con 0x562a4c5030e0 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe03cdc1740 10 client.26033 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe03cdc1740 10 client.26033 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe03cdc1740 20 client.26033 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c587660 0x562a4c589a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe03cdc1740 4 client.26033 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a4c46f530 con 0x562a4c587660 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.029+0000 7fe033400640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c587660 0x562a4c589a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe033400640 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c587660 0x562a4c589a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dc6655bc137d133 server_cookie=7ef6bcc6f5ee3100 in_seq=0 out_seq=0 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe01cc00640 10 client.26033 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe01cc00640 1 -- 172.21.15.175:0/177218057 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a4c46f530 con 0x562a4c587660 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe01cc00640 10 client.26033 handle_command_reply: tid=0 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 client.26033 shutdown 2024-07-29T00:39:33.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 msgr2=0x562a4c515230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c515230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe0200ccf20 tx=0x7fe020005c50 comp rx=0 tx=0).stop 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 shutdown_connections 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a4c50c910 0x562a4c5061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x562a4c587660 0x562a4c589a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a4c5030e0 0x562a4c515230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 --2- 172.21.15.175:0/177218057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4c501b70 0x562a4c514cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe01cc00640 0 client.26033 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 >> 172.21.15.175:0/177218057 conn(0x562a4c4ca950 msgr2=0x562a4c5036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.030+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 shutdown_connections 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.031+0000 7fe03cdc1740 1 -- 172.21.15.175:0/177218057 wait complete. 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.031+0000 7fe03cdc1740 20 client.26033 trim_cache size 0 max 16384 2024-07-29T00:39:33.033 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.031+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0180a2cf0 msgr2=0x7fe0180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.031+0000 7fe038c00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0180a2cf0 0x7fe0180a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe034069440 tx=0x7fe02c009500 comp rx=0 tx=0).stop 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.031+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 msgr2=0x7fe03419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.031+0000 7fe038c00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 0x7fe03419a770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9f0 tx=0x7fe02000edf0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 shutdown_connections 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03410dfc0 0x7fe03419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe0180a2cf0 0x7fe0180a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe034100dc0 0x7fe03419a770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 --2- 172.21.15.175:0/1693087685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe034100410 0x7fe03419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:33.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 >> 172.21.15.175:0/1693087685 conn(0x7fe0340fc300 msgr2=0x7fe0340fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:33.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 shutdown_connections 2024-07-29T00:39:33.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:33.032+0000 7fe038c00640 1 -- 172.21.15.175:0/1693087685 wait complete. 2024-07-29T00:39:33.863 INFO:tasks.workunit.client.0.smithi175.stdout: 1 68600 4.60 MB/sec warmup 84 sec latency 26.841 ms 2024-07-29T00:39:34.084 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:33 smithi175 ceph-mon[32138]: pgmap v94: 97 pgs: 97 active+clean; 49 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-07-29T00:39:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:33 smithi204 ceph-mon[37368]: pgmap v94: 97 pgs: 97 active+clean; 49 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-07-29T00:39:34.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:39:34.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:33 smithi181 ceph-mon[36706]: pgmap v94: 97 pgs: 97 active+clean; 49 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-07-29T00:39:34.557 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:34.863 INFO:tasks.workunit.client.0.smithi175.stdout: 1 70594 4.68 MB/sec warmup 85 sec latency 42.488 ms 2024-07-29T00:39:35.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 -- 172.21.15.175:0/3418358264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b01031a0 msgr2=0x7f88b010b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 --2- 172.21.15.175:0/3418358264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b01031a0 0x7f88b010b690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f889c009900 tx=0x7f889c02a480 comp rx=0 tx=0).stop 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 -- 172.21.15.175:0/3418358264 shutdown_connections 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 --2- 172.21.15.175:0/3418358264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b010bcb0 0x7f88b010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 --2- 172.21.15.175:0/3418358264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b01031a0 0x7f88b010b690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 --2- 172.21.15.175:0/3418358264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b0102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 -- 172.21.15.175:0/3418358264 >> 172.21.15.175:0/3418358264 conn(0x7f88b00fc300 msgr2=0x7f88b00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 -- 172.21.15.175:0/3418358264 shutdown_connections 2024-07-29T00:39:35.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.214+0000 7f88b7000640 1 -- 172.21.15.175:0/3418358264 wait complete. 2024-07-29T00:39:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.215+0000 7f88b7000640 1 Processor -- start 2024-07-29T00:39:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.215+0000 7f88b7000640 1 -- start start 2024-07-29T00:39:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b7000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b01031a0 0x7f88b019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b010bcb0 0x7f88b019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0111b00 con 0x7f88b01031a0 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b7000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0111980 con 0x7f88b0102880 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0111c80 con 0x7f88b010bcb0 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b01031a0 0x7f88b019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59662/0 (socket says 172.21.15.175:59662) 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b5c00640 1 -- 172.21.15.175:0/340456044 learned_addr learned my addr 172.21.15.175:0/340456044 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.216+0000 7f88b6600640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b010bcb0 0x7f88b019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b5c00640 1 -- 172.21.15.175:0/340456044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b010bcb0 msgr2=0x7f88b019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b5c00640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b010bcb0 0x7f88b019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b5c00640 1 -- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b01031a0 msgr2=0x7f88b019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b5c00640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b01031a0 0x7f88b019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b5c00640 1 -- 172.21.15.175:0/340456044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b019f180 con 0x7f88b0102880 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b6600640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b010bcb0 0x7f88b019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b5c00640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b019a1d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88a400e9f0 tx=0x7f88a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88ae000640 1 -- 172.21.15.175:0/340456044 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a400cd80 con 0x7f88b0102880 2024-07-29T00:39:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88ae000640 1 -- 172.21.15.175:0/340456044 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a4010910 con 0x7f88b0102880 2024-07-29T00:39:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88ae000640 1 -- 172.21.15.175:0/340456044 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4018990 con 0x7f88b0102880 2024-07-29T00:39:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.217+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b019f470 con 0x7f88b0102880 2024-07-29T00:39:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.218+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b01a6d50 con 0x7f88b0102880 2024-07-29T00:39:35.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.221+0000 7f88ae000640 1 -- 172.21.15.175:0/340456044 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f88a4010ac0 con 0x7f88b0102880 2024-07-29T00:39:35.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.221+0000 7f88bb25b740 1 Processor -- start 2024-07-29T00:39:35.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.221+0000 7f88bb25b740 1 -- start start 2024-07-29T00:39:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 0x563c2ac5e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac5efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac69910 0x563c2ac6bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88bb25b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c2abc4ab0 con 0x563c2ac600e0 2024-07-29T00:39:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88bb25b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563c2ab5ca80 con 0x563c2ac5eb70 2024-07-29T00:39:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88bb25b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563c2abc0a70 con 0x563c2ac69910 2024-07-29T00:39:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88b6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 0x563c2ac5e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88b6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 0x563c2ac5e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38930/0 (socket says 172.21.15.175:38930) 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88b6600640 1 -- 172.21.15.175:0/3390145388 learned_addr learned my addr 172.21.15.175:0/3390145388 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.222+0000 7f88b5200640 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac69910 0x563c2ac6bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 -- 172.21.15.175:0/3390145388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac69910 msgr2=0x563c2ac6bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac69910 0x563c2ac6bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 -- 172.21.15.175:0/3390145388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 msgr2=0x563c2ac5efb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac5efb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88ae000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88980a2d70 0x7f88980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 -- 172.21.15.175:0/3390145388 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2a9363c0 con 0x563c2ac600e0 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b5200640 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac69910 0x563c2ac6bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88ae000640 1 -- 172.21.15.175:0/340456044 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f88a40cf740 con 0x7f88b0102880 2024-07-29T00:39:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88980a2d70 0x7f88980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.223+0000 7f88b6600640 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 0x563c2ac5e630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88a8011790 tx=0x7f88a8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd445c97f161f3f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.224+0000 7f88a2a00640 1 -- 172.21.15.175:0/3390145388 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a800f9a0 con 0x563c2ac600e0 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.224+0000 7f88a2a00640 1 -- 172.21.15.175:0/3390145388 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a800fc80 con 0x563c2ac600e0 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.224+0000 7f88a2a00640 1 -- 172.21.15.175:0/3390145388 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a801cc60 con 0x563c2ac600e0 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 -- 172.21.15.175:0/3390145388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 msgr2=0x563c2ac5e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 0x563c2ac5e630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88a8011790 tx=0x7f88a8011c50 comp rx=0 tx=0).stop 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88b6600640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88980a2d70 0x7f88980a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88a8020000 tx=0x7f88a8021000 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 -- 172.21.15.175:0/3390145388 shutdown_connections 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac69910 0x563c2ac6bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac5efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 --2- 172.21.15.175:0/3390145388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac600e0 0x563c2ac5e630 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.225+0000 7f88bb25b740 1 -- 172.21.15.175:0/3390145388 >> 172.21.15.175:0/3390145388 conn(0x563c2ac27950 msgr2=0x563c2ac5b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.226+0000 7f88bb25b740 1 -- 172.21.15.175:0/3390145388 shutdown_connections 2024-07-29T00:39:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.226+0000 7f88bb25b740 1 -- 172.21.15.175:0/3390145388 wait complete. 2024-07-29T00:39:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.226+0000 7f88bb25b740 1 -- start start 2024-07-29T00:39:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.226+0000 7f88bb25b740 1 Processor -- start 2024-07-29T00:39:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 0x563c2ac66940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac600e0 0x563c2ac66e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c2ac6c6b0 con 0x563c2ac69910 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563c2ab7fc10 con 0x563c2ac600e0 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563c2ac6c830 con 0x563c2ac5eb70 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88bb25b740 10 client.0 _ll_register_callbacks cb 0x563c2ab48a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 0x563c2ac66940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 0x563c2ac66940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52296/0 (socket says 172.21.15.175:52296) 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88b5c00640 1 -- 172.21.15.175:0/4077016347 learned_addr learned my addr 172.21.15.175:0/4077016347 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88b5200640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac600e0 0x563c2ac66e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.227+0000 7f88b6600640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5c00640 1 -- 172.21.15.175:0/4077016347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac600e0 msgr2=0x563c2ac66e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5c00640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac600e0 0x563c2ac66e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5c00640 1 -- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 msgr2=0x563c2ac673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5c00640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5c00640 1 -- 172.21.15.175:0/4077016347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2ab831e0 con 0x563c2ac5eb70 2024-07-29T00:39:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5200640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac600e0 0x563c2ac66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b6600640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88b5c00640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 0x563c2ac66940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f88a40cce70 tx=0x7f88a409b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f78acc0594db27 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f8897400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f8897400640 1 -- 172.21.15.175:0/4077016347 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4018b30 con 0x563c2ac5eb70 2024-07-29T00:39:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f8897400640 1 -- 172.21.15.175:0/4077016347 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f88a4018cd0 con 0x563c2ac5eb70 2024-07-29T00:39:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a800fc80 con 0x563c2ac5eb70 2024-07-29T00:39:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f8897400640 1 -- 172.21.15.175:0/4077016347 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88a40b9050 con 0x563c2ac5eb70 2024-07-29T00:39:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.228+0000 7f88bb25b740 10 client.26002 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c2ac78530 con 0x563c2ac5eb70 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f8897400640 1 -- 172.21.15.175:0/4077016347 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a800fc80 con 0x563c2ac5eb70 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f8897400640 1 -- 172.21.15.175:0/4077016347 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f88a4097b70 con 0x563c2ac5eb70 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 10 client.26002 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 10 client.26002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 20 client.26002 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ac79a80 0x563c2ac7be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 4 client.26002 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.229+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2abd2020 con 0x563c2ac79a80 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.230+0000 7f88b5200640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ac79a80 0x563c2ac7be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.230+0000 7f88b5200640 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ac79a80 0x563c2ac7be70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f38c06210dc72d7 server_cookie=2243d7fc9f234aef in_seq=0 out_seq=0 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.230+0000 7f8897400640 10 client.26002 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.231+0000 7f8897400640 1 -- 172.21.15.175:0/4077016347 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c2abd2020 con 0x563c2ac79a80 2024-07-29T00:39:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.231+0000 7f8897400640 10 client.26002 handle_command_reply: tid=0 2024-07-29T00:39:35.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.231+0000 7f88bb25b740 1 client.26002 shutdown 2024-07-29T00:39:35.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 msgr2=0x563c2ac66940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 0x563c2ac66940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f88a40cce70 tx=0x7f88a409b6e0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 shutdown_connections 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ac79a80 0x563c2ac7be70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac600e0 0x563c2ac66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4077016347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac5eb70 0x563c2ac66940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 >> 172.21.15.175:0/4077016347 conn(0x563c2ac27950 msgr2=0x563c2ac60f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f8897400640 0 client.26002 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 shutdown_connections 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 1 -- 172.21.15.175:0/4077016347 wait complete. 2024-07-29T00:39:35.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.232+0000 7f88bb25b740 20 client.26002 trim_cache size 0 max 16384 2024-07-29T00:39:35.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 Processor -- start 2024-07-29T00:39:35.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 -- start start 2024-07-29T00:39:35.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac65920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac81590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac81ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01af820 con 0x563c2ac69910 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01b1420 con 0x563c2ac5eb70 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.248+0000 7f88bb25b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01afef0 con 0x563c2ac600e0 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac65920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac65920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59678/0 (socket says 172.21.15.175:59678) 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 -- 172.21.15.175:0/2486128608 learned_addr learned my addr 172.21.15.175:0/2486128608 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5200640 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac81590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b6600640 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac81ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 -- 172.21.15.175:0/2486128608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 msgr2=0x563c2ac81590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac81590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 -- 172.21.15.175:0/2486128608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 msgr2=0x563c2ac81ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac81ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b5c00640 1 -- 172.21.15.175:0/2486128608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2ac64e90 con 0x563c2ac5eb70 2024-07-29T00:39:35.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.249+0000 7f88b6600640 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac81ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:39:35.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.250+0000 7f88b5c00640 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac65920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097b0 tx=0x7f88a409be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1930e8464eff935f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.250+0000 7f88a2000640 1 -- 172.21.15.175:0/2486128608 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a409b0e0 con 0x563c2ac5eb70 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.250+0000 7f88a2000640 1 -- 172.21.15.175:0/2486128608 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a409b280 con 0x563c2ac5eb70 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.250+0000 7f88a2000640 1 -- 172.21.15.175:0/2486128608 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4018d90 con 0x563c2ac5eb70 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 -- 172.21.15.175:0/2486128608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 msgr2=0x563c2ac65920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac65920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097b0 tx=0x7f88a409be10 comp rx=0 tx=0).stop 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 -- 172.21.15.175:0/2486128608 shutdown_connections 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac69910 0x563c2ac81ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac81590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 --2- 172.21.15.175:0/2486128608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac5eb70 0x563c2ac65920 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 -- 172.21.15.175:0/2486128608 >> 172.21.15.175:0/2486128608 conn(0x563c2ac27950 msgr2=0x563c2ac65210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 -- 172.21.15.175:0/2486128608 shutdown_connections 2024-07-29T00:39:35.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.251+0000 7f88bb25b740 1 -- 172.21.15.175:0/2486128608 wait complete. 2024-07-29T00:39:35.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 -- start start 2024-07-29T00:39:35.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 Processor -- start 2024-07-29T00:39:35.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 0x563c2ac71f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac69910 0x563c2ac6d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01b0c90 con 0x563c2ac5eb70 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01b05c0 con 0x563c2ac69910 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.252+0000 7f88bb25b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01ad800 con 0x563c2ac600e0 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88bb25b740 10 client.0 _ll_register_callbacks cb 0x7f88a4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 0x563c2ac71f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 0x563c2ac71f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38950/0 (socket says 172.21.15.175:38950) 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b5c00640 1 -- 172.21.15.175:0/4067124251 learned_addr learned my addr 172.21.15.175:0/4067124251 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac6d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52314/0 (socket says 172.21.15.175:52314) 2024-07-29T00:39:35.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b6600640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac69910 0x563c2ac6d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b6600640 1 -- 172.21.15.175:0/4067124251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 msgr2=0x563c2ac6d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b6600640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac6d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b6600640 1 -- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 msgr2=0x563c2ac71f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b6600640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 0x563c2ac71f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b6600640 1 -- 172.21.15.175:0/4067124251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2ac6dd20 con 0x563c2ac69910 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.253+0000 7f88b5c00640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 0x563c2ac71f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f88b6600640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac69910 0x563c2ac6d7e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x563c2ac6c040 tx=0x7f88a8009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78d5ad8645cdc25 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:35.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f8897400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:35.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f8897400640 1 -- 172.21.15.175:0/4067124251 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8037060 con 0x563c2ac69910 2024-07-29T00:39:35.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f8897400640 1 -- 172.21.15.175:0/4067124251 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f88a8011040 con 0x563c2ac69910 2024-07-29T00:39:35.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a409b280 con 0x563c2ac69910 2024-07-29T00:39:35.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f8897400640 1 -- 172.21.15.175:0/4067124251 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88a802e040 con 0x563c2ac69910 2024-07-29T00:39:35.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f88bb25b740 10 client.26048 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:35.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c2ace41e0 con 0x563c2ac69910 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.254+0000 7f8897400640 1 -- 172.21.15.175:0/4067124251 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a409b280 con 0x563c2ac69910 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f8897400640 1 -- 172.21.15.175:0/4067124251 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f88a8031070 con 0x563c2ac69910 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88bb25b740 10 client.26048 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88bb25b740 10 client.26048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88bb25b740 20 client.26048 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ace4790 0x563c2ace6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88bb25b740 4 client.26048 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c2abcc530 con 0x563c2ace4790 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.255+0000 7f88b5c00640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ace4790 0x563c2ace6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.256+0000 7f88b5c00640 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ace4790 0x563c2ace6bb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f2ff99c709b9b28 server_cookie=30baf522417c38c2 in_seq=0 out_seq=0 2024-07-29T00:39:35.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.256+0000 7f8897400640 10 client.26048 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:39:35.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f8897400640 1 -- 172.21.15.175:0/4067124251 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c2abcc530 con 0x563c2ace4790 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f8897400640 10 client.26048 handle_command_reply: tid=0 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 client.26048 shutdown 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac69910 msgr2=0x563c2ac6d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac69910 0x563c2ac6d7e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x563c2ac6c040 tx=0x7f88a8009fb0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 shutdown_connections 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x563c2ace4790 0x563c2ace6bb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c2ac69910 0x563c2ac6d7e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c2ac600e0 0x563c2ac6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f8897400640 0 client.26048 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 --2- 172.21.15.175:0/4067124251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c2ac5eb70 0x563c2ac71f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 >> 172.21.15.175:0/4067124251 conn(0x563c2ac27950 msgr2=0x563c2ac65a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:35.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.257+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 shutdown_connections 2024-07-29T00:39:35.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.258+0000 7f88bb25b740 1 -- 172.21.15.175:0/4067124251 wait complete. 2024-07-29T00:39:35.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.258+0000 7f88bb25b740 20 client.26048 trim_cache size 0 max 16384 2024-07-29T00:39:35.260 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:39:35.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.258+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88980a2d70 msgr2=0x7f88980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.258+0000 7f88b7000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88980a2d70 0x7f88980a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88a8020000 tx=0x7f88a8021000 comp rx=0 tx=0).stop 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.258+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 msgr2=0x7f88b019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.258+0000 7f88b7000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b019a1d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88a400e9f0 tx=0x7f88a400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 shutdown_connections 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b010bcb0 0x7f88b019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88980a2d70 0x7f88980a5230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b01031a0 0x7f88b019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 --2- 172.21.15.175:0/340456044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0102880 0x7f88b019a1d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 >> 172.21.15.175:0/340456044 conn(0x7f88b00fc300 msgr2=0x7f88b00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 shutdown_connections 2024-07-29T00:39:35.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:35.259+0000 7f88b7000640 1 -- 172.21.15.175:0/340456044 wait complete. 2024-07-29T00:39:35.863 INFO:tasks.workunit.client.0.smithi175.stdout: 1 71349 4.69 MB/sec warmup 86 sec latency 26.908 ms 2024-07-29T00:39:36.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:35 smithi175 ceph-mon[32138]: pgmap v95: 97 pgs: 97 active+clean; 49 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 777 KiB/s wr, 84 op/s 2024-07-29T00:39:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:35 smithi204 ceph-mon[37368]: pgmap v95: 97 pgs: 97 active+clean; 49 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 777 KiB/s wr, 84 op/s 2024-07-29T00:39:36.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:35 smithi181 ceph-mon[36706]: pgmap v95: 97 pgs: 97 active+clean; 49 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 777 KiB/s wr, 84 op/s 2024-07-29T00:39:36.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:39:36.714 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:36.863 INFO:tasks.workunit.client.0.smithi175.stdout: 1 72388 4.69 MB/sec warmup 87 sec latency 18.236 ms 2024-07-29T00:39:37.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 -- 172.21.15.175:0/121260056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741031a0 msgr2=0x7f187410b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 --2- 172.21.15.175:0/121260056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741031a0 0x7f187410b690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1860009900 tx=0x7f186002a4a0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 -- 172.21.15.175:0/121260056 shutdown_connections 2024-07-29T00:39:37.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 --2- 172.21.15.175:0/121260056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 0x7f187410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 --2- 172.21.15.175:0/121260056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741031a0 0x7f187410b690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 --2- 172.21.15.175:0/121260056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1874102880 0x7f1874102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 -- 172.21.15.175:0/121260056 >> 172.21.15.175:0/121260056 conn(0x7f18740fc300 msgr2=0x7f18740fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 -- 172.21.15.175:0/121260056 shutdown_connections 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.430+0000 7f1879c00640 1 -- 172.21.15.175:0/121260056 wait complete. 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.432+0000 7f1879c00640 1 Processor -- start 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.432+0000 7f1879c00640 1 -- start start 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 0x7f1874197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18741031a0 0x7f18741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 0x7f187419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111b00 con 0x7f1874102880 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111980 con 0x7f18741031a0 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111c80 con 0x7f187410bcb0 2024-07-29T00:39:37.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.433+0000 7f1879200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 0x7f187419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 0x7f1874197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 0x7f1874197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38966/0 (socket says 172.21.15.175:38966) 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 -- 172.21.15.175:0/4139623599 learned_addr learned my addr 172.21.15.175:0/4139623599 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 -- 172.21.15.175:0/4139623599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 msgr2=0x7f187419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 0x7f187419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 -- 172.21.15.175:0/4139623599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18741031a0 msgr2=0x7f18741984e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18741031a0 0x7f18741984e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.434+0000 7f1873e00640 1 -- 172.21.15.175:0/4139623599 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187419cf70 con 0x7f1874102880 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.435+0000 7f1879200640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 0x7f187419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.435+0000 7f1873e00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 0x7f1874197fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f186400e9f0 tx=0x7f186400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.435+0000 7f1870c00640 1 -- 172.21.15.175:0/4139623599 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186400cd80 con 0x7f1874102880 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.436+0000 7f1870c00640 1 -- 172.21.15.175:0/4139623599 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1864010910 con 0x7f1874102880 2024-07-29T00:39:37.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.436+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187419d260 con 0x7f1874102880 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.436+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187406b900 con 0x7f1874102880 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.436+0000 7f1870c00640 1 -- 172.21.15.175:0/4139623599 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864018990 con 0x7f1874102880 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 Processor -- start 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 -- start start 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b71d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 0x559e6b71dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 0x559e6b72ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559e6b683ab0 con 0x559e6b71f0e0 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e6b61ba80 con 0x559e6b728910 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.440+0000 7f187de8e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559e6b67fa70 con 0x559e6b71db70 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1870c00640 1 -- 172.21.15.175:0/4139623599 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f1864018b30 con 0x7f1874102880 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b71d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b71d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38968/0 (socket says 172.21.15.175:38968) 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 -- 172.21.15.175:0/3622654473 learned_addr learned my addr 172.21.15.175:0/3622654473 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 -- 172.21.15.175:0/3622654473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 msgr2=0x559e6b71dfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 0x559e6b71dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 -- 172.21.15.175:0/3622654473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 msgr2=0x559e6b72ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 0x559e6b72ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.441+0000 7f1879200640 1 -- 172.21.15.175:0/3622654473 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6b3f53c0 con 0x559e6b71f0e0 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.442+0000 7f1879200640 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b71d630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f186c0117e0 tx=0x7f186c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609953d1e2381a63 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.442+0000 7f1869600640 1 -- 172.21.15.175:0/3622654473 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c003c30 con 0x559e6b71f0e0 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.443+0000 7f1869600640 1 -- 172.21.15.175:0/3622654473 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c00fe80 con 0x559e6b71f0e0 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.443+0000 7f1869600640 1 -- 172.21.15.175:0/3622654473 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c0193e0 con 0x559e6b71f0e0 2024-07-29T00:39:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 -- 172.21.15.175:0/3622654473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 msgr2=0x559e6b71d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f1870c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f18580a2d70 0x7f18580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b71d630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f186c0117e0 tx=0x7f186c011ca0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 -- 172.21.15.175:0/3622654473 shutdown_connections 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 0x559e6b72ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 0x559e6b71dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 --2- 172.21.15.175:0/3622654473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b71d630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.444+0000 7f187de8e740 1 -- 172.21.15.175:0/3622654473 >> 172.21.15.175:0/3622654473 conn(0x559e6b6e6950 msgr2=0x559e6b71a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.445+0000 7f1873e00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f18580a2d70 0x7f18580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.445+0000 7f1870c00640 1 -- 172.21.15.175:0/4139623599 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f18640cf710 con 0x7f1874102880 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.446+0000 7f187de8e740 1 -- 172.21.15.175:0/3622654473 shutdown_connections 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.446+0000 7f187de8e740 1 -- 172.21.15.175:0/3622654473 wait complete. 2024-07-29T00:39:37.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.447+0000 7f1873e00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f18580a2d70 0x7f18580a5230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x559e6b71b060 tx=0x7f1864018420 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.447+0000 7f187de8e740 1 -- start start 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.447+0000 7f187de8e740 1 Processor -- start 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.448+0000 7f187de8e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71db70 0x559e6b725850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.448+0000 7f187de8e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 0x559e6b725d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.448+0000 7f187de8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b728910 0x559e6b7262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f1879200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71db70 0x559e6b725850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f1879200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71db70 0x559e6b725850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59730/0 (socket says 172.21.15.175:59730) 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f1873400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 0x559e6b725d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f1873400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 0x559e6b725d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52348/0 (socket says 172.21.15.175:52348) 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f1879200640 1 -- 172.21.15.175:0/440619489 learned_addr learned my addr 172.21.15.175:0/440619489 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559e6b72fac0 con 0x559e6b728910 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e6b63ec10 con 0x559e6b71db70 2024-07-29T00:39:37.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559e6b72fc40 con 0x559e6b71f0e0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f187de8e740 10 client.0 _ll_register_callbacks cb 0x559e6b607a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.449+0000 7f1879200640 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b728910 0x559e6b7262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.450+0000 7f1873400640 1 -- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71db70 msgr2=0x559e6b725850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.450+0000 7f1873400640 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71db70 0x559e6b725850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.450+0000 7f1873400640 1 -- 172.21.15.175:0/440619489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b728910 msgr2=0x559e6b7262d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.450+0000 7f1873400640 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b728910 0x559e6b7262d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.450+0000 7f1873400640 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6b6421e0 con 0x559e6b71f0e0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.451+0000 7f1873400640 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 0x559e6b725d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x559e6b71e610 tx=0x7f18600048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b920d2b14f5086e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.451+0000 7f185e000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.451+0000 7f185e000640 1 -- 172.21.15.175:0/440619489 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860040070 con 0x559e6b71f0e0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.451+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e6b691020 con 0x559e6b71f0e0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.452+0000 7f185e000640 1 -- 172.21.15.175:0/440619489 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1860005410 con 0x559e6b71f0e0 2024-07-29T00:39:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.452+0000 7f185e000640 1 -- 172.21.15.175:0/440619489 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f186004b050 con 0x559e6b71f0e0 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.452+0000 7f187de8e740 10 client.26017 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.452+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e6b73b940 con 0x559e6b71f0e0 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.452+0000 7f185e000640 1 -- 172.21.15.175:0/440619489 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559e6b691020 con 0x559e6b71f0e0 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f185e000640 1 -- 172.21.15.175:0/440619489 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1860031440 con 0x559e6b71f0e0 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f187de8e740 10 client.26017 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f187de8e740 10 client.26017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f187de8e740 20 client.26017 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f187de8e740 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b73ce40 0x559e6b73f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f187de8e740 4 client.26017 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6b68b530 con 0x559e6b73ce40 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.454+0000 7f1879200640 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b73ce40 0x559e6b73f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.455+0000 7f1879200640 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b73ce40 0x559e6b73f260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24b7e7e2b90d517 server_cookie=b7a559bdb2baf47d in_seq=0 out_seq=0 2024-07-29T00:39:37.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.455+0000 7f185e000640 10 client.26017 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.455+0000 7f185e000640 1 -- 172.21.15.175:0/440619489 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559e6b68b530 con 0x559e6b73ce40 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.455+0000 7f185e000640 10 client.26017 handle_command_reply: tid=0 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.455+0000 7f187de8e740 1 client.26017 shutdown 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 msgr2=0x559e6b725d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 0x559e6b725d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x559e6b71e610 tx=0x7f18600048d0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 shutdown_connections 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b73ce40 0x559e6b73f260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b728910 0x559e6b7262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71f0e0 0x559e6b725d90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 --2- 172.21.15.175:0/440619489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71db70 0x559e6b725850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f185e000640 0 client.26017 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:37.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 >> 172.21.15.175:0/440619489 conn(0x559e6b6e6950 msgr2=0x559e6b71fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 shutdown_connections 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 1 -- 172.21.15.175:0/440619489 wait complete. 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.456+0000 7f187de8e740 20 client.26017 trim_cache size 0 max 16384 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 Processor -- start 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 -- start start 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 0x559e6b715430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b726120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 0x559e6b744b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f185400e850 con 0x559e6b71f0e0 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1854010450 con 0x559e6b728910 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.472+0000 7f187de8e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f185400ef20 con 0x559e6b71db70 2024-07-29T00:39:37.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.473+0000 7f1873400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b726120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.473+0000 7f1873400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b726120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38996/0 (socket says 172.21.15.175:38996) 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.473+0000 7f1873400640 1 -- 172.21.15.175:0/275708375 learned_addr learned my addr 172.21.15.175:0/275708375 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.474+0000 7f1873400640 1 -- 172.21.15.175:0/275708375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 msgr2=0x559e6b715430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.474+0000 7f1873400640 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 0x559e6b715430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.474+0000 7f1873400640 1 -- 172.21.15.175:0/275708375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 msgr2=0x559e6b744b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.474+0000 7f1873400640 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 0x559e6b744b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.474+0000 7f1873400640 1 -- 172.21.15.175:0/275708375 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6b7206c0 con 0x559e6b71f0e0 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.474+0000 7f1873400640 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b726120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f186002a9c0 tx=0x7f1860009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dd44d48032006d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.475+0000 7f1868c00640 1 -- 172.21.15.175:0/275708375 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860040050 con 0x559e6b71f0e0 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.475+0000 7f1868c00640 1 -- 172.21.15.175:0/275708375 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1860004940 con 0x559e6b71f0e0 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.475+0000 7f1868c00640 1 -- 172.21.15.175:0/275708375 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186003c440 con 0x559e6b71f0e0 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 -- 172.21.15.175:0/275708375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 msgr2=0x559e6b726120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b726120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f186002a9c0 tx=0x7f1860009230 comp rx=0 tx=0).stop 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 -- 172.21.15.175:0/275708375 shutdown_connections 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b728910 0x559e6b744b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71f0e0 0x559e6b726120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 --2- 172.21.15.175:0/275708375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b71db70 0x559e6b715430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.476+0000 7f187de8e740 1 -- 172.21.15.175:0/275708375 >> 172.21.15.175:0/275708375 conn(0x559e6b6e6950 msgr2=0x559e6b724940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.477+0000 7f187de8e740 1 -- 172.21.15.175:0/275708375 shutdown_connections 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.477+0000 7f187de8e740 1 -- 172.21.15.175:0/275708375 wait complete. 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.477+0000 7f187de8e740 1 -- start start 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.477+0000 7f187de8e740 1 Processor -- start 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 0x559e6b730510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71f0e0 0x559e6b730a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b728910 0x559e6b7a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f185400fcc0 con 0x559e6b71db70 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f185400f5f0 con 0x559e6b71f0e0 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f185400c830 con 0x559e6b728910 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f187de8e740 10 client.0 _ll_register_callbacks cb 0x7f187406acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 0x559e6b730510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 0x559e6b730510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38998/0 (socket says 172.21.15.175:38998) 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 -- 172.21.15.175:0/1763958225 learned_addr learned my addr 172.21.15.175:0/1763958225 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1873400640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71f0e0 0x559e6b730a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b728910 0x559e6b7a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 -- 172.21.15.175:0/1763958225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b728910 msgr2=0x559e6b7a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b728910 0x559e6b7a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.478+0000 7f1879200640 1 -- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71f0e0 msgr2=0x559e6b730a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f1879200640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71f0e0 0x559e6b730a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f1879200640 1 -- 172.21.15.175:0/1763958225 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6b7a60c0 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f1873400640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71f0e0 0x559e6b730a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f1879200640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 0x559e6b730510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f186c0045b0 tx=0x7f186c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d94f0f46494229 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f185e000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f185e000640 1 -- 172.21.15.175:0/1763958225 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c00fce0 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f185e000640 1 -- 172.21.15.175:0/1763958225 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f186c011030 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.479+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1860004940 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f185e000640 1 -- 172.21.15.175:0/1763958225 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f186c011230 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 10 client.15936 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e6b72b290 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f185e000640 1 -- 172.21.15.175:0/1763958225 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1860004940 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f185e000640 1 -- 172.21.15.175:0/1763958225 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f186c025040 con 0x559e6b71db70 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 10 client.15936 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:37.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 20 client.15936 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b72b820 0x559e6b72dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 4 client.15936 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.480+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e6b608010 con 0x559e6b72b820 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.481+0000 7f1873400640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b72b820 0x559e6b72dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.481+0000 7f1873400640 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b72b820 0x559e6b72dc40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245b1bf721a622fd server_cookie=77e8f724dfd122f0 in_seq=0 out_seq=0 2024-07-29T00:39:37.509 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.481+0000 7f185e000640 10 client.15936 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f185e000640 1 -- 172.21.15.175:0/1763958225 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e6b608010 con 0x559e6b72b820 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f185e000640 10 client.15936 handle_command_reply: tid=0 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 client.15936 shutdown 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 msgr2=0x559e6b730510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 0x559e6b730510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f186c0045b0 tx=0x7f186c011ca0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 shutdown_connections 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559e6b72b820 0x559e6b72dc40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f185e000640 0 client.15936 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e6b728910 0x559e6b7a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6b71f0e0 0x559e6b730a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 --2- 172.21.15.175:0/1763958225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559e6b71db70 0x559e6b730510 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 >> 172.21.15.175:0/1763958225 conn(0x559e6b6e6950 msgr2=0x559e6b724940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.482+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 shutdown_connections 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f187de8e740 1 -- 172.21.15.175:0/1763958225 wait complete. 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f187de8e740 20 client.15936 trim_cache size 0 max 16384 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f18580a2d70 msgr2=0x7f18580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f18580a2d70 0x7f18580a5230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x559e6b71b060 tx=0x7f1864018420 comp rx=0 tx=0).stop 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 msgr2=0x7f1874197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 0x7f1874197fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f186400e9f0 tx=0x7f186400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 shutdown_connections 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f187410bcb0 0x7f187419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18741031a0 0x7f18741984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.483+0000 7f1879c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f18580a2d70 0x7f18580a5230 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.484+0000 7f1879c00640 1 --2- 172.21.15.175:0/4139623599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874102880 0x7f1874197fa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.484+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 >> 172.21.15.175:0/4139623599 conn(0x7f18740fc300 msgr2=0x7f18740fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.484+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 shutdown_connections 2024-07-29T00:39:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:37.484+0000 7f1879c00640 1 -- 172.21.15.175:0/4139623599 wait complete. 2024-07-29T00:39:37.874 INFO:tasks.workunit.client.0.smithi175.stdout: 1 73445 4.66 MB/sec warmup 88 sec latency 347.896 ms 2024-07-29T00:39:38.111 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:37 smithi175 ceph-mon[32138]: pgmap v96: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-07-29T00:39:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:37 smithi181 ceph-mon[36706]: pgmap v96: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-07-29T00:39:38.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:37 smithi204 ceph-mon[37368]: pgmap v96: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-07-29T00:39:38.874 INFO:tasks.workunit.client.0.smithi175.stdout: 1 74624 4.73 MB/sec warmup 89 sec latency 573.137 ms 2024-07-29T00:39:39.875 INFO:tasks.workunit.client.0.smithi175.stdout: 1 75002 4.69 MB/sec warmup 90 sec latency 19.718 ms 2024-07-29T00:39:39.988 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:39:39.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:39:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:39 smithi181 ceph-mon[36706]: pgmap v97: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 133 op/s 2024-07-29T00:39:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:39 smithi175 ceph-mon[32138]: pgmap v97: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 133 op/s 2024-07-29T00:39:40.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:39 smithi204 ceph-mon[37368]: pgmap v97: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 133 op/s 2024-07-29T00:39:40.493 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:40.875 INFO:tasks.workunit.client.0.smithi175.stdout: 1 76404 4.70 MB/sec warmup 91 sec latency 18.566 ms 2024-07-29T00:39:41.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 -- 172.21.15.175:0/4130146620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 msgr2=0x7fdc64103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/4130146620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc64103ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009930 tx=0x7fdc5002a420 comp rx=0 tx=0).stop 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 -- 172.21.15.175:0/4130146620 shutdown_connections 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/4130146620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 0x7fdc64110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/4130146620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc64103ee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/4130146620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc64109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 -- 172.21.15.175:0/4130146620 >> 172.21.15.175:0/4130146620 conn(0x7fdc640ff780 msgr2=0x7fdc64101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 -- 172.21.15.175:0/4130146620 shutdown_connections 2024-07-29T00:39:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.201+0000 7fdc6ac00640 1 -- 172.21.15.175:0/4130146620 wait complete. 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 Processor -- start 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 -- start start 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc6419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 0x7fdc6419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc6419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64113ed0 con 0x7fdc64109a80 2024-07-29T00:39:41.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64113d50 con 0x7fdc64104420 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.203+0000 7fdc6ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64114050 con 0x7fdc64103a80 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 0x7fdc6419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc6419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc6419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 0x7fdc6419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37262/0 (socket says 172.21.15.175:37262) 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc6419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53256/0 (socket says 172.21.15.175:53256) 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63400640 1 -- 172.21.15.175:0/1110111353 learned_addr learned my addr 172.21.15.175:0/1110111353 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 -- 172.21.15.175:0/1110111353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 msgr2=0x7fdc6419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 0x7fdc6419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 -- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 msgr2=0x7fdc6419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc6419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 -- 172.21.15.175:0/1110111353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6419f300 con 0x7fdc64103a80 2024-07-29T00:39:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc68c00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:41.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.204+0000 7fdc63e00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc6419a350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e9f0 tx=0x7fdc5400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:41.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.205+0000 7fdc60c00640 1 -- 172.21.15.175:0/1110111353 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400ccc0 con 0x7fdc64103a80 2024-07-29T00:39:41.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.205+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6419f5f0 con 0x7fdc64103a80 2024-07-29T00:39:41.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.205+0000 7fdc60c00640 1 -- 172.21.15.175:0/1110111353 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc54010910 con 0x7fdc64103a80 2024-07-29T00:39:41.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.205+0000 7fdc60c00640 1 -- 172.21.15.175:0/1110111353 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54018790 con 0x7fdc64103a80 2024-07-29T00:39:41.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.205+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641a6ed0 con 0x7fdc64103a80 2024-07-29T00:39:41.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.208+0000 7fdc60c00640 1 -- 172.21.15.175:0/1110111353 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fdc54010ac0 con 0x7fdc64103a80 2024-07-29T00:39:41.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 Processor -- start 2024-07-29T00:39:41.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 -- start start 2024-07-29T00:39:41.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee57f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e61edbd8a0 con 0x55e61ee62d20 2024-07-29T00:39:41.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e61ed55710 con 0x55e61ee57b40 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.209+0000 7fdc6d658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e61edb9860 con 0x55e61ee58460 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee57f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee57f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37278/0 (socket says 172.21.15.175:37278) 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc68c00640 1 -- 172.21.15.175:0/2893630413 learned_addr learned my addr 172.21.15.175:0/2893630413 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc63e00640 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc68c00640 1 -- 172.21.15.175:0/2893630413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 msgr2=0x55e61ee627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc68c00640 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc68c00640 1 -- 172.21.15.175:0/2893630413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 msgr2=0x55e61ee65110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.210+0000 7fdc60c00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdc480a2cf0 0x7fdc480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.211+0000 7fdc63400640 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.211+0000 7fdc60c00640 1 -- 172.21.15.175:0/1110111353 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdc540cf7d0 con 0x7fdc64103a80 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.211+0000 7fdc63e00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdc480a2cf0 0x7fdc480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.211+0000 7fdc68c00640 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee65110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.211+0000 7fdc68c00640 1 -- 172.21.15.175:0/2893630413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61eb2f6a0 con 0x55e61ee57b40 2024-07-29T00:39:41.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.212+0000 7fdc68c00640 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee57f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c017fa0 tx=0x7fdc5c00e840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415a93d7bdd74c6d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:41.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.212+0000 7fdc63e00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdc480a2cf0 0x7fdc480a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e61ee54f30 tx=0x7fdc540d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:41.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.212+0000 7fdc59200640 1 -- 172.21.15.175:0/2893630413 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004b20 con 0x55e61ee57b40 2024-07-29T00:39:41.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.212+0000 7fdc59200640 1 -- 172.21.15.175:0/2893630413 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c004cc0 con 0x55e61ee57b40 2024-07-29T00:39:41.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.212+0000 7fdc59200640 1 -- 172.21.15.175:0/2893630413 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c020810 con 0x55e61ee57b40 2024-07-29T00:39:41.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 -- 172.21.15.175:0/2893630413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 msgr2=0x55e61ee57f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee57f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c017fa0 tx=0x7fdc5c00e840 comp rx=0 tx=0).stop 2024-07-29T00:39:41.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 -- 172.21.15.175:0/2893630413 shutdown_connections 2024-07-29T00:39:41.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 --2- 172.21.15.175:0/2893630413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee57f20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 -- 172.21.15.175:0/2893630413 >> 172.21.15.175:0/2893630413 conn(0x55e61ee540d0 msgr2=0x55e61ee544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:41.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 -- 172.21.15.175:0/2893630413 shutdown_connections 2024-07-29T00:39:41.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.213+0000 7fdc6d658740 1 -- 172.21.15.175:0/2893630413 wait complete. 2024-07-29T00:39:41.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.214+0000 7fdc6d658740 1 -- start start 2024-07-29T00:39:41.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.214+0000 7fdc6d658740 1 Processor -- start 2024-07-29T00:39:41.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee57b40 0x55e61ee5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee62d20 0x55e61ee60660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e61ee658f0 con 0x55e61ee57b40 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e61ed788c0 con 0x55e61ee62d20 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e61ee65a70 con 0x55e61ee58460 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc6d658740 10 client.0 _ll_register_callbacks cb 0x55e61ed64a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee57b40 0x55e61ee5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee60120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33482/0 (socket says 172.21.15.175:33482) 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 -- 172.21.15.175:0/4283328217 learned_addr learned my addr 172.21.15.175:0/4283328217 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc68c00640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee62d20 0x55e61ee60660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 -- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee62d20 msgr2=0x55e61ee60660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee62d20 0x55e61ee60660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 -- 172.21.15.175:0/4283328217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee57b40 msgr2=0x55e61ee5fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee57b40 0x55e61ee5fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc63400640 1 -- 172.21.15.175:0/4283328217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61ed5bbc0 con 0x55e61ee58460 2024-07-29T00:39:41.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.215+0000 7fdc68c00640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee62d20 0x55e61ee60660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:41.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.216+0000 7fdc63400640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee60120 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009900 tx=0x7fdc50004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08950384bc1e2b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:41.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.216+0000 7fdc4d600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:41.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.216+0000 7fdc4d600640 1 -- 172.21.15.175:0/4283328217 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50035040 con 0x55e61ee58460 2024-07-29T00:39:41.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.216+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c004cc0 con 0x55e61ee58460 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.216+0000 7fdc4d600640 1 -- 172.21.15.175:0/4283328217 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdc50005230 con 0x55e61ee58460 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.217+0000 7fdc4d600640 1 -- 172.21.15.175:0/4283328217 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc50005430 con 0x55e61ee58460 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.217+0000 7fdc6d658740 10 client.26032 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.217+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e61ee71950 con 0x55e61ee58460 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.217+0000 7fdc4d600640 1 -- 172.21.15.175:0/4283328217 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c004cc0 con 0x55e61ee58460 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.217+0000 7fdc4d600640 1 -- 172.21.15.175:0/4283328217 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fdc50003a00 con 0x55e61ee58460 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc6d658740 10 client.26032 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc6d658740 10 client.26032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc6d658740 20 client.26032 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc6d658740 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61ee72e60 0x55e61ee752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc6d658740 4 client.26032 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61ed7dc90 con 0x55e61ee72e60 2024-07-29T00:39:41.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.218+0000 7fdc68c00640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61ee72e60 0x55e61ee752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.219+0000 7fdc68c00640 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61ee72e60 0x55e61ee752e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80130932c337cf03 server_cookie=425f54d4f7f8cf40 in_seq=0 out_seq=0 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.219+0000 7fdc4d600640 10 client.26032 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc4d600640 1 -- 172.21.15.175:0/4283328217 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e61ed7dc90 con 0x55e61ee72e60 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc4d600640 10 client.26032 handle_command_reply: tid=0 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 client.26032 shutdown 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 msgr2=0x55e61ee60120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee60120 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009900 tx=0x7fdc50004270 comp rx=0 tx=0).stop 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 shutdown_connections 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61ee72e60 0x55e61ee752e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc4d600640 0 client.26032 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee62d20 0x55e61ee60660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee58460 0x55e61ee60120 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 --2- 172.21.15.175:0/4283328217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee57b40 0x55e61ee5fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 >> 172.21.15.175:0/4283328217 conn(0x55e61ee540d0 msgr2=0x55e61ee59db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 shutdown_connections 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.220+0000 7fdc6d658740 1 -- 172.21.15.175:0/4283328217 wait complete. 2024-07-29T00:39:41.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.221+0000 7fdc6d658740 20 client.26032 trim_cache size 0 max 16384 2024-07-29T00:39:41.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 Processor -- start 2024-07-29T00:39:41.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 -- start start 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee57b40 0x55e61ee5ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 0x55e61ee7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ab4a0 con 0x55e61ee62d20 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ad0a0 con 0x55e61ee58460 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.243+0000 7fdc6d658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641abb70 con 0x55e61ee57b40 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc63400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 0x55e61ee7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee57b40 0x55e61ee5ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc63400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 0x55e61ee7abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37302/0 (socket says 172.21.15.175:37302) 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc63400640 1 -- 172.21.15.175:0/968388237 learned_addr learned my addr 172.21.15.175:0/968388237 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 -- 172.21.15.175:0/968388237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 msgr2=0x55e61ee7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 0x55e61ee7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 -- 172.21.15.175:0/968388237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 msgr2=0x55e61ee7b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee7b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 -- 172.21.15.175:0/968388237 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61ee65480 con 0x55e61ee57b40 2024-07-29T00:39:41.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc63400640 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 0x55e61ee7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:39:41.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.244+0000 7fdc68c00640 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee57b40 0x55e61ee5ec20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c020470 tx=0x7fdc5c0059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63400ae678b52 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:41.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.245+0000 7fdc4fe00640 1 -- 172.21.15.175:0/968388237 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c003970 con 0x55e61ee57b40 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.245+0000 7fdc4fe00640 1 -- 172.21.15.175:0/968388237 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c0052a0 con 0x55e61ee57b40 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.245+0000 7fdc4fe00640 1 -- 172.21.15.175:0/968388237 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004050 con 0x55e61ee57b40 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.245+0000 7fdc6d658740 1 -- 172.21.15.175:0/968388237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee57b40 msgr2=0x55e61ee5ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.245+0000 7fdc6d658740 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee57b40 0x55e61ee5ec20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c020470 tx=0x7fdc5c0059d0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 -- 172.21.15.175:0/968388237 shutdown_connections 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee62d20 0x55e61ee7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee58460 0x55e61ee7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 --2- 172.21.15.175:0/968388237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee57b40 0x55e61ee5ec20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 -- 172.21.15.175:0/968388237 >> 172.21.15.175:0/968388237 conn(0x55e61ee540d0 msgr2=0x55e61ee5e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 -- 172.21.15.175:0/968388237 shutdown_connections 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 -- 172.21.15.175:0/968388237 wait complete. 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.246+0000 7fdc6d658740 1 -- start start 2024-07-29T00:39:41.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 Processor -- start 2024-07-29T00:39:41.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee58460 0x55e61ee6b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee62d20 0x55e61ee67c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ac910 con 0x55e61ee58460 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ac240 con 0x55e61ee57b40 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641a9480 con 0x55e61ee62d20 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc6d658740 10 client.0 _ll_register_callbacks cb 0x7fdc50005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc68c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee6acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37308/0 (socket says 172.21.15.175:37308) 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc68c00640 1 -- 172.21.15.175:0/632720130 learned_addr learned my addr 172.21.15.175:0/632720130 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc68c00640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee62d20 0x55e61ee67c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.247+0000 7fdc63400640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee58460 0x55e61ee6b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc68c00640 1 -- 172.21.15.175:0/632720130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee62d20 msgr2=0x55e61ee67c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc68c00640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee62d20 0x55e61ee67c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc68c00640 1 -- 172.21.15.175:0/632720130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee58460 msgr2=0x55e61ee6b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc68c00640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee58460 0x55e61ee6b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc68c00640 1 -- 172.21.15.175:0/632720130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61ee681d0 con 0x55e61ee57b40 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc68c00640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee6acc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c017db0 tx=0x7fdc5c032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba7d993e2de0bfd server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc4d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc4d600640 1 -- 172.21.15.175:0/632720130 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c005510 con 0x55e61ee57b40 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.248+0000 7fdc4d600640 1 -- 172.21.15.175:0/632720130 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdc5c042070 con 0x55e61ee57b40 2024-07-29T00:39:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.249+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c0052a0 con 0x55e61ee57b40 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.249+0000 7fdc4d600640 1 -- 172.21.15.175:0/632720130 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c0056b0 con 0x55e61ee57b40 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.249+0000 7fdc6d658740 10 client.26075 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.249+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e61eede2e0 con 0x55e61ee57b40 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.249+0000 7fdc4d600640 1 -- 172.21.15.175:0/632720130 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c0052a0 con 0x55e61ee57b40 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.249+0000 7fdc4d600640 1 -- 172.21.15.175:0/632720130 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fdc5c045020 con 0x55e61ee57b40 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc6d658740 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc6d658740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc6d658740 20 client.26075 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc6d658740 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61eede730 0x55e61eee0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc6d658740 4 client.26075 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e61edcae10 con 0x55e61eede730 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc63400640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61eede730 0x55e61eee0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc4d600640 10 client.26075 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.250+0000 7fdc63400640 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61eede730 0x55e61eee0b50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78bf7bdf63bef135 server_cookie=d69232dfd64ec156 in_seq=0 out_seq=0 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.251+0000 7fdc4d600640 1 -- 172.21.15.175:0/632720130 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e61edcae10 con 0x55e61eede730 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.251+0000 7fdc4d600640 10 client.26075 handle_command_reply: tid=0 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.251+0000 7fdc6d658740 1 client.26075 shutdown 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 msgr2=0x55e61ee6acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee6acc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c017db0 tx=0x7fdc5c032040 comp rx=0 tx=0).stop 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 shutdown_connections 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e61eede730 0x55e61eee0b50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e61ee62d20 0x55e61ee67c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61ee58460 0x55e61ee6b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc4d600640 0 client.26075 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:41.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 --2- 172.21.15.175:0/632720130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61ee57b40 0x55e61ee6acc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 >> 172.21.15.175:0/632720130 conn(0x55e61ee540d0 msgr2=0x55e61ee5ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:41.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 shutdown_connections 2024-07-29T00:39:41.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 1 -- 172.21.15.175:0/632720130 wait complete. 2024-07-29T00:39:41.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.252+0000 7fdc6d658740 20 client.26075 trim_cache size 0 max 16384 2024-07-29T00:39:41.254 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:39:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:39:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "6ccb3f58-4fbb-434a-9f1d-69a756a82e85", 2024-07-29T00:39:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:39:41.255 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:39:41.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdc480a2cf0 msgr2=0x7fdc480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdc480a2cf0 0x7fdc480a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e61ee54f30 tx=0x7fdc540d5530 comp rx=0 tx=0).stop 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 msgr2=0x7fdc6419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc6419a350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e9f0 tx=0x7fdc5400edf0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 shutdown_connections 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdc480a2cf0 0x7fdc480a51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc64109a80 0x7fdc6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc64104420 0x7fdc6419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 --2- 172.21.15.175:0/1110111353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64103a80 0x7fdc6419a350 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.253+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 >> 172.21.15.175:0/1110111353 conn(0x7fdc640ff780 msgr2=0x7fdc640fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.254+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 shutdown_connections 2024-07-29T00:39:41.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:41.254+0000 7fdc6ac00640 1 -- 172.21.15.175:0/1110111353 wait complete. 2024-07-29T00:39:41.875 INFO:tasks.workunit.client.0.smithi175.stdout: 1 78071 4.78 MB/sec warmup 92 sec latency 18.530 ms 2024-07-29T00:39:42.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: pgmap v98: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 133 op/s 2024-07-29T00:39:42.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:39:42.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:41 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: pgmap v98: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 133 op/s 2024-07-29T00:39:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:39:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:41 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: pgmap v98: 97 pgs: 97 active+clean; 52 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 133 op/s 2024-07-29T00:39:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:39:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:41 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:42.358 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:39:42.861 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:39:42.874 INFO:tasks.workunit.client.0.smithi175.stdout: 1 78558 4.75 MB/sec warmup 93 sec latency 21.210 ms 2024-07-29T00:39:43.219 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:43.219 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:43.219 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:39:43.220 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:43.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:43.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:43.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:39:43.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:43.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:43.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:43.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:39:43.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:39:43.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 -- 172.21.15.175:0/1607259863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f372410dd50 msgr2=0x7f372410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 --2- 172.21.15.175:0/1607259863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f372410dd50 0x7f372410e130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3714009930 tx=0x7f371402a420 comp rx=0 tx=0).stop 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 -- 172.21.15.175:0/1607259863 shutdown_connections 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 --2- 172.21.15.175:0/1607259863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3724108670 0x7f3724114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 --2- 172.21.15.175:0/1607259863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3724107d10 0x7f3724108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 --2- 172.21.15.175:0/1607259863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f372410dd50 0x7f372410e130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 -- 172.21.15.175:0/1607259863 >> 172.21.15.175:0/1607259863 conn(0x7f3724077670 msgr2=0x7f3724079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 -- 172.21.15.175:0/1607259863 shutdown_connections 2024-07-29T00:39:43.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.526+0000 7f3728a00640 1 -- 172.21.15.175:0/1607259863 wait complete. 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.527+0000 7f3728a00640 1 Processor -- start 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.527+0000 7f3728a00640 1 -- start start 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3728a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 0x7f372419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3728a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3724108670 0x7f372419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3728a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372410dd50 0x7f37241a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3728a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37241180d0 con 0x7f372410dd50 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3728a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3724117f50 con 0x7f3724108670 2024-07-29T00:39:43.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3728a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3724118250 con 0x7f3724107d10 2024-07-29T00:39:43.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3722800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 0x7f372419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3722800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 0x7f372419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33518/0 (socket says 172.21.15.175:33518) 2024-07-29T00:39:43.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3722800640 1 -- 172.21.15.175:0/213949577 learned_addr learned my addr 172.21.15.175:0/213949577 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3721e00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3724108670 0x7f372419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3723200640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372410dd50 0x7f37241a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3722800640 1 -- 172.21.15.175:0/213949577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3724108670 msgr2=0x7f372419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.528+0000 7f3722800640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3724108670 0x7f372419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f3722800640 1 -- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372410dd50 msgr2=0x7f37241a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f3722800640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372410dd50 0x7f37241a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f3722800640 1 -- 172.21.15.175:0/213949577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37241a3540 con 0x7f3724107d10 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f3722800640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 0x7f372419e590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f371402a940 tx=0x7f3714003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f371b400640 1 -- 172.21.15.175:0/213949577 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371403f070 con 0x7f3724107d10 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f371b400640 1 -- 172.21.15.175:0/213949577 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3714031c90 con 0x7f3724107d10 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f371b400640 1 -- 172.21.15.175:0/213949577 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714035030 con 0x7f3724107d10 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37241a37d0 con 0x7f3724107d10 2024-07-29T00:39:43.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.529+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241a3c60 con 0x7f3724107d10 2024-07-29T00:39:43.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.532+0000 7f371b400640 1 -- 172.21.15.175:0/213949577 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3714042070 con 0x7f3724107d10 2024-07-29T00:39:43.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.533+0000 7f372be93740 1 Processor -- start 2024-07-29T00:39:43.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.533+0000 7f372be93740 1 -- start start 2024-07-29T00:39:43.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.533+0000 7f372be93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.533+0000 7f372be93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.533+0000 7f372be93740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f372be93740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556177b70ad0 con 0x556177c16cf0 2024-07-29T00:39:43.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f372be93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556177b08af0 con 0x556177c0c430 2024-07-29T00:39:43.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f372be93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556177b6ca90 con 0x556177c0bb10 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33520/0 (socket says 172.21.15.175:33520) 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 -- 172.21.15.175:0/166962758 learned_addr learned my addr 172.21.15.175:0/166962758 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3722800640 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3721e00640 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 -- 172.21.15.175:0/166962758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 msgr2=0x556177c167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 -- 172.21.15.175:0/166962758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 msgr2=0x556177c190e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c190e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f3723200640 1 -- 172.21.15.175:0/166962758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561778e23c0 con 0x556177c0bb10 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.534+0000 7f371b400640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f37080a2d70 0x7f37080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.535+0000 7f3723200640 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0bef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f371c009f40 tx=0x7f371c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96dfd568271dd9c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.535+0000 7f371b400640 1 -- 172.21.15.175:0/213949577 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f37140efa50 con 0x7f3724107d10 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.535+0000 7f370f400640 1 -- 172.21.15.175:0/166962758 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004d20 con 0x556177c0bb10 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.535+0000 7f370f400640 1 -- 172.21.15.175:0/166962758 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c004050 con 0x556177c0bb10 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.535+0000 7f370f400640 1 -- 172.21.15.175:0/166962758 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c005280 con 0x556177c0bb10 2024-07-29T00:39:43.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.535+0000 7f3722800640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f37080a2d70 0x7f37080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 -- 172.21.15.175:0/166962758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 msgr2=0x556177c0bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0bef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f371c009f40 tx=0x7f371c0118d0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f3722800640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f37080a2d70 0x7f37080a5230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x556177c08f00 tx=0x7f3714009560 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:43.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 -- 172.21.15.175:0/166962758 shutdown_connections 2024-07-29T00:39:43.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c190e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 --2- 172.21.15.175:0/166962758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0bef0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 -- 172.21.15.175:0/166962758 >> 172.21.15.175:0/166962758 conn(0x556177c080a0 msgr2=0x556177c08480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:43.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 -- 172.21.15.175:0/166962758 shutdown_connections 2024-07-29T00:39:43.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.536+0000 7f372be93740 1 -- 172.21.15.175:0/166962758 wait complete. 2024-07-29T00:39:43.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 -- start start 2024-07-29T00:39:43.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 Processor -- start 2024-07-29T00:39:43.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0bb10 0x556177c13bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 0x556177c14100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c14640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556177c198c0 con 0x556177c16cf0 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556177b2bbc0 con 0x556177c0bb10 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.537+0000 7f372be93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556177c19a40 con 0x556177c0c430 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f372be93740 10 client.0 _ll_register_callbacks cb 0x556177b17d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 0x556177c14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 0x556177c14100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33534/0 (socket says 172.21.15.175:33534) 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 -- 172.21.15.175:0/3715774153 learned_addr learned my addr 172.21.15.175:0/3715774153 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3723200640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0bb10 0x556177c13bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3723200640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c14640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 -- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0bb10 msgr2=0x556177c13bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0bb10 0x556177c13bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 -- 172.21.15.175:0/3715774153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 msgr2=0x556177c14640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c14640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3721e00640 1 -- 172.21.15.175:0/3715774153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556177c22db0 con 0x556177c0c430 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.538+0000 7f3723200640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0bb10 0x556177c13bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f3721e00640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 0x556177c14100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3710012710 tx=0x7f3710012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4901fb299e1a27fb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f3703e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f3703e00640 1 -- 172.21.15.175:0/3715774153 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000f920 con 0x556177c0c430 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f3703e00640 1 -- 172.21.15.175:0/3715774153 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f371000fac0 con 0x556177c0c430 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c004050 con 0x556177c0c430 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f3703e00640 1 -- 172.21.15.175:0/3715774153 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f371000fcc0 con 0x556177c0c430 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f372be93740 10 client.26050 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:43.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.539+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556177c25780 con 0x556177c0c430 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f3703e00640 1 -- 172.21.15.175:0/3715774153 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c004050 con 0x556177c0c430 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f3703e00640 1 -- 172.21.15.175:0/3715774153 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f37100244b0 con 0x556177c0c430 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f372be93740 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f372be93740 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f372be93740 20 client.26050 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f372be93740 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c26d00 0x556177c29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f372be93740 4 client.26050 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556177b7e040 con 0x556177c26d00 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.540+0000 7f3723200640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c26d00 0x556177c29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.541+0000 7f3723200640 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c26d00 0x556177c29120 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91145cfb5bb1daf server_cookie=7ff38d0b1d957351 in_seq=0 out_seq=0 2024-07-29T00:39:43.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.541+0000 7f3703e00640 10 client.26050 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.542+0000 7f3703e00640 1 -- 172.21.15.175:0/3715774153 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556177b7e040 con 0x556177c26d00 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.542+0000 7f3703e00640 10 client.26050 handle_command_reply: tid=0 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.542+0000 7f372be93740 1 client.26050 shutdown 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.542+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 msgr2=0x556177c14100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.542+0000 7f372be93740 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 0x556177c14100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3710012710 tx=0x7f3710012bd0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 shutdown_connections 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c26d00 0x556177c29120 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c14640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0c430 0x556177c14100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 --2- 172.21.15.175:0/3715774153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0bb10 0x556177c13bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 >> 172.21.15.175:0/3715774153 conn(0x556177c080a0 msgr2=0x556177c0dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:43.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f3703e00640 0 client.26050 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:43.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 shutdown_connections 2024-07-29T00:39:43.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 1 -- 172.21.15.175:0/3715774153 wait complete. 2024-07-29T00:39:43.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.543+0000 7f372be93740 20 client.26050 trim_cache size 0 max 16384 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 Processor -- start 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 -- start start 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c14490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c15b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37241b3b20 con 0x556177c16cf0 2024-07-29T00:39:43.565 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f37241b5720 con 0x556177c0c430 2024-07-29T00:39:43.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f372be93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37241b41f0 con 0x556177c0bb10 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f3723200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f3721e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c14490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f3721e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c14490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37358/0 (socket says 172.21.15.175:37358) 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f3723200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33540/0 (socket says 172.21.15.175:33540) 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.565+0000 7f3721e00640 1 -- 172.21.15.175:0/3461705713 learned_addr learned my addr 172.21.15.175:0/3461705713 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3723200640 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c15b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3721e00640 1 -- 172.21.15.175:0/3461705713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 msgr2=0x556177c0f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3721e00640 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3721e00640 1 -- 172.21.15.175:0/3461705713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 msgr2=0x556177c15b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3721e00640 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c15b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3721e00640 1 -- 172.21.15.175:0/3461705713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556177c07ed0 con 0x556177c0c430 2024-07-29T00:39:43.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f3721e00640 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c14490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3710024ae0 tx=0x7f37100096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e709ccd8ccb2ed6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:43.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.566+0000 7f370ea00640 1 -- 172.21.15.175:0/3461705713 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710005710 con 0x556177c0c430 2024-07-29T00:39:43.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.567+0000 7f370ea00640 1 -- 172.21.15.175:0/3461705713 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371000f920 con 0x556177c0c430 2024-07-29T00:39:43.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.567+0000 7f370ea00640 1 -- 172.21.15.175:0/3461705713 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710010b70 con 0x556177c0c430 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.567+0000 7f372be93740 1 -- 172.21.15.175:0/3461705713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 msgr2=0x556177c14490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.567+0000 7f372be93740 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c14490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3710024ae0 tx=0x7f37100096a0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.567+0000 7f372be93740 1 -- 172.21.15.175:0/3461705713 shutdown_connections 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.568+0000 7f372be93740 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c15b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.568+0000 7f372be93740 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c14490 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.568+0000 7f372be93740 1 --2- 172.21.15.175:0/3461705713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c0f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.568+0000 7f372be93740 1 -- 172.21.15.175:0/3461705713 >> 172.21.15.175:0/3461705713 conn(0x556177c080a0 msgr2=0x556177c15420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.568+0000 7f372be93740 1 -- 172.21.15.175:0/3461705713 shutdown_connections 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.568+0000 7f372be93740 1 -- 172.21.15.175:0/3461705713 wait complete. 2024-07-29T00:39:43.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.569+0000 7f372be93740 1 -- start start 2024-07-29T00:39:43.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.569+0000 7f372be93740 1 Processor -- start 2024-07-29T00:39:43.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c1a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37241b4f90 con 0x556177c16cf0 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f37241b48c0 con 0x556177c0c430 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37241b1b00 con 0x556177c0bb10 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f372be93740 10 client.0 _ll_register_callbacks cb 0x7f3710009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c1f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37360/0 (socket says 172.21.15.175:37360) 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 -- 172.21.15.175:0/3373502788 learned_addr learned my addr 172.21.15.175:0/3373502788 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3723200640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3723200640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c1a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 -- 172.21.15.175:0/3373502788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 msgr2=0x556177c1ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c1ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.575 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 -- 172.21.15.175:0/3373502788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 msgr2=0x556177c1a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:39:43.575 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c1a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.575 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.570+0000 7f3721e00640 1 -- 172.21.15.175:0/3373502788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556177c1aa40 con 0x556177c0c430 2024-07-29T00:39:43.575 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f3721e00640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c1f340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f371000a010 tx=0x7f3710005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c7e19d423ac4857 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:39:43.575 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f3703e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:39:43.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f3703e00640 1 -- 172.21.15.175:0/3373502788 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710005710 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f3703e00640 1 -- 172.21.15.175:0/3373502788 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f37100050c0 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371000f920 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f3703e00640 1 -- 172.21.15.175:0/3373502788 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f371003c050 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f372be93740 10 client.26093 fetch_fsmap learned FSMap version 31 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.571+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556177c914b0 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f3703e00640 1 -- 172.21.15.175:0/3373502788 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371000f920 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f3703e00640 1 -- 172.21.15.175:0/3373502788 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f3710034070 con 0x556177c0c430 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f372be93740 10 client.26093 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f372be93740 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f372be93740 20 client.26093 populate_metadata read hostname 'smithi175' 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f372be93740 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c91b50 0x556177c93f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f372be93740 4 client.26093 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:39:43.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556177b78550 con 0x556177c91b50 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.572+0000 7f3723200640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c91b50 0x556177c93f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f3723200640 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c91b50 0x556177c93f70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be67f09e3a2f79d1 server_cookie=f5bb8b760099cf3 in_seq=0 out_seq=0 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f3703e00640 10 client.26093 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f3703e00640 1 -- 172.21.15.175:0/3373502788 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556177b78550 con 0x556177c91b50 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f3703e00640 10 client.26093 handle_command_reply: tid=0 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 client.26093 shutdown 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 msgr2=0x556177c1f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c1f340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f371000a010 tx=0x7f3710005bb0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 shutdown_connections 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556177c91b50 0x556177c93f70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556177c16cf0 0x556177c1a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556177c0c430 0x556177c1f340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f3703e00640 0 client.26093 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 --2- 172.21.15.175:0/3373502788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556177c0bb10 0x556177c1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.573+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 >> 172.21.15.175:0/3373502788 conn(0x556177c080a0 msgr2=0x556177c15420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.574+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 shutdown_connections 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.574+0000 7f372be93740 1 -- 172.21.15.175:0/3373502788 wait complete. 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.574+0000 7f372be93740 20 client.26093 trim_cache size 0 max 16384 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stdout: "6ccb3f58-4fbb-434a-9f1d-69a756a82e85": { 2024-07-29T00:39:43.579 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:39:43.580 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "6ccb3f58-4fbb-434a-9f1d-69a756a82e85", 2024-07-29T00:39:43.580 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:39:43.580 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:43.580 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:39:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f37080a2d70 msgr2=0x7f37080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f37080a2d70 0x7f37080a5230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x556177c08f00 tx=0x7f3714009560 comp rx=0 tx=0).stop 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 msgr2=0x7f372419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 0x7f372419e590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f371402a940 tx=0x7f3714003b20 comp rx=0 tx=0).stop 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 shutdown_connections 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f37080a2d70 0x7f37080a5230 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372410dd50 0x7f37241a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3724108670 0x7f372419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 --2- 172.21.15.175:0/213949577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3724107d10 0x7f372419e590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 >> 172.21.15.175:0/213949577 conn(0x7f3724077670 msgr2=0x7f3724078e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 shutdown_connections 2024-07-29T00:39:43.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:39:43.575+0000 7f3728a00640 1 -- 172.21.15.175:0/213949577 wait complete. 2024-07-29T00:39:43.875 INFO:tasks.workunit.client.0.smithi175.stdout: 1 80268 4.76 MB/sec warmup 94 sec latency 15.551 ms 2024-07-29T00:39:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:43 smithi181 ceph-mon[36706]: pgmap v99: 97 pgs: 97 active+clean; 54 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-07-29T00:39:44.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:43 smithi175 ceph-mon[32138]: pgmap v99: 97 pgs: 97 active+clean; 54 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-07-29T00:39:44.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:43 smithi204 ceph-mon[37368]: pgmap v99: 97 pgs: 97 active+clean; 54 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-07-29T00:39:44.608 INFO:tasks.cephfs.filesystem:scrub status for tag:6ccb3f58-4fbb-434a-9f1d-69a756a82e85 - {'path': '/', 'tag': '6ccb3f58-4fbb-434a-9f1d-69a756a82e85', 'options': 'recursive,force'} 2024-07-29T00:39:44.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 81595 4.83 MB/sec warmup 95 sec latency 26.417 ms 2024-07-29T00:39:45.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 82032 4.80 MB/sec warmup 96 sec latency 24.977 ms 2024-07-29T00:39:46.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:45 smithi181 ceph-mon[36706]: pgmap v100: 97 pgs: 97 active+clean; 54 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-07-29T00:39:46.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:45 smithi175 ceph-mon[32138]: pgmap v100: 97 pgs: 97 active+clean; 54 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-07-29T00:39:46.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:45 smithi204 ceph-mon[37368]: pgmap v100: 97 pgs: 97 active+clean; 54 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-07-29T00:39:46.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 82893 4.77 MB/sec warmup 97 sec latency 17.009 ms 2024-07-29T00:39:47.235 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:46 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:39:47.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:46 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:39:47.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:46 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:39:47.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 84802 4.84 MB/sec warmup 98 sec latency 28.711 ms 2024-07-29T00:39:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:47 smithi181 ceph-mon[36706]: pgmap v101: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-07-29T00:39:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:47 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:39:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:47 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:39:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:47 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:39:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:47 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:39:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:47 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:39:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:47 smithi175 ceph-mon[32138]: pgmap v101: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-07-29T00:39:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:47 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:39:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:47 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:39:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:47 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:39:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:47 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:39:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:47 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:39:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:47 smithi204 ceph-mon[37368]: pgmap v101: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-07-29T00:39:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:47 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:39:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:47 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:39:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:47 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:39:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:47 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:39:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:47 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:39:48.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 85472 4.84 MB/sec warmup 99 sec latency 21.970 ms 2024-07-29T00:39:49.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:48 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:39:49.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:48 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:39:49.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:48 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:39:49.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:48 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:39:49.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:48 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:39:49.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:48 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:39:49.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 86089 4.81 MB/sec warmup 100 sec latency 33.284 ms 2024-07-29T00:39:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:49 smithi181 ceph-mon[36706]: pgmap v102: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-07-29T00:39:50.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:49 smithi175 ceph-mon[32138]: pgmap v102: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-07-29T00:39:50.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:49 smithi204 ceph-mon[37368]: pgmap v102: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-07-29T00:39:50.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 87631 4.82 MB/sec warmup 101 sec latency 17.103 ms 2024-07-29T00:39:51.876 INFO:tasks.workunit.client.0.smithi175.stdout: 1 88882 4.88 MB/sec warmup 102 sec latency 18.637 ms 2024-07-29T00:39:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:51 smithi181 ceph-mon[36706]: pgmap v103: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-07-29T00:39:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:51 smithi175 ceph-mon[32138]: pgmap v103: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-07-29T00:39:52.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:51 smithi204 ceph-mon[37368]: pgmap v103: 97 pgs: 97 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-07-29T00:39:52.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 89200 4.84 MB/sec warmup 103 sec latency 22.696 ms 2024-07-29T00:39:53.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 90366 4.85 MB/sec warmup 104 sec latency 16.969 ms 2024-07-29T00:39:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:53 smithi181 ceph-mon[36706]: pgmap v104: 97 pgs: 97 active+clean; 60 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-07-29T00:39:54.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:53 smithi175 ceph-mon[32138]: pgmap v104: 97 pgs: 97 active+clean; 60 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-07-29T00:39:54.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:53 smithi204 ceph-mon[37368]: pgmap v104: 97 pgs: 97 active+clean; 60 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-07-29T00:39:54.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 91920 4.88 MB/sec warmup 105 sec latency 17.254 ms 2024-07-29T00:39:55.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 92630 4.89 MB/sec warmup 106 sec latency 29.434 ms 2024-07-29T00:39:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:55 smithi181 ceph-mon[36706]: pgmap v105: 97 pgs: 97 active+clean; 60 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 679 KiB/s wr, 124 op/s 2024-07-29T00:39:56.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:55 smithi175 ceph-mon[32138]: pgmap v105: 97 pgs: 97 active+clean; 60 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 679 KiB/s wr, 124 op/s 2024-07-29T00:39:56.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:55 smithi204 ceph-mon[37368]: pgmap v105: 97 pgs: 97 active+clean; 60 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 679 KiB/s wr, 124 op/s 2024-07-29T00:39:56.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 93424 4.86 MB/sec warmup 107 sec latency 19.468 ms 2024-07-29T00:39:57.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 94879 4.87 MB/sec warmup 108 sec latency 60.587 ms 2024-07-29T00:39:58.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:57 smithi181 ceph-mon[36706]: pgmap v106: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 948 KiB/s wr, 180 op/s 2024-07-29T00:39:58.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:57 smithi175 ceph-mon[32138]: pgmap v106: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 948 KiB/s wr, 180 op/s 2024-07-29T00:39:58.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:57 smithi204 ceph-mon[37368]: pgmap v106: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 948 KiB/s wr, 180 op/s 2024-07-29T00:39:58.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 96026 4.93 MB/sec warmup 109 sec latency 132.168 ms 2024-07-29T00:39:59.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:39:58 smithi181 ceph-mon[36706]: pgmap v107: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 632 KiB/s wr, 117 op/s 2024-07-29T00:39:59.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:39:58 smithi175 ceph-mon[32138]: pgmap v107: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 632 KiB/s wr, 117 op/s 2024-07-29T00:39:59.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:39:58 smithi204 ceph-mon[37368]: pgmap v107: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 632 KiB/s wr, 117 op/s 2024-07-29T00:39:59.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 96276 4.88 MB/sec warmup 110 sec latency 22.808 ms 2024-07-29T00:40:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:00 smithi204 ceph-mon[37368]: overall HEALTH_OK 2024-07-29T00:40:00.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:00 smithi181 ceph-mon[36706]: overall HEALTH_OK 2024-07-29T00:40:00.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:00 smithi175 ceph-mon[32138]: overall HEALTH_OK 2024-07-29T00:40:00.877 INFO:tasks.workunit.client.0.smithi175.stdout: 1 97592 4.89 MB/sec warmup 111 sec latency 19.264 ms 2024-07-29T00:40:01.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:01 smithi181 ceph-mon[36706]: pgmap v108: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 632 KiB/s wr, 117 op/s 2024-07-29T00:40:01.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:01 smithi175 ceph-mon[32138]: pgmap v108: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 632 KiB/s wr, 117 op/s 2024-07-29T00:40:01.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:01 smithi204 ceph-mon[37368]: pgmap v108: 97 pgs: 97 active+clean; 62 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 632 KiB/s wr, 117 op/s 2024-07-29T00:40:01.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 98366 4.87 MB/sec warmup 112 sec latency 130.960 ms 2024-07-29T00:40:02.148 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-29T00:40:02.148 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.8 2024-07-29T00:40:02.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:40:02.645 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:02.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 99702 4.92 MB/sec warmup 113 sec latency 15.043 ms 2024-07-29T00:40:03.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:03 smithi181 ceph-mon[36706]: pgmap v109: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-07-29T00:40:03.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:03 smithi175 ceph-mon[32138]: pgmap v109: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-07-29T00:40:03.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:03 smithi204 ceph-mon[37368]: pgmap v109: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-07-29T00:40:03.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 99828 4.88 MB/sec warmup 114 sec latency 169.554 ms 2024-07-29T00:40:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 -- 172.21.15.175:0/3684517187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24104240 msgr2=0x7f7d24110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 --2- 172.21.15.175:0/3684517187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24104240 0x7f7d24110770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009c10 tx=0x7f7d1c02a6e0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 -- 172.21.15.175:0/3684517187 shutdown_connections 2024-07-29T00:40:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 --2- 172.21.15.175:0/3684517187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24104240 0x7f7d24110770 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 --2- 172.21.15.175:0/3684517187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d24103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 --2- 172.21.15.175:0/3684517187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d241098a0 0x7f7d24109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 -- 172.21.15.175:0/3684517187 >> 172.21.15.175:0/3684517187 conn(0x7f7d240ff560 msgr2=0x7f7d24101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 -- 172.21.15.175:0/3684517187 shutdown_connections 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.511+0000 7f7d2a200640 1 -- 172.21.15.175:0/3684517187 wait complete. 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.512+0000 7f7d2a200640 1 Processor -- start 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 -- start start 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d2419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 0x7f7d2419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d241098a0 0x7f7d2419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24113ca0 con 0x7f7d241098a0 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24113b20 con 0x7f7d24104240 2024-07-29T00:40:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d2a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24113e20 con 0x7f7d241038a0 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d28e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d2419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 0x7f7d2419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d28e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d2419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33046/0 (socket says 172.21.15.175:33046) 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 0x7f7d2419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:45608/0 (socket says 172.21.15.175:45608) 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d28e00640 1 -- 172.21.15.175:0/416292298 learned_addr learned my addr 172.21.15.175:0/416292298 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.513+0000 7f7d29800640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d241098a0 0x7f7d2419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d28e00640 1 -- 172.21.15.175:0/416292298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 msgr2=0x7f7d2419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d28e00640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 0x7f7d2419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d28e00640 1 -- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d241098a0 msgr2=0x7f7d2419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d28e00640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d241098a0 0x7f7d2419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d28e00640 1 -- 172.21.15.175:0/416292298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419f130 con 0x7f7d241038a0 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d23e00640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 0x7f7d2419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:04.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d28e00640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d2419a180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e9c0 tx=0x7f7d1800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800ccc0 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.514+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419f420 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.515+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d18005500 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.515+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18010430 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.515+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241a6d00 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.516+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d24104fe0 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.518+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7d180046c0 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.519+0000 7f7d21600640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d0c0a2d70 0x7f7d0c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.519+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f7d180cf690 con 0x7f7d241038a0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.520+0000 7f7d23e00640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d0c0a2d70 0x7f7d0c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:04.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.520+0000 7f7d23e00640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d0c0a2d70 0x7f7d0c0a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d24101b90 tx=0x7f7d10005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:04.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.523+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7d1809c230 con 0x7f7d241038a0 2024-07-29T00:40:04.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.778+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d24103d00 con 0x7f7d241038a0 2024-07-29T00:40:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.779+0000 7f7d21600640 1 -- 172.21.15.175:0/416292298 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7f7d1809bb30 con 0x7f7d241038a0 2024-07-29T00:40:04.780 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:40:04.780 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:40:04.780 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:40:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d0c0a2d70 msgr2=0x7f7d0c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d0c0a2d70 0x7f7d0c0a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d24101b90 tx=0x7f7d10005f90 comp rx=0 tx=0).stop 2024-07-29T00:40:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 msgr2=0x7f7d2419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d2419a180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e9c0 tx=0x7f7d1800ee80 comp rx=0 tx=0).stop 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 shutdown_connections 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d0c0a2d70 0x7f7d0c0a5230 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d241098a0 0x7f7d2419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24104240 0x7f7d2419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 --2- 172.21.15.175:0/416292298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d241038a0 0x7f7d2419a180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 >> 172.21.15.175:0/416292298 conn(0x7f7d240ff560 msgr2=0x7f7d240ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 shutdown_connections 2024-07-29T00:40:04.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:04.783+0000 7f7d2a200640 1 -- 172.21.15.175:0/416292298 wait complete. 2024-07-29T00:40:04.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 100618 4.86 MB/sec warmup 115 sec latency 358.874 ms 2024-07-29T00:40:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:05 smithi181 ceph-mon[36706]: pgmap v110: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 710 KiB/s wr, 128 op/s 2024-07-29T00:40:05.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:05 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/416292298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:05.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:05 smithi175 ceph-mon[32138]: pgmap v110: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 710 KiB/s wr, 128 op/s 2024-07-29T00:40:05.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:05 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/416292298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:05.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:05 smithi204 ceph-mon[37368]: pgmap v110: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 710 KiB/s wr, 128 op/s 2024-07-29T00:40:05.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:05 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/416292298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:05.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 90 --expiration 139.8 --await-for 45.0' 2024-07-29T00:40:05.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db / --timeout 90 --expiration 139.8 --await-for 45.0 2024-07-29T00:40:05.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 102722 4.92 MB/sec warmup 116 sec latency 13.684 ms 2024-07-29T00:40:06.248 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:06.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103382 4.92 MB/sec warmup 117 sec latency 17.227 ms 2024-07-29T00:40:07.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.082+0000 7fe2a0600640 1 -- 172.21.15.175:0/823414143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 msgr2=0x7fe29810e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.082+0000 7fe2a0600640 1 --2- 172.21.15.175:0/823414143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29810e730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe290009cd0 tx=0x7fe29002a8e0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 -- 172.21.15.175:0/823414143 shutdown_connections 2024-07-29T00:40:07.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 --2- 172.21.15.175:0/823414143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29810e730 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 --2- 172.21.15.175:0/823414143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe298101860 0x7fe298101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 --2- 172.21.15.175:0/823414143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe298107860 0x7fe298107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 -- 172.21.15.175:0/823414143 >> 172.21.15.175:0/823414143 conn(0x7fe2980fd5a0 msgr2=0x7fe2980ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:07.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 -- 172.21.15.175:0/823414143 shutdown_connections 2024-07-29T00:40:07.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.083+0000 7fe2a0600640 1 -- 172.21.15.175:0/823414143 wait complete. 2024-07-29T00:40:07.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.084+0000 7fe2a0600640 1 Processor -- start 2024-07-29T00:40:07.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.084+0000 7fe2a0600640 1 -- start start 2024-07-29T00:40:07.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.084+0000 7fe2a0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe298101860 0x7fe29819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.084+0000 7fe2a0600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.084+0000 7fe2a0600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe298107860 0x7fe29819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe2a0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe298111c60 con 0x7fe298101860 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe2a0600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe298111ae0 con 0x7fe298107860 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe2a0600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe298111de0 con 0x7fe298102200 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe298107860 0x7fe29819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29819a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33062/0 (socket says 172.21.15.175:33062) 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 -- 172.21.15.175:0/2192611441 learned_addr learned my addr 172.21.15.175:0/2192611441 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:07.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29dc00640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe298101860 0x7fe29819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 -- 172.21.15.175:0/2192611441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe298107860 msgr2=0x7fe29819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe298107860 0x7fe29819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 -- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe298101860 msgr2=0x7fe29819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe298101860 0x7fe29819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.085+0000 7fe29d200640 1 -- 172.21.15.175:0/2192611441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29819f2c0 con 0x7fe298102200 2024-07-29T00:40:07.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.086+0000 7fe29d200640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29819a850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e970 tx=0x7fe28400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:07.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.086+0000 7fe296000640 1 -- 172.21.15.175:0/2192611441 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28400ccc0 con 0x7fe298102200 2024-07-29T00:40:07.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.086+0000 7fe296000640 1 -- 172.21.15.175:0/2192611441 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe284010910 con 0x7fe298102200 2024-07-29T00:40:07.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.086+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29819f5b0 con 0x7fe298102200 2024-07-29T00:40:07.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.086+0000 7fe296000640 1 -- 172.21.15.175:0/2192611441 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284018860 con 0x7fe298102200 2024-07-29T00:40:07.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.086+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2981a6e90 con 0x7fe298102200 2024-07-29T00:40:07.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.090+0000 7fe2a2d22740 1 Processor -- start 2024-07-29T00:40:07.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.090+0000 7fe296000640 1 -- 172.21.15.175:0/2192611441 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe284010ac0 con 0x7fe298102200 2024-07-29T00:40:07.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.090+0000 7fe2a2d22740 1 -- start start 2024-07-29T00:40:07.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585fc430 0x55f7585fe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f758556a80 con 0x55f7585f1250 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f7584ee080 con 0x55f7585fc430 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f758552a40 con 0x55f7585f1b70 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe29dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe29e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe29dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585fbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33070/0 (socket says 172.21.15.175:33070) 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe29d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585fc430 0x55f7585fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.091+0000 7fe29dc00640 1 -- 172.21.15.175:0/2067820816 learned_addr learned my addr 172.21.15.175:0/2067820816 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:07.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29e600640 1 -- 172.21.15.175:0/2067820816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 msgr2=0x55f7585fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29e600640 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29e600640 1 -- 172.21.15.175:0/2067820816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585fc430 msgr2=0x55f7585fe820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29e600640 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585fc430 0x55f7585fe820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29e600640 1 -- 172.21.15.175:0/2067820816 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7582c8b90 con 0x55f7585f1250 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29dc00640 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.092+0000 7fe29e600640 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f1630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe290009ca0 tx=0x7fe2900314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc9e0db366d6b3c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.093+0000 7fe28aa00640 1 -- 172.21.15.175:0/2067820816 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290049070 con 0x55f7585f1250 2024-07-29T00:40:07.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.093+0000 7fe28aa00640 1 -- 172.21.15.175:0/2067820816 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe290036040 con 0x55f7585f1250 2024-07-29T00:40:07.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.093+0000 7fe28aa00640 1 -- 172.21.15.175:0/2067820816 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2900042a0 con 0x55f7585f1250 2024-07-29T00:40:07.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.093+0000 7fe296000640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe2800a2dc0 0x7fe2800a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.093+0000 7fe296000640 1 -- 172.21.15.175:0/2192611441 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe2840cf7a0 con 0x7fe298102200 2024-07-29T00:40:07.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.093+0000 7fe29dc00640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe2800a2dc0 0x7fe2800a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 -- 172.21.15.175:0/2067820816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 msgr2=0x55f7585f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f1630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe290009ca0 tx=0x7fe2900314c0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 -- 172.21.15.175:0/2067820816 shutdown_connections 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585fc430 0x55f7585fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/2067820816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f1630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 -- 172.21.15.175:0/2067820816 >> 172.21.15.175:0/2067820816 conn(0x55f7585ed800 msgr2=0x55f7585edbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 -- 172.21.15.175:0/2067820816 shutdown_connections 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe2a2d22740 1 -- 172.21.15.175:0/2067820816 wait complete. 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.094+0000 7fe29dc00640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe2800a2dc0 0x7fe2800a5280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55f7585ed300 tx=0x7fe28c005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.095+0000 7fe2a2d22740 1 -- start start 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.095+0000 7fe2a2d22740 1 Processor -- start 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 0x55f7585f9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585fc430 0x55f7585f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7585fefa0 con 0x55f7585f1250 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f758511750 con 0x55f7585f1b70 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f7585ff120 con 0x55f7585fc430 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe2a2d22740 10 client.0 _ll_register_callbacks cb 0x55f758266090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 0x55f7585f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 0x55f7585f9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:45642/0 (socket says 172.21.15.175:45642) 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 -- 172.21.15.175:0/275440289 learned_addr learned my addr 172.21.15.175:0/275440289 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29e600640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585fc430 0x55f7585f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29dc00640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 -- 172.21.15.175:0/275440289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585fc430 msgr2=0x55f7585f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585fc430 0x55f7585f9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 -- 172.21.15.175:0/275440289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 msgr2=0x55f7585f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.096+0000 7fe29d200640 1 -- 172.21.15.175:0/275440289 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f758608490 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe29d200640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 0x55f7585f9960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2840cced0 tx=0x7fe28400ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a05b15d0f9acd7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe27f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe27f400640 1 -- 172.21.15.175:0/275440289 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840cd300 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe290036040 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe27f400640 1 -- 172.21.15.175:0/275440289 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe2840cfd90 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe27f400640 1 -- 172.21.15.175:0/275440289 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe28409fff0 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe2a2d22740 10 client.26111 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f75860ae00 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.097+0000 7fe27f400640 1 -- 172.21.15.175:0/275440289 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe290036040 con 0x55f7585f1b70 2024-07-29T00:40:07.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe27f400640 1 -- 172.21.15.175:0/275440289 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe284097c40 con 0x55f7585f1b70 2024-07-29T00:40:07.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe2a2d22740 10 client.26111 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:07.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe2a2d22740 10 client.26111 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:07.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe2a2d22740 20 client.26111 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f75860c280 0x55f75860e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe2a2d22740 4 client.26111 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7585641b0 con 0x55f75860c280 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.098+0000 7fe29dc00640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f75860c280 0x55f75860e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.099+0000 7fe29dc00640 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f75860c280 0x55f75860e6a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92fb12ff774e9eaa server_cookie=fbfe0859d5b84ca2 in_seq=0 out_seq=0 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.099+0000 7fe27f400640 10 client.26111 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.099+0000 7fe27f400640 1 -- 172.21.15.175:0/275440289 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f7585641b0 con 0x55f75860c280 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.099+0000 7fe27f400640 10 client.26111 handle_command_reply: tid=0 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.099+0000 7fe2a2d22740 1 client.26111 shutdown 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 msgr2=0x55f7585f9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 0x55f7585f9960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2840cced0 tx=0x7fe28400ce60 comp rx=0 tx=0).stop 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 shutdown_connections 2024-07-29T00:40:07.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f75860c280 0x55f75860e6a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585fc430 0x55f7585f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1b70 0x55f7585f9960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/275440289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585f1250 0x55f7585f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 >> 172.21.15.175:0/275440289 conn(0x55f7585ed800 msgr2=0x55f7585f34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe27f400640 0 client.26111 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 shutdown_connections 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 1 -- 172.21.15.175:0/275440289 wait complete. 2024-07-29T00:40:07.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.100+0000 7fe2a2d22740 20 client.26111 trim_cache size 0 max 16384 2024-07-29T00:40:07.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.124+0000 7fe2a2d22740 1 Processor -- start 2024-07-29T00:40:07.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.124+0000 7fe2a2d22740 1 -- start start 2024-07-29T00:40:07.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f7585f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758614000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981af7f0 con 0x55f7585fc430 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981b13f0 con 0x55f7585f1250 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981afec0 con 0x55f7585f1b70 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f7585f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f7585f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:45646/0 (socket says 172.21.15.175:45646) 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 -- 172.21.15.175:0/3349500573 learned_addr learned my addr 172.21.15.175:0/3349500573 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29e600640 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758614000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 -- 172.21.15.175:0/3349500573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 msgr2=0x55f7585f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 -- 172.21.15.175:0/3349500573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 msgr2=0x55f758614000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758614000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29dc00640 1 -- 172.21.15.175:0/3349500573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7585f19a0 con 0x55f7585f1250 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29e600640 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758614000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.125+0000 7fe29d200640 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:07.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.126+0000 7fe29dc00640 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f7585f8360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe28c004110 tx=0x7fe28c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8898337a5e58e891 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.126+0000 7fe28a000640 1 -- 172.21.15.175:0/3349500573 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c003730 con 0x55f7585f1250 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.126+0000 7fe28a000640 1 -- 172.21.15.175:0/3349500573 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28c003db0 con 0x55f7585f1250 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.126+0000 7fe28a000640 1 -- 172.21.15.175:0/3349500573 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c01f430 con 0x55f7585f1250 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 -- 172.21.15.175:0/3349500573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 msgr2=0x55f7585f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f7585f8360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe28c004110 tx=0x7fe28c017000 comp rx=0 tx=0).stop 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 -- 172.21.15.175:0/3349500573 shutdown_connections 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758614000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/3349500573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f7585f8360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 -- 172.21.15.175:0/3349500573 >> 172.21.15.175:0/3349500573 conn(0x55f7585ed800 msgr2=0x55f7585ec980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 -- 172.21.15.175:0/3349500573 shutdown_connections 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.127+0000 7fe2a2d22740 1 -- 172.21.15.175:0/3349500573 wait complete. 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 -- start start 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 Processor -- start 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f758604920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758600190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981b0c60 con 0x55f7585fc430 2024-07-29T00:40:07.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981b0590 con 0x55f7585f1250 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981ad7d0 con 0x55f7585f1b70 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.128+0000 7fe2a2d22740 10 client.0 _ll_register_callbacks cb 0x7fe2840d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f758604920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f758604920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:45650/0 (socket says 172.21.15.175:45650) 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 -- 172.21.15.175:0/735416243 learned_addr learned my addr 172.21.15.175:0/735416243 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29d200640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29e600640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758600190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 -- 172.21.15.175:0/735416243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 msgr2=0x55f7585ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 -- 172.21.15.175:0/735416243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 msgr2=0x55f758600190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758600190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 -- 172.21.15.175:0/735416243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f758600800 con 0x55f7585f1250 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29d200640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe29dc00640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f758604920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe28c019040 tx=0x7fe28c003ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9e224d9b4444fe server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.129+0000 7fe27f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.130+0000 7fe27f400640 1 -- 172.21.15.175:0/735416243 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c037020 con 0x55f7585f1250 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.130+0000 7fe27f400640 1 -- 172.21.15.175:0/735416243 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fe28c00f480 con 0x55f7585f1250 2024-07-29T00:40:07.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28c003db0 con 0x55f7585f1250 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe27f400640 1 -- 172.21.15.175:0/735416243 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe28c00f680 con 0x55f7585f1250 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 10 client.26117 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f758676e20 con 0x55f7585f1250 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe27f400640 1 -- 172.21.15.175:0/735416243 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28c003db0 con 0x55f7585f1250 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe27f400640 1 -- 172.21.15.175:0/735416243 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe28c0046c0 con 0x55f7585f1250 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 10 client.26117 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 10 client.26117 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 20 client.26117 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f7586775a0 0x55f7586799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 4 client.26117 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.8, "await_for": 45.0}] 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.131+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.8, "await_for": 45.0}) -- 0x55f75855e500 con 0x55f7586775a0 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.132+0000 7fe29d200640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f7586775a0 0x55f7586799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:07.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.132+0000 7fe29d200640 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f7586775a0 0x55f7586799c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69747fa398ff442d server_cookie=92a1e2b039ce3a79 in_seq=0 out_seq=0 2024-07-29T00:40:07.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.132+0000 7fe27f400640 10 client.26117 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:07.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.198+0000 7fe27f400640 1 -- 172.21.15.175:0/735416243 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f75855e500 con 0x55f7586775a0 2024-07-29T00:40:07.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.198+0000 7fe27f400640 10 client.26117 handle_command_reply: tid=0 2024-07-29T00:40:07.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 client.26117 shutdown 2024-07-29T00:40:07.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 msgr2=0x55f758604920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f758604920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe28c019040 tx=0x7fe28c003ff0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 shutdown_connections 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f7586775a0 0x55f7586799c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7585fc430 0x55f758600190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7585f1b70 0x55f7585ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 --2- 172.21.15.175:0/735416243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7585f1250 0x55f758604920 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe27f400640 0 client.26117 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 >> 172.21.15.175:0/735416243 conn(0x55f7585ed800 msgr2=0x55f7585ec160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 shutdown_connections 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.199+0000 7fe2a2d22740 1 -- 172.21.15.175:0/735416243 wait complete. 2024-07-29T00:40:07.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.200+0000 7fe2a2d22740 20 client.26117 trim_cache size 0 max 16384 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 31, 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 10, 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "3b49758e": { 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 10, 2024-07-29T00:40:07.201 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 139.8, 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:07.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:07.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:07.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:07.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:07.203 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:07.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe2800a2dc0 msgr2=0x7fe2800a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe2800a2dc0 0x7fe2800a5280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55f7585ed300 tx=0x7fe28c005f90 comp rx=0 tx=0).stop 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 msgr2=0x7fe29819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29819a850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e970 tx=0x7fe28400ee30 comp rx=0 tx=0).stop 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 shutdown_connections 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe298107860 0x7fe29819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe298102200 0x7fe29819a850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe2800a2dc0 0x7fe2800a5280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 --2- 172.21.15.175:0/2192611441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe298101860 0x7fe29819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 >> 172.21.15.175:0/2192611441 conn(0x7fe2980fd5a0 msgr2=0x7fe29810cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 shutdown_connections 2024-07-29T00:40:07.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:07.201+0000 7fe2a0600640 1 -- 172.21.15.175:0/2192611441 wait complete. 2024-07-29T00:40:07.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:07 smithi175 ceph-mon[32138]: pgmap v111: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-29T00:40:07.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:07 smithi181 ceph-mon[36706]: pgmap v111: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-29T00:40:07.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:07 smithi204 ceph-mon[37368]: pgmap v111: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-29T00:40:07.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 4.88 MB/sec warmup 118 sec latency 734.541 ms 2024-07-29T00:40:08.424 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3b49758e', quiesce took 6.3 seconds. Will release after: 13.5 2024-07-29T00:40:08.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 4.84 MB/sec warmup 119 sec latency 1734.590 ms 2024-07-29T00:40:09.465 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:09 smithi175 ceph-mon[32138]: pgmap v112: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-07-29T00:40:09.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:09 smithi181 ceph-mon[36706]: pgmap v112: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-07-29T00:40:09.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:09 smithi204 ceph-mon[37368]: pgmap v112: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-07-29T00:40:10.878 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 1 sec latency 3734.715 ms 2024-07-29T00:40:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:11 smithi175 ceph-mon[32138]: pgmap v113: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-07-29T00:40:11.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:11 smithi181 ceph-mon[36706]: pgmap v113: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-07-29T00:40:11.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:11 smithi204 ceph-mon[37368]: pgmap v113: 97 pgs: 97 active+clean; 68 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-07-29T00:40:11.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 2 sec latency 4735.278 ms 2024-07-29T00:40:12.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 3 sec latency 5735.465 ms 2024-07-29T00:40:13.394 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:13 smithi175 ceph-mon[32138]: pgmap v114: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-07-29T00:40:13.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:13 smithi181 ceph-mon[36706]: pgmap v114: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-07-29T00:40:13.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:13 smithi204 ceph-mon[37368]: pgmap v114: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-07-29T00:40:13.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 4 sec latency 6735.537 ms 2024-07-29T00:40:14.609 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:40:14.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 5 sec latency 7735.600 ms 2024-07-29T00:40:15.068 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:15.413 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:15 smithi175 ceph-mon[32138]: pgmap v115: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-07-29T00:40:15.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:15 smithi181 ceph-mon[36706]: pgmap v115: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-07-29T00:40:15.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:15 smithi204 ceph-mon[37368]: pgmap v115: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-07-29T00:40:15.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 -- 172.21.15.175:0/3891440749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 msgr2=0x7f75a410da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 --2- 172.21.15.175:0/3891440749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a410da80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7590009900 tx=0x7f759002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 -- 172.21.15.175:0/3891440749 shutdown_connections 2024-07-29T00:40:15.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 --2- 172.21.15.175:0/3891440749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a410dfc0 0x7f75a41103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 --2- 172.21.15.175:0/3891440749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a410da80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 --2- 172.21.15.175:0/3891440749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a4100410 0x7f75a41007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 -- 172.21.15.175:0/3891440749 >> 172.21.15.175:0/3891440749 conn(0x7f75a40fc300 msgr2=0x7f75a40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:15.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 -- 172.21.15.175:0/3891440749 shutdown_connections 2024-07-29T00:40:15.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.728+0000 7f75a9000640 1 -- 172.21.15.175:0/3891440749 wait complete. 2024-07-29T00:40:15.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.729+0000 7f75a9000640 1 Processor -- start 2024-07-29T00:40:15.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.729+0000 7f75a9000640 1 -- start start 2024-07-29T00:40:15.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.729+0000 7f75a9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a4100410 0x7f75a419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.729+0000 7f75a9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.729+0000 7f75a9000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a410dfc0 0x7f75a419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a9000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4113d10 con 0x7f75a4100dc0 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a9000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4113b90 con 0x7f75a4100410 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a9000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4113e90 con 0x7f75a410dfc0 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:32962/0 (socket says 172.21.15.175:32962) 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 -- 172.21.15.175:0/3472366376 learned_addr learned my addr 172.21.15.175:0/3472366376 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a3400640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a4100410 0x7f75a419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a3e00640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a410dfc0 0x7f75a419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 -- 172.21.15.175:0/3472366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a410dfc0 msgr2=0x7f75a419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a410dfc0 0x7f75a419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 -- 172.21.15.175:0/3472366376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a4100410 msgr2=0x7f75a419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a4100410 0x7f75a419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.730+0000 7f75a2a00640 1 -- 172.21.15.175:0/3472366376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a419f1e0 con 0x7f75a4100dc0 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.731+0000 7f75a2a00640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a419a770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7590004770 tx=0x7f7590003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.731+0000 7f759be00640 1 -- 172.21.15.175:0/3472366376 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759003f070 con 0x7f75a4100dc0 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.731+0000 7f759be00640 1 -- 172.21.15.175:0/3472366376 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7590031ca0 con 0x7f75a4100dc0 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.731+0000 7f759be00640 1 -- 172.21.15.175:0/3472366376 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590035030 con 0x7f75a4100dc0 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.731+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a419f470 con 0x7f75a4100dc0 2024-07-29T00:40:15.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.731+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a419f900 con 0x7f75a4100dc0 2024-07-29T00:40:15.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 Processor -- start 2024-07-29T00:40:15.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f759be00640 1 -- 172.21.15.175:0/3472366376 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7590042070 con 0x7f75a4100dc0 2024-07-29T00:40:15.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 -- start start 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864241ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588641a6ad0 con 0x558864242430 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55886413eaf0 con 0x558864241b10 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.735+0000 7f75ad226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5588641a2a90 con 0x55886424ccf0 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864241ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864241ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51672/0 (socket says 172.21.15.175:51672) 2024-07-29T00:40:15.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 -- 172.21.15.175:0/634355050 learned_addr learned my addr 172.21.15.175:0/634355050 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:15.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a2a00640 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 -- 172.21.15.175:0/634355050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 msgr2=0x55886424f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 -- 172.21.15.175:0/634355050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 msgr2=0x55886424c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 -- 172.21.15.175:0/634355050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558863f183c0 con 0x558864241b10 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f759be00640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75880a2cf0 0x7f75880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.736+0000 7f75a3e00640 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864241ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f759c009f20 tx=0x7f759c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19081f3d6455db9b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.737+0000 7f759be00640 1 -- 172.21.15.175:0/3472366376 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f75900ef8a0 con 0x7f75a4100dc0 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.737+0000 7f75a3400640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75880a2cf0 0x7f75880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.737+0000 7f7598c00640 1 -- 172.21.15.175:0/634355050 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c003c30 con 0x558864241b10 2024-07-29T00:40:15.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.737+0000 7f7598c00640 1 -- 172.21.15.175:0/634355050 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c015070 con 0x558864241b10 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.737+0000 7f7598c00640 1 -- 172.21.15.175:0/634355050 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c011920 con 0x558864241b10 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 -- 172.21.15.175:0/634355050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 msgr2=0x558864241ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864241ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f759c009f20 tx=0x7f759c010860 comp rx=0 tx=0).stop 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75a3400640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75880a2cf0 0x7f75880a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55886423ef00 tx=0x7f759400b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 -- 172.21.15.175:0/634355050 shutdown_connections 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 --2- 172.21.15.175:0/634355050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864241ef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 -- 172.21.15.175:0/634355050 >> 172.21.15.175:0/634355050 conn(0x55886423e0a0 msgr2=0x55886423e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 -- 172.21.15.175:0/634355050 shutdown_connections 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.738+0000 7f75ad226740 1 -- 172.21.15.175:0/634355050 wait complete. 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 -- start start 2024-07-29T00:40:15.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 Processor -- start 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864249be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55886424f8c0 con 0x558864242430 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558864161bc0 con 0x558864241b10 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55886424fa40 con 0x55886424ccf0 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.739+0000 7f75ad226740 10 client.0 _ll_register_callbacks cb 0x55886414dd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:15.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864249be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864249be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51684/0 (socket says 172.21.15.175:51684) 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:32982/0 (socket says 172.21.15.175:32982) 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 -- 172.21.15.175:0/990900513 learned_addr learned my addr 172.21.15.175:0/990900513 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3e00640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 -- 172.21.15.175:0/990900513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 msgr2=0x55886424a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 -- 172.21.15.175:0/990900513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 msgr2=0x55886424a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3400640 1 -- 172.21.15.175:0/990900513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558864258db0 con 0x558864241b10 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a2a00640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.740+0000 7f75a3e00640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:15.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f75a3400640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864249be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f759401f4b0 tx=0x7f7594004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=257853b5679e7b0d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f758d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f758d600640 1 -- 172.21.15.175:0/990900513 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759402c060 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759c015070 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f758d600640 1 -- 172.21.15.175:0/990900513 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f7594017030 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f758d600640 1 -- 172.21.15.175:0/990900513 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f759401c600 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f75ad226740 10 client.26129 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.741+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55886425b7f0 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f758d600640 1 -- 172.21.15.175:0/990900513 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c015070 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f758d600640 1 -- 172.21.15.175:0/990900513 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f7594010a10 con 0x558864241b10 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f75ad226740 10 client.26129 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f75ad226740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f75ad226740 20 client.26129 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f75ad226740 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55886425cca0 0x55886425f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f75ad226740 4 client.26129 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:15.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.742+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588641b4040 con 0x55886425cca0 2024-07-29T00:40:15.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.743+0000 7f75a2a00640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55886425cca0 0x55886425f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.743+0000 7f75a2a00640 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55886425cca0 0x55886425f090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d80105a2f6752c server_cookie=4d01ee0dbf610dfd in_seq=0 out_seq=0 2024-07-29T00:40:15.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.743+0000 7f758d600640 10 client.26129 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f758d600640 1 -- 172.21.15.175:0/990900513 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588641b4040 con 0x55886425cca0 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f758d600640 10 client.26129 handle_command_reply: tid=0 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 client.26129 shutdown 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 msgr2=0x558864249be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864249be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f759401f4b0 tx=0x7f7594004730 comp rx=0 tx=0).stop 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 shutdown_connections 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55886425cca0 0x55886425f090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55886424ccf0 0x55886424a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x55886424a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f758d600640 0 client.26129 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:15.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 --2- 172.21.15.175:0/990900513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558864241b10 0x558864249be0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 >> 172.21.15.175:0/990900513 conn(0x55886423e0a0 msgr2=0x558864243dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:15.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 shutdown_connections 2024-07-29T00:40:15.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 1 -- 172.21.15.175:0/990900513 wait complete. 2024-07-29T00:40:15.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.744+0000 7f75ad226740 20 client.26129 trim_cache size 0 max 16384 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 Processor -- start 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 -- start start 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 0x5588642445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864242430 0x55886424a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41af740 con 0x558864241b10 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b1340 con 0x55886424ccf0 2024-07-29T00:40:15.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.763+0000 7f75ad226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41afe10 con 0x558864242430 2024-07-29T00:40:15.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 0x5588642445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864242430 0x55886424a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 0x5588642445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:32992/0 (socket says 172.21.15.175:32992) 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3400640 1 -- 172.21.15.175:0/2264540573 learned_addr learned my addr 172.21.15.175:0/2264540573 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3e00640 1 -- 172.21.15.175:0/2264540573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864242430 msgr2=0x55886424a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3e00640 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864242430 0x55886424a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3e00640 1 -- 172.21.15.175:0/2264540573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 msgr2=0x5588642445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3e00640 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 0x5588642445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3e00640 1 -- 172.21.15.175:0/2264540573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886423ded0 con 0x55886424ccf0 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.764+0000 7f75a3400640 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 0x5588642445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.765+0000 7f75a3e00640 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642649a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f759c010d60 tx=0x7f759c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2526fdb9b2650 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.765+0000 7f758fe00640 1 -- 172.21.15.175:0/2264540573 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c003cb0 con 0x55886424ccf0 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.765+0000 7f758fe00640 1 -- 172.21.15.175:0/2264540573 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c015040 con 0x55886424ccf0 2024-07-29T00:40:15.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.765+0000 7f758fe00640 1 -- 172.21.15.175:0/2264540573 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c0053b0 con 0x55886424ccf0 2024-07-29T00:40:15.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 -- 172.21.15.175:0/2264540573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 msgr2=0x5588642649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642649a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f759c010d60 tx=0x7f759c013040 comp rx=0 tx=0).stop 2024-07-29T00:40:15.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 -- 172.21.15.175:0/2264540573 shutdown_connections 2024-07-29T00:40:15.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642649a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864242430 0x55886424a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 --2- 172.21.15.175:0/2264540573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864241b10 0x5588642445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 -- 172.21.15.175:0/2264540573 >> 172.21.15.175:0/2264540573 conn(0x55886423e0a0 msgr2=0x55886424bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 -- 172.21.15.175:0/2264540573 shutdown_connections 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.766+0000 7f75ad226740 1 -- 172.21.15.175:0/2264540573 wait complete. 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 -- start start 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 Processor -- start 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864241b10 0x558864246c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x558864247160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b0bb0 con 0x558864242430 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b04e0 con 0x55886424ccf0 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41ad720 con 0x558864241b10 2024-07-29T00:40:15.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.767+0000 7f75ad226740 10 client.0 _ll_register_callbacks cb 0x7f759400be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x558864247160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864241b10 0x558864246c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51696/0 (socket says 172.21.15.175:51696) 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 -- 172.21.15.175:0/2189244155 learned_addr learned my addr 172.21.15.175:0/2189244155 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 -- 172.21.15.175:0/2189244155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864241b10 msgr2=0x558864246c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864241b10 0x558864246c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 -- 172.21.15.175:0/2189244155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 msgr2=0x558864247160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x558864247160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 -- 172.21.15.175:0/2189244155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886423ed30 con 0x55886424ccf0 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f75a3e00640 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642476a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f759c003d40 tx=0x7f759c003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2eb298b0e3ee7e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.768+0000 7f758d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f758d600640 1 -- 172.21.15.175:0/2189244155 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c02e070 con 0x55886424ccf0 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759c015040 con 0x55886424ccf0 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f758d600640 1 -- 172.21.15.175:0/2189244155 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f759c005540 con 0x55886424ccf0 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f758d600640 1 -- 172.21.15.175:0/2189244155 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f759c03a050 con 0x55886424ccf0 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f75ad226740 10 client.26135 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55886423f150 con 0x55886424ccf0 2024-07-29T00:40:15.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f758d600640 1 -- 172.21.15.175:0/2189244155 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c015040 con 0x55886424ccf0 2024-07-29T00:40:15.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.769+0000 7f758d600640 1 -- 172.21.15.175:0/2189244155 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f759c011cb0 con 0x55886424ccf0 2024-07-29T00:40:15.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75ad226740 10 client.26135 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:15.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75ad226740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:15.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75ad226740 20 client.26135 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:15.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75ad226740 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588642c7d40 0x5588642ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75ad226740 4 client.26135 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588641ae550 con 0x5588642c7d40 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.770+0000 7f75a3400640 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588642c7d40 0x5588642ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75a3400640 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588642c7d40 0x5588642ca130 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728717543bff3d0 server_cookie=c9d9a868ba1b873e in_seq=0 out_seq=0 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f758d600640 10 client.26135 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f758d600640 1 -- 172.21.15.175:0/2189244155 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5588641ae550 con 0x5588642c7d40 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f758d600640 10 client.26135 handle_command_reply: tid=0 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 client.26135 shutdown 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 msgr2=0x5588642476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642476a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f759c003d40 tx=0x7f759c003d70 comp rx=0 tx=0).stop 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 shutdown_connections 2024-07-29T00:40:15.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5588642c7d40 0x5588642ca130 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886424ccf0 0x5588642476a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558864242430 0x558864247160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 --2- 172.21.15.175:0/2189244155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558864241b10 0x558864246c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 >> 172.21.15.175:0/2189244155 conn(0x55886423e0a0 msgr2=0x55886423c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f758d600640 0 client.26135 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 shutdown_connections 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 1 -- 172.21.15.175:0/2189244155 wait complete. 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.771+0000 7f75ad226740 20 client.26135 trim_cache size 0 max 16384 2024-07-29T00:40:15.775 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:15.776 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:40:15.776 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:40:15.776 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:15.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75880a2cf0 msgr2=0x7f75880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75880a2cf0 0x7f75880a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55886423ef00 tx=0x7f759400b410 comp rx=0 tx=0).stop 2024-07-29T00:40:15.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 msgr2=0x7f75a419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a419a770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7590004770 tx=0x7f7590003c10 comp rx=0 tx=0).stop 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 shutdown_connections 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75a410dfc0 0x7f75a419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75880a2cf0 0x7f75880a51b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4100dc0 0x7f75a419a770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 --2- 172.21.15.175:0/3472366376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a4100410 0x7f75a419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 >> 172.21.15.175:0/3472366376 conn(0x7f75a40fc300 msgr2=0x7f75a40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 shutdown_connections 2024-07-29T00:40:15.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:15.772+0000 7f75a9000640 1 -- 172.21.15.175:0/3472366376 wait complete. 2024-07-29T00:40:15.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 6 sec latency 8735.771 ms 2024-07-29T00:40:16.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 7 sec latency 9735.858 ms 2024-07-29T00:40:16.898 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:40:16.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:40:17.333 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:17.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:17 smithi204 ceph-mon[37368]: pgmap v116: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-07-29T00:40:17.477 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:17 smithi175 ceph-mon[32138]: pgmap v116: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-07-29T00:40:17.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:17 smithi181 ceph-mon[36706]: pgmap v116: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-07-29T00:40:17.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 8 sec latency 10735.940 ms 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 -- 172.21.15.175:0/2760413322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc104420 msgr2=0x7efcfc110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 --2- 172.21.15.175:0/2760413322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc104420 0x7efcfc110950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009cd0 tx=0x7efcf402a8e0 comp rx=0 tx=0).stop 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 -- 172.21.15.175:0/2760413322 shutdown_connections 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 --2- 172.21.15.175:0/2760413322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc104420 0x7efcfc110950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 --2- 172.21.15.175:0/2760413322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 0x7efcfc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 --2- 172.21.15.175:0/2760413322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc109a80 0x7efcfc109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 -- 172.21.15.175:0/2760413322 >> 172.21.15.175:0/2760413322 conn(0x7efcfc0ff760 msgr2=0x7efcfc101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 -- 172.21.15.175:0/2760413322 shutdown_connections 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.951+0000 7efd03000640 1 -- 172.21.15.175:0/2760413322 wait complete. 2024-07-29T00:40:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.952+0000 7efd03000640 1 Processor -- start 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.952+0000 7efd03000640 1 -- start start 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd03000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 0x7efcfc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd03000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 0x7efcfc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd03000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc109a80 0x7efcfc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd03000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113ed0 con 0x7efcfc109a80 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd03000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113d50 con 0x7efcfc104420 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd03000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc114050 con 0x7efcfc103a80 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efcf3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 0x7efcfc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efd01000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc109a80 0x7efcfc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efcf3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 0x7efcfc19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51720/0 (socket says 172.21.15.175:51720) 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efcf3e00640 1 -- 172.21.15.175:0/107505950 learned_addr learned my addr 172.21.15.175:0/107505950 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.953+0000 7efcfbe00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 0x7efcfc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf3e00640 1 -- 172.21.15.175:0/107505950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 msgr2=0x7efcfc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:17.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf3e00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 0x7efcfc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf3e00640 1 -- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc109a80 msgr2=0x7efcfc19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf3e00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc109a80 0x7efcfc19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf3e00640 1 -- 172.21.15.175:0/107505950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc19f2e0 con 0x7efcfc104420 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcfbe00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 0x7efcfc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf3e00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 0x7efcfc19a870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9f0 tx=0x7efce400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.954+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce400cd80 con 0x7efcfc104420 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.955+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc19f5d0 con 0x7efcfc104420 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.955+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc1a6eb0 con 0x7efcfc104420 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.955+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce4010910 con 0x7efcfc104420 2024-07-29T00:40:17.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.955+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce40188f0 con 0x7efcfc104420 2024-07-29T00:40:17.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.956+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd8005290 con 0x7efcfc104420 2024-07-29T00:40:17.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.959+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efce4018bb0 con 0x7efcfc104420 2024-07-29T00:40:17.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.960+0000 7efcf9600640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efce00a2d70 0x7efce00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:17.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.961+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7efce40cf6a0 con 0x7efcfc104420 2024-07-29T00:40:17.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.961+0000 7efcfbe00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efce00a2d70 0x7efce00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:17.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.962+0000 7efcfbe00640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efce00a2d70 0x7efce00a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcec009ba0 tx=0x7efcec005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:17.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:17.965+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efce409c240 con 0x7efcfc104420 2024-07-29T00:40:18.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.193+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcd8005850 con 0x7efcfc104420 2024-07-29T00:40:18.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.194+0000 7efcf9600640 1 -- 172.21.15.175:0/107505950 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7efce409bb40 con 0x7efcfc104420 2024-07-29T00:40:18.195 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:40:18.195 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:40:18.195 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:40:18.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efce00a2d70 msgr2=0x7efce00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:18.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efce00a2d70 0x7efce00a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcec009ba0 tx=0x7efcec005f90 comp rx=0 tx=0).stop 2024-07-29T00:40:18.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 msgr2=0x7efcfc19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:18.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 0x7efcfc19a870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9f0 tx=0x7efce400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:40:18.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 shutdown_connections 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7efce00a2d70 0x7efce00a5230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcfc109a80 0x7efcfc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcfc104420 0x7efcfc19a870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 --2- 172.21.15.175:0/107505950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcfc103a80 0x7efcfc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.197+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 >> 172.21.15.175:0/107505950 conn(0x7efcfc0ff760 msgr2=0x7efcfc0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.198+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 shutdown_connections 2024-07-29T00:40:18.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:18.198+0000 7efd03000640 1 -- 172.21.15.175:0/107505950 wait complete. 2024-07-29T00:40:18.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 9 sec latency 11736.011 ms 2024-07-29T00:40:19.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:40:19.470 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:19 smithi175 ceph-mon[32138]: pgmap v117: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 849 KiB/s wr, 56 op/s 2024-07-29T00:40:19.471 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:19 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/107505950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:19.492 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:19.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:19 smithi181 ceph-mon[36706]: pgmap v117: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 849 KiB/s wr, 56 op/s 2024-07-29T00:40:19.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:19 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/107505950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:19 smithi204 ceph-mon[37368]: pgmap v117: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 849 KiB/s wr, 56 op/s 2024-07-29T00:40:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:19 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/107505950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:19.879 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 10 sec latency 12736.105 ms 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 -- 172.21.15.175:0/1444943596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 msgr2=0x7f4610101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 --2- 172.21.15.175:0/1444943596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f4610101b20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 -- 172.21.15.175:0/1444943596 shutdown_connections 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 --2- 172.21.15.175:0/1444943596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4610102060 0x7f461010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 --2- 172.21.15.175:0/1444943596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f4610101b20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 --2- 172.21.15.175:0/1444943596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f4610107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 -- 172.21.15.175:0/1444943596 >> 172.21.15.175:0/1444943596 conn(0x7f46100fd400 msgr2=0x7f46100ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:20.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.164+0000 7f4616000640 1 -- 172.21.15.175:0/1444943596 shutdown_connections 2024-07-29T00:40:20.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.165+0000 7f4616000640 1 -- 172.21.15.175:0/1444943596 wait complete. 2024-07-29T00:40:20.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.165+0000 7f4616000640 1 Processor -- start 2024-07-29T00:40:20.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 -- start start 2024-07-29T00:40:20.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f461019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4610102060 0x7f461019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f461019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4610111ad0 con 0x7f4610102060 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4610111950 con 0x7f46101016c0 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4616000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4610111c50 con 0x7f46101076c0 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4615600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f461019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4615600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f461019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35150/0 (socket says 172.21.15.175:35150) 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4615600640 1 -- 172.21.15.175:0/2703733061 learned_addr learned my addr 172.21.15.175:0/2703733061 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4607e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4610102060 0x7f461019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.166+0000 7f4614c00640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f461019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4615600640 1 -- 172.21.15.175:0/2703733061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 msgr2=0x7f461019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4615600640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f461019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4615600640 1 -- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4610102060 msgr2=0x7f461019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4615600640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4610102060 0x7f461019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4615600640 1 -- 172.21.15.175:0/2703733061 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461019f130 con 0x7f46101076c0 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4614c00640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f461019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f4615600640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f461019ea50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f460800e970 tx=0x7f460800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f460e000640 1 -- 172.21.15.175:0/2703733061 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460800ccc0 con 0x7f46101076c0 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f460e000640 1 -- 172.21.15.175:0/2703733061 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460800ce60 con 0x7f46101076c0 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.167+0000 7f460e000640 1 -- 172.21.15.175:0/2703733061 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608010490 con 0x7f46101076c0 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.168+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461019f420 con 0x7f46101076c0 2024-07-29T00:40:20.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.168+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101a6d00 con 0x7f46101076c0 2024-07-29T00:40:20.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f460e000640 1 -- 172.21.15.175:0/2703733061 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4608005020 con 0x7f46101076c0 2024-07-29T00:40:20.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 Processor -- start 2024-07-29T00:40:20.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 -- start start 2024-07-29T00:40:20.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c6fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c6fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c709d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55665c662ab0 con 0x55665c6fcb70 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55665c5faa80 con 0x55665c707910 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.171+0000 7f461a1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55665c65ea70 con 0x55665c6fe0e0 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4607e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c6fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4614c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c6fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4607e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c6fc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35160/0 (socket says 172.21.15.175:35160) 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4607e00640 1 -- 172.21.15.175:0/2470215462 learned_addr learned my addr 172.21.15.175:0/2470215462 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4607e00640 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c709d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4614c00640 1 -- 172.21.15.175:0/2470215462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 msgr2=0x55665c6fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4614c00640 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c6fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4614c00640 1 -- 172.21.15.175:0/2470215462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 msgr2=0x55665c709d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4614c00640 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c709d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f4614c00640 1 -- 172.21.15.175:0/2470215462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665c3d43c0 con 0x55665c6fcb70 2024-07-29T00:40:20.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.172+0000 7f460e000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45f40a2cf0 0x7f45f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.173+0000 7f4614c00640 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c6fcfb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4600005fd0 tx=0x7f4600011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94072ae67532cbe server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:20.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.173+0000 7f460e000640 1 -- 172.21.15.175:0/2703733061 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f46080cf920 con 0x7f46101076c0 2024-07-29T00:40:20.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.173+0000 7f4607e00640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45f40a2cf0 0x7f45f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.173+0000 7f4605c00640 1 -- 172.21.15.175:0/2470215462 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600011e60 con 0x55665c6fcb70 2024-07-29T00:40:20.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.173+0000 7f4605c00640 1 -- 172.21.15.175:0/2470215462 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600004d20 con 0x55665c6fcb70 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.173+0000 7f4605c00640 1 -- 172.21.15.175:0/2470215462 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600004260 con 0x55665c6fcb70 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 -- 172.21.15.175:0/2470215462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 msgr2=0x55665c6fcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f4607e00640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45f40a2cf0 0x7f45f40a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55665c6fd610 tx=0x7f45f802a3f0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c6fcfb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4600005fd0 tx=0x7f4600011c70 comp rx=0 tx=0).stop 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 -- 172.21.15.175:0/2470215462 shutdown_connections 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c709d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c6fcfb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 --2- 172.21.15.175:0/2470215462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c6fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 -- 172.21.15.175:0/2470215462 >> 172.21.15.175:0/2470215462 conn(0x55665c6c5950 msgr2=0x55665c6f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 -- 172.21.15.175:0/2470215462 shutdown_connections 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.174+0000 7f461a1df740 1 -- 172.21.15.175:0/2470215462 wait complete. 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 -- start start 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 Processor -- start 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c704870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c704db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c7052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55665c70a6b0 con 0x55665c6fcb70 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55665c61dc10 con 0x55665c707910 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55665c70a830 con 0x55665c6fe0e0 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.175+0000 7f461a1df740 10 client.0 _ll_register_callbacks cb 0x55665c5e6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:20.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4614c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c704870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c7052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4607e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c704db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4614c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c704870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33800/0 (socket says 172.21.15.175:33800) 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c7052f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41258/0 (socket says 172.21.15.175:41258) 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4614c00640 1 -- 172.21.15.175:0/3841519452 learned_addr learned my addr 172.21.15.175:0/3841519452 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 -- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 msgr2=0x55665c704db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c704db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 -- 172.21.15.175:0/3841519452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 msgr2=0x55665c704870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c704870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 -- 172.21.15.175:0/3841519452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665c6211e0 con 0x55665c707910 2024-07-29T00:40:20.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f4615600640 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c7052f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f46080ccfc0 tx=0x7f460809c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132799884d039042 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:20.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.176+0000 7f45fe000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f45fe000640 1 -- 172.21.15.175:0/3841519452 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080a4a30 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4600004d20 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f45fe000640 1 -- 172.21.15.175:0/3841519452 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f4608005310 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f45fe000640 1 -- 172.21.15.175:0/3841519452 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f460807abe0 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 10 client.26153 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55665c716480 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f45fe000640 1 -- 172.21.15.175:0/3841519452 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600004d20 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f45fe000640 1 -- 172.21.15.175:0/3841519452 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f460809b380 con 0x55665c707910 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 10 client.26153 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 10 client.26153 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 20 client.26153 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:20.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c717890 0x55665c719d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 4 client.26153 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.177+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55665c670020 con 0x55665c717890 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.178+0000 7f4614c00640 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c717890 0x55665c719d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.178+0000 7f4614c00640 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c717890 0x55665c719d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=358cbb2dfe775f86 server_cookie=a8a3395cc58ba468 in_seq=0 out_seq=0 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.178+0000 7f45fe000640 10 client.26153 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f45fe000640 1 -- 172.21.15.175:0/3841519452 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55665c670020 con 0x55665c717890 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f45fe000640 10 client.26153 handle_command_reply: tid=0 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 client.26153 shutdown 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 msgr2=0x55665c7052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c7052f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f46080ccfc0 tx=0x7f460809c580 comp rx=0 tx=0).stop 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 shutdown_connections 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c717890 0x55665c719d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c7052f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fe0e0 0x55665c704db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f45fe000640 0 client.26153 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:40:20.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 --2- 172.21.15.175:0/3841519452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fcb70 0x55665c704870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.179+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 >> 172.21.15.175:0/3841519452 conn(0x55665c6c5950 msgr2=0x55665c6fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:20.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.180+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 shutdown_connections 2024-07-29T00:40:20.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.180+0000 7f461a1df740 1 -- 172.21.15.175:0/3841519452 wait complete. 2024-07-29T00:40:20.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.180+0000 7f461a1df740 20 client.26153 trim_cache size 0 max 16384 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 Processor -- start 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 -- start start 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c708d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c709240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c703140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46101af770 con 0x55665c6fe0e0 2024-07-29T00:40:20.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46101b1370 con 0x55665c707910 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.194+0000 7f461a1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46101afe40 con 0x55665c6fcb70 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4607e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c709240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4614c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c708d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4614c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c708d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35178/0 (socket says 172.21.15.175:35178) 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4607e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c709240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33814/0 (socket says 172.21.15.175:33814) 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4614c00640 1 -- 172.21.15.175:0/2056667052 learned_addr learned my addr 172.21.15.175:0/2056667052 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c703140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 -- 172.21.15.175:0/2056667052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 msgr2=0x55665c708d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c708d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 -- 172.21.15.175:0/2056667052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 msgr2=0x55665c709240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c709240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 -- 172.21.15.175:0/2056667052 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665c6ff6c0 con 0x55665c707910 2024-07-29T00:40:20.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.195+0000 7f4615600640 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c703140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4608098220 tx=0x7f460809a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89164d3209f53e9d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:20.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f4605200640 1 -- 172.21.15.175:0/2056667052 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080a4a30 con 0x55665c707910 2024-07-29T00:40:20.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f4605200640 1 -- 172.21.15.175:0/2056667052 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4608005310 con 0x55665c707910 2024-07-29T00:40:20.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f4605200640 1 -- 172.21.15.175:0/2056667052 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080a4800 con 0x55665c707910 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 -- 172.21.15.175:0/2056667052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 msgr2=0x55665c703140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c703140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4608098220 tx=0x7f460809a2d0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 -- 172.21.15.175:0/2056667052 shutdown_connections 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c703140 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c709240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 --2- 172.21.15.175:0/2056667052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c708d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 -- 172.21.15.175:0/2056667052 >> 172.21.15.175:0/2056667052 conn(0x55665c6c5950 msgr2=0x55665c6ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:20.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 -- 172.21.15.175:0/2056667052 shutdown_connections 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.196+0000 7f461a1df740 1 -- 172.21.15.175:0/2056667052 wait complete. 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.197+0000 7f461a1df740 1 -- start start 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.197+0000 7f461a1df740 1 Processor -- start 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c70fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c710380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c71ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46101b0be0 con 0x55665c6fe0e0 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46101b0510 con 0x55665c707910 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46101ad750 con 0x55665c6fcb70 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f461a1df740 10 client.0 _ll_register_callbacks cb 0x7f4608005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c70fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c70fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35194/0 (socket says 172.21.15.175:35194) 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 -- 172.21.15.175:0/4167701461 learned_addr learned my addr 172.21.15.175:0/4167701461 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4607e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c710380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4615600640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c71ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 -- 172.21.15.175:0/4167701461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 msgr2=0x55665c71ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c71ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 -- 172.21.15.175:0/4167701461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 msgr2=0x55665c710380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c710380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4614c00640 1 -- 172.21.15.175:0/4167701461 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665c71f5a0 con 0x55665c6fcb70 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.198+0000 7f4607e00640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c710380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f4614c00640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c70fe40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4600000f80 tx=0x7f4600003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbfb1fd11804713 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f45fe000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f45fe000640 1 -- 172.21.15.175:0/4167701461 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460001d070 con 0x55665c6fcb70 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4608005310 con 0x55665c6fcb70 2024-07-29T00:40:20.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f45fe000640 1 -- 172.21.15.175:0/4167701461 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f4600011240 con 0x55665c6fcb70 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f45fe000640 1 -- 172.21.15.175:0/4167701461 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f460000fa60 con 0x55665c6fcb70 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f461a1df740 10 client.26122 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.199+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55665c70f800 con 0x55665c6fcb70 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f45fe000640 1 -- 172.21.15.175:0/4167701461 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4608005310 con 0x55665c6fcb70 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f45fe000640 1 -- 172.21.15.175:0/4167701461 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4600023040 con 0x55665c6fcb70 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f461a1df740 10 client.26122 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f461a1df740 10 client.26122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f461a1df740 20 client.26122 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f461a1df740 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c782820 0x55665c784c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f461a1df740 4 client.26122 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55665c66a530 con 0x55665c782820 2024-07-29T00:40:20.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.200+0000 7f4607e00640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c782820 0x55665c784c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f4607e00640 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c782820 0x55665c784c40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7bfae06b378acb2 server_cookie=505060e858d455b0 in_seq=0 out_seq=0 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f45fe000640 10 client.26122 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f45fe000640 1 -- 172.21.15.175:0/4167701461 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55665c66a530 con 0x55665c782820 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f45fe000640 10 client.26122 handle_command_reply: tid=0 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 client.26122 shutdown 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 msgr2=0x55665c70fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c70fe40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4600000f80 tx=0x7f4600003ec0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 shutdown_connections 2024-07-29T00:40:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55665c782820 0x55665c784c40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55665c707910 0x55665c71ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f45fe000640 0 client.26122 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55665c6fe0e0 0x55665c710380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 --2- 172.21.15.175:0/4167701461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665c6fcb70 0x55665c70fe40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 >> 172.21.15.175:0/4167701461 conn(0x55665c6c5950 msgr2=0x55665c7030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 shutdown_connections 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.201+0000 7f461a1df740 1 -- 172.21.15.175:0/4167701461 wait complete. 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f461a1df740 20 client.26122 trim_cache size 0 max 16384 2024-07-29T00:40:20.207 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45f40a2cf0 msgr2=0x7f45f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45f40a2cf0 0x7f45f40a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55665c6fd610 tx=0x7f45f802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 msgr2=0x7f461019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f461019ea50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f460800e970 tx=0x7f460800ed70 comp rx=0 tx=0).stop 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 shutdown_connections 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46101076c0 0x7f461019ea50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45f40a2cf0 0x7f45f40a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4610102060 0x7f461019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 --2- 172.21.15.175:0/2703733061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46101016c0 0x7f461019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:20.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 >> 172.21.15.175:0/2703733061 conn(0x7f46100fd400 msgr2=0x7f46100fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:20.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.202+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 shutdown_connections 2024-07-29T00:40:20.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:20.203+0000 7f4616000640 1 -- 172.21.15.175:0/2703733061 wait complete. 2024-07-29T00:40:20.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 11 sec latency 13736.230 ms 2024-07-29T00:40:21.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:40:21.429 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:21 smithi175 ceph-mon[32138]: pgmap v118: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 850 KiB/s wr, 56 op/s 2024-07-29T00:40:21.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:21 smithi181 ceph-mon[36706]: pgmap v118: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 850 KiB/s wr, 56 op/s 2024-07-29T00:40:21.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:21 smithi204 ceph-mon[37368]: pgmap v118: 97 pgs: 97 active+clean; 70 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 850 KiB/s wr, 56 op/s 2024-07-29T00:40:21.809 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:21.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 12 sec latency 14736.327 ms 2024-07-29T00:40:21.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:40:22.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.456+0000 7fcd7ce00640 1 -- 172.21.15.175:0/119688900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 msgr2=0x7fcd78109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.456+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/119688900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd78109e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009930 tx=0x7fcd6802a420 comp rx=0 tx=0).stop 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 -- 172.21.15.175:0/119688900 shutdown_connections 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/119688900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78104420 0x7fcd78110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/119688900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78103a80 0x7fcd78103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/119688900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd78109e60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 -- 172.21.15.175:0/119688900 >> 172.21.15.175:0/119688900 conn(0x7fcd780ff760 msgr2=0x7fcd78101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 -- 172.21.15.175:0/119688900 shutdown_connections 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.457+0000 7fcd7ce00640 1 -- 172.21.15.175:0/119688900 wait complete. 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 Processor -- start 2024-07-29T00:40:22.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 -- start start 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 0x7fcd7819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78104420 0x7fcd7819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd7819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113ed0 con 0x7fcd78109a80 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113d50 con 0x7fcd78104420 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd7ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78114050 con 0x7fcd78103a80 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd76800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd7819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 0x7fcd7819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 0x7fcd7819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35218/0 (socket says 172.21.15.175:35218) 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd76800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd7819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33848/0 (socket says 172.21.15.175:33848) 2024-07-29T00:40:22.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.458+0000 7fcd75e00640 1 -- 172.21.15.175:0/674389713 learned_addr learned my addr 172.21.15.175:0/674389713 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd6d400640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78104420 0x7fcd7819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd76800640 1 -- 172.21.15.175:0/674389713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 msgr2=0x7fcd7819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd76800640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 0x7fcd7819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd76800640 1 -- 172.21.15.175:0/674389713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78104420 msgr2=0x7fcd7819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd76800640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78104420 0x7fcd7819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd76800640 1 -- 172.21.15.175:0/674389713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7819f340 con 0x7fcd78109a80 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd75e00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 0x7fcd7819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd6d400640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78104420 0x7fcd7819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd76800640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd7819ec60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e940 tx=0x7fcd7000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:22.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd6f400640 1 -- 172.21.15.175:0/674389713 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70004d20 con 0x7fcd78109a80 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.459+0000 7fcd6f400640 1 -- 172.21.15.175:0/674389713 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd70005020 con 0x7fcd78109a80 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.460+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7819f630 con 0x7fcd78109a80 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.460+0000 7fcd6f400640 1 -- 172.21.15.175:0/674389713 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700116b0 con 0x7fcd78109a80 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.460+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a6f10 con 0x7fcd78109a80 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd6f400640 1 -- 172.21.15.175:0/674389713 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcd700054d0 con 0x7fcd78109a80 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 Processor -- start 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 -- start start 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff42f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 0x55b2ff42fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff43cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b2ff395ab0 con 0x55b2ff42f980 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b2ff32da80 con 0x55b2ff430ef0 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.463+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b2ff391a70 con 0x55b2ff43a720 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 0x55b2ff42fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 0x55b2ff42fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33862/0 (socket says 172.21.15.175:33862) 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 -- 172.21.15.175:0/3512778605 learned_addr learned my addr 172.21.15.175:0/3512778605 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd6d400640 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff42f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd6d400640 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff43cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 -- 172.21.15.175:0/3512778605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 msgr2=0x55b2ff43cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff43cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 -- 172.21.15.175:0/3512778605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 msgr2=0x55b2ff42f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff42f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 -- 172.21.15.175:0/3512778605 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2ff1073c0 con 0x55b2ff42f980 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd6d400640 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff43cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd6f400640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcd580a2cf0 0x7fcd580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.464+0000 7fcd75e00640 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 0x55b2ff42fdc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcd680044d0 tx=0x7fcd68005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ad565faf512d42 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.465+0000 7fcd6f400640 1 -- 172.21.15.175:0/674389713 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fcd700cf8c0 con 0x7fcd78109a80 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.465+0000 7fcd6d400640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcd580a2cf0 0x7fcd580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.465+0000 7fcd67400640 1 -- 172.21.15.175:0/3512778605 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68035040 con 0x55b2ff42f980 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.465+0000 7fcd67400640 1 -- 172.21.15.175:0/3512778605 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd68004760 con 0x55b2ff42f980 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.465+0000 7fcd67400640 1 -- 172.21.15.175:0/3512778605 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68003b50 con 0x55b2ff42f980 2024-07-29T00:40:22.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 -- 172.21.15.175:0/3512778605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 msgr2=0x55b2ff42fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd6d400640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcd580a2cf0 0x7fcd580a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55b2ff430420 tx=0x7fcd6000b470 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 0x55b2ff42fdc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcd680044d0 tx=0x7fcd68005c00 comp rx=0 tx=0).stop 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 -- 172.21.15.175:0/3512778605 shutdown_connections 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff43cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff42f980 0x55b2ff42fdc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/3512778605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff42f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 -- 172.21.15.175:0/3512778605 >> 172.21.15.175:0/3512778605 conn(0x55b2ff3f8950 msgr2=0x55b2ff42c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 -- 172.21.15.175:0/3512778605 shutdown_connections 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.466+0000 7fcd7f80b740 1 -- 172.21.15.175:0/3512778605 wait complete. 2024-07-29T00:40:22.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 -- start start 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 Processor -- start 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff42f980 0x55b2ff4376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 0x55b2ff437bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff438130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b2ff43d460 con 0x55b2ff430ef0 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b2ff350c10 con 0x55b2ff42f980 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b2ff43d5e0 con 0x55b2ff43a720 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.467+0000 7fcd7f80b740 10 client.0 _ll_register_callbacks cb 0x55b2ff319a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 0x55b2ff437bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff42f980 0x55b2ff4376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 0x55b2ff437bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33866/0 (socket says 172.21.15.175:33866) 2024-07-29T00:40:22.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 -- 172.21.15.175:0/2881079327 learned_addr learned my addr 172.21.15.175:0/2881079327 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd76800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff438130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 -- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 msgr2=0x55b2ff438130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff438130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 -- 172.21.15.175:0/2881079327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff42f980 msgr2=0x55b2ff4376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff42f980 0x55b2ff4376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 -- 172.21.15.175:0/2881079327 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2ff3541e0 con 0x55b2ff430ef0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd6d400640 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 0x55b2ff437bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd600118c0 tx=0x7fcd60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f0f90a8f1c2f0c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.468+0000 7fcd5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2881079327 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6001eb40 con 0x55b2ff430ef0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2881079327 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fcd6001ece0 con 0x55b2ff430ef0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd68004760 con 0x55b2ff430ef0 2024-07-29T00:40:22.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2881079327 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcd60030480 con 0x55b2ff430ef0 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd7f80b740 10 client.16044 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b2ff4491c0 con 0x55b2ff430ef0 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2881079327 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd68004760 con 0x55b2ff430ef0 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.469+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2881079327 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fcd6001c5d0 con 0x55b2ff430ef0 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd7f80b740 10 client.16044 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd7f80b740 10 client.16044 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd7f80b740 20 client.16044 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff44a680 0x55b2ff44caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd7f80b740 4 client.16044 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:22.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2ff3a3020 con 0x55b2ff44a680 2024-07-29T00:40:22.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.470+0000 7fcd75e00640 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff44a680 0x55b2ff44caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.471+0000 7fcd75e00640 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff44a680 0x55b2ff44caf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9066c579d4c4e2ee server_cookie=d1cf35082e0bd403 in_seq=0 out_seq=0 2024-07-29T00:40:22.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.471+0000 7fcd5fe00640 10 client.16044 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:40:22.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.471+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2881079327 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b2ff3a3020 con 0x55b2ff44a680 2024-07-29T00:40:22.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.471+0000 7fcd5fe00640 10 client.16044 handle_command_reply: tid=0 2024-07-29T00:40:22.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 client.16044 shutdown 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 msgr2=0x55b2ff437bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 0x55b2ff437bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd600118c0 tx=0x7fcd60004730 comp rx=0 tx=0).stop 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 shutdown_connections 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff44a680 0x55b2ff44caf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff43a720 0x55b2ff438130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff430ef0 0x55b2ff437bf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2881079327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff42f980 0x55b2ff4376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 >> 172.21.15.175:0/2881079327 conn(0x55b2ff3f8950 msgr2=0x55b2ff431ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:22.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd5fe00640 0 client.16044 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:40:22.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 shutdown_connections 2024-07-29T00:40:22.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2881079327 wait complete. 2024-07-29T00:40:22.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.472+0000 7fcd7f80b740 20 client.16044 trim_cache size 0 max 16384 2024-07-29T00:40:22.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 Processor -- start 2024-07-29T00:40:22.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 -- start start 2024-07-29T00:40:22.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff42c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff4523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff452900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781af9a0 con 0x55b2ff43a720 2024-07-29T00:40:22.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781b15a0 con 0x55b2ff430ef0 2024-07-29T00:40:22.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.487+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781b0070 con 0x55b2ff42f980 2024-07-29T00:40:22.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff452900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff452900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33874/0 (socket says 172.21.15.175:33874) 2024-07-29T00:40:22.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 -- 172.21.15.175:0/1221510142 learned_addr learned my addr 172.21.15.175:0/1221510142 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:22.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd6d400640 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff4523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 -- 172.21.15.175:0/1221510142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 msgr2=0x55b2ff42c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff42c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 -- 172.21.15.175:0/1221510142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 msgr2=0x55b2ff4523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff4523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 -- 172.21.15.175:0/1221510142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2ff4365f0 con 0x55b2ff43a720 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd6d400640 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff4523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.488+0000 7fcd76800640 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff452900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcd7009ba60 tx=0x7fcd7009c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0bf779651a0b40 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:22.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd66a00640 1 -- 172.21.15.175:0/1221510142 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7001f030 con 0x55b2ff43a720 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd66a00640 1 -- 172.21.15.175:0/1221510142 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd700a0110 con 0x55b2ff43a720 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd66a00640 1 -- 172.21.15.175:0/1221510142 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700a4540 con 0x55b2ff43a720 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 -- 172.21.15.175:0/1221510142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 msgr2=0x55b2ff452900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff452900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcd7009ba60 tx=0x7fcd7009c520 comp rx=0 tx=0).stop 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 -- 172.21.15.175:0/1221510142 shutdown_connections 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff452900 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff4523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/1221510142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff42c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.489+0000 7fcd7f80b740 1 -- 172.21.15.175:0/1221510142 >> 172.21.15.175:0/1221510142 conn(0x55b2ff3f8950 msgr2=0x55b2ff42bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.490+0000 7fcd7f80b740 1 -- 172.21.15.175:0/1221510142 shutdown_connections 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.490+0000 7fcd7f80b740 1 -- 172.21.15.175:0/1221510142 wait complete. 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.490+0000 7fcd7f80b740 1 -- start start 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.490+0000 7fcd7f80b740 1 Processor -- start 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff442c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff434090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff4345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781b0e10 con 0x55b2ff43a720 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781b0740 con 0x55b2ff430ef0 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781ad980 con 0x55b2ff42f980 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd7f80b740 10 client.0 _ll_register_callbacks cb 0x7fcd6000bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff442c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff442c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35258/0 (socket says 172.21.15.175:35258) 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd75e00640 1 -- 172.21.15.175:0/2407701844 learned_addr learned my addr 172.21.15.175:0/2407701844 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd76800640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff4345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.491+0000 7fcd6d400640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff434090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd75e00640 1 -- 172.21.15.175:0/2407701844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 msgr2=0x55b2ff434090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd75e00640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff434090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd75e00640 1 -- 172.21.15.175:0/2407701844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 msgr2=0x55b2ff4345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd75e00640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff4345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd75e00640 1 -- 172.21.15.175:0/2407701844 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2ff434b10 con 0x55b2ff42f980 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd75e00640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff442c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcd680044a0 tx=0x7fcd68000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7344bd277ee0f95 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2407701844 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68035040 con 0x55b2ff42f980 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2407701844 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fcd68005240 con 0x55b2ff42f980 2024-07-29T00:40:22.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.492+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd700a0110 con 0x55b2ff42f980 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2407701844 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcd68005440 con 0x55b2ff42f980 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd7f80b740 10 client.26134 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b2ff4b50a0 con 0x55b2ff42f980 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2407701844 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd700a0110 con 0x55b2ff42f980 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2407701844 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fcd68031640 con 0x55b2ff42f980 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd7f80b740 10 client.26134 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd7f80b740 10 client.26134 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.493+0000 7fcd7f80b740 20 client.26134 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.494+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff4b5390 0x55b2ff4b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.494+0000 7fcd7f80b740 4 client.26134 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.494+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2ff39d530 con 0x55b2ff4b5390 2024-07-29T00:40:22.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.494+0000 7fcd6d400640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff4b5390 0x55b2ff4b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:22.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.494+0000 7fcd6d400640 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff4b5390 0x55b2ff4b7780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=993d767ebf69a671 server_cookie=7bd519dfd7d8e204 in_seq=0 out_seq=0 2024-07-29T00:40:22.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.494+0000 7fcd5fe00640 10 client.26134 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:40:22.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd5fe00640 1 -- 172.21.15.175:0/2407701844 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2ff39d530 con 0x55b2ff4b5390 2024-07-29T00:40:22.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd5fe00640 10 client.26134 handle_command_reply: tid=0 2024-07-29T00:40:22.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 client.26134 shutdown 2024-07-29T00:40:22.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 msgr2=0x55b2ff442c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff442c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcd680044a0 tx=0x7fcd68000bc0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 shutdown_connections 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55b2ff4b5390 0x55b2ff4b7780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2ff43a720 0x55b2ff4345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b2ff430ef0 0x55b2ff434090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd5fe00640 0 client.26134 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 --2- 172.21.15.175:0/2407701844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2ff42f980 0x55b2ff442c70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 >> 172.21.15.175:0/2407701844 conn(0x55b2ff3f8950 msgr2=0x55b2ff4375c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 shutdown_connections 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 1 -- 172.21.15.175:0/2407701844 wait complete. 2024-07-29T00:40:22.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.495+0000 7fcd7f80b740 20 client.26134 trim_cache size 0 max 16384 2024-07-29T00:40:22.504 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:40:22.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcd580a2cf0 msgr2=0x7fcd580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcd580a2cf0 0x7fcd580a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55b2ff430420 tx=0x7fcd6000b470 comp rx=0 tx=0).stop 2024-07-29T00:40:22.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 msgr2=0x7fcd7819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd7819ec60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e940 tx=0x7fcd7000ee00 comp rx=0 tx=0).stop 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 shutdown_connections 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcd580a2cf0 0x7fcd580a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd78109a80 0x7fcd7819ec60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd78104420 0x7fcd7819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 --2- 172.21.15.175:0/674389713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd78103a80 0x7fcd7819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 >> 172.21.15.175:0/674389713 conn(0x7fcd780ff760 msgr2=0x7fcd780ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 shutdown_connections 2024-07-29T00:40:22.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:22.496+0000 7fcd7ce00640 1 -- 172.21.15.175:0/674389713 wait complete. 2024-07-29T00:40:22.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 13 sec latency 15736.403 ms 2024-07-29T00:40:23.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:23 smithi175 ceph-mon[32138]: pgmap v119: 97 pgs: 97 active+clean; 72 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2024-07-29T00:40:23.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:23 smithi181 ceph-mon[36706]: pgmap v119: 97 pgs: 97 active+clean; 72 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2024-07-29T00:40:23.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:23 smithi204 ceph-mon[37368]: pgmap v119: 97 pgs: 97 active+clean; 72 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2024-07-29T00:40:23.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:40:23.664 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:23.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 14 sec latency 16736.473 ms 2024-07-29T00:40:24.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.438+0000 7eff56c00640 1 -- 172.21.15.175:0/3619481014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff50074ba0 msgr2=0x7eff50112190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:24.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.438+0000 7eff56c00640 1 --2- 172.21.15.175:0/3619481014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff50074ba0 0x7eff50112190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009900 tx=0x7eff3c02a480 comp rx=0 tx=0).stop 2024-07-29T00:40:24.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 -- 172.21.15.175:0/3619481014 shutdown_connections 2024-07-29T00:40:24.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 --2- 172.21.15.175:0/3619481014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff501126d0 0x7eff50114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 --2- 172.21.15.175:0/3619481014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff50074ba0 0x7eff50112190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 --2- 172.21.15.175:0/3619481014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff50074280 0x7eff50074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 -- 172.21.15.175:0/3619481014 >> 172.21.15.175:0/3619481014 conn(0x7eff500fd400 msgr2=0x7eff500ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:24.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 -- 172.21.15.175:0/3619481014 shutdown_connections 2024-07-29T00:40:24.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.439+0000 7eff56c00640 1 -- 172.21.15.175:0/3619481014 wait complete. 2024-07-29T00:40:24.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 Processor -- start 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 -- start start 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 0x7eff5019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff50074ba0 0x7eff5019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff501126d0 0x7eff501a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff50118090 con 0x7eff501126d0 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff50117f10 con 0x7eff50074280 2024-07-29T00:40:24.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.440+0000 7eff56c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff50118210 con 0x7eff50074ba0 2024-07-29T00:40:24.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 0x7eff5019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:24.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 0x7eff5019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41348/0 (socket says 172.21.15.175:41348) 2024-07-29T00:40:24.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 -- 172.21.15.175:0/2072764801 learned_addr learned my addr 172.21.15.175:0/2072764801 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:24.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff56200640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff501126d0 0x7eff501a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:24.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff54e00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff50074ba0 0x7eff5019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 -- 172.21.15.175:0/2072764801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff50074ba0 msgr2=0x7eff5019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff50074ba0 0x7eff5019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 -- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff501126d0 msgr2=0x7eff501a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff501126d0 0x7eff501a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.441+0000 7eff55800640 1 -- 172.21.15.175:0/2072764801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff501a35a0 con 0x7eff50074280 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.442+0000 7eff55800640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 0x7eff5019e5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.442+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400cd80 con 0x7eff50074280 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.442+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff44005500 con 0x7eff50074280 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.442+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff501a3890 con 0x7eff50074280 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.442+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44010690 con 0x7eff50074280 2024-07-29T00:40:24.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.442+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501ab170 con 0x7eff50074280 2024-07-29T00:40:24.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.443+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff50074660 con 0x7eff50074280 2024-07-29T00:40:24.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.445+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7eff440046c0 con 0x7eff50074280 2024-07-29T00:40:24.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.446+0000 7eff4e000640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7eff380a2d70 0x7eff380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:24.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.446+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7eff440cf900 con 0x7eff50074280 2024-07-29T00:40:24.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.447+0000 7eff54e00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7eff380a2d70 0x7eff380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:24.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.448+0000 7eff54e00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7eff380a2d70 0x7eff380a5230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff500ffc50 tx=0x7eff3c005a20 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:24.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.450+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7eff4409c4a0 con 0x7eff50074280 2024-07-29T00:40:24.508 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:24.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.692+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff50104b10 con 0x7eff50074280 2024-07-29T00:40:24.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.693+0000 7eff4e000640 1 -- 172.21.15.175:0/2072764801 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7eff4409bda0 con 0x7eff50074280 2024-07-29T00:40:24.694 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:40:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:40:24.695 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:40:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.696+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7eff380a2d70 msgr2=0x7eff380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.696+0000 7eff56c00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7eff380a2d70 0x7eff380a5230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff500ffc50 tx=0x7eff3c005a20 comp rx=0 tx=0).stop 2024-07-29T00:40:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.696+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 msgr2=0x7eff5019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.696+0000 7eff56c00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 0x7eff5019e5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400edf0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 shutdown_connections 2024-07-29T00:40:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7eff380a2d70 0x7eff380a5230 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff501126d0 0x7eff501a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff50074ba0 0x7eff5019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 --2- 172.21.15.175:0/2072764801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50074280 0x7eff5019e5f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 >> 172.21.15.175:0/2072764801 conn(0x7eff500fd400 msgr2=0x7eff500fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 shutdown_connections 2024-07-29T00:40:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:24.699+0000 7eff56c00640 1 -- 172.21.15.175:0/2072764801 wait complete. 2024-07-29T00:40:24.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 15 sec latency 17736.518 ms 2024-07-29T00:40:25.171 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:25 smithi175 ceph-mon[32138]: pgmap v120: 97 pgs: 97 active+clean; 72 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 964 KiB/s wr, 45 op/s 2024-07-29T00:40:25.171 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:25 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2072764801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 -- 172.21.15.175:0/3932052310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d04102a80 msgr2=0x7f0d04102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 --2- 172.21.15.175:0/3932052310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d04102a80 0x7f0d04102e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009900 tx=0x7f0cf802a480 comp rx=0 tx=0).stop 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 -- 172.21.15.175:0/3932052310 shutdown_connections 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 --2- 172.21.15.175:0/3932052310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d0410be60 0x7f0d0410e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 --2- 172.21.15.175:0/3932052310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d041033a0 0x7f0d0410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 --2- 172.21.15.175:0/3932052310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d04102a80 0x7f0d04102e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.294+0000 7f0d0b400640 1 -- 172.21.15.175:0/3932052310 >> 172.21.15.175:0/3932052310 conn(0x7f0d040fc480 msgr2=0x7f0d040fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:25.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.296+0000 7f0d0b400640 1 -- 172.21.15.175:0/3932052310 shutdown_connections 2024-07-29T00:40:25.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.296+0000 7f0d0b400640 1 -- 172.21.15.175:0/3932052310 wait complete. 2024-07-29T00:40:25.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 Processor -- start 2024-07-29T00:40:25.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 -- start start 2024-07-29T00:40:25.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d04102a80 0x7f0d0419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 0x7f0d0419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0410be60 0x7f0d0419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04111c90 con 0x7f0d0410be60 2024-07-29T00:40:25.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04111b10 con 0x7f0d041033a0 2024-07-29T00:40:25.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.297+0000 7f0d0b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04111e10 con 0x7f0d04102a80 2024-07-29T00:40:25.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 0x7f0d0419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d09400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0410be60 0x7f0d0419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 0x7f0d0419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41360/0 (socket says 172.21.15.175:41360) 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 -- 172.21.15.175:0/355691061 learned_addr learned my addr 172.21.15.175:0/355691061 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d08a00640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d04102a80 0x7f0d0419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 -- 172.21.15.175:0/355691061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d04102a80 msgr2=0x7f0d0419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d04102a80 0x7f0d0419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 -- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0410be60 msgr2=0x7f0d0419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0410be60 0x7f0d0419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.298+0000 7f0d03e00640 1 -- 172.21.15.175:0/355691061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0419f2e0 con 0x7f0d041033a0 2024-07-29T00:40:25.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.299+0000 7f0d03e00640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 0x7f0d0419a870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000e9f0 tx=0x7f0cf000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.299+0000 7f0d01600640 1 -- 172.21.15.175:0/355691061 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000cd80 con 0x7f0d041033a0 2024-07-29T00:40:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.299+0000 7f0d01600640 1 -- 172.21.15.175:0/355691061 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0010910 con 0x7f0d041033a0 2024-07-29T00:40:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.299+0000 7f0d01600640 1 -- 172.21.15.175:0/355691061 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0018980 con 0x7f0d041033a0 2024-07-29T00:40:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.299+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0419f5d0 con 0x7f0d041033a0 2024-07-29T00:40:25.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.299+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d041a6eb0 con 0x7f0d041033a0 2024-07-29T00:40:25.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.303+0000 7f0d01600640 1 -- 172.21.15.175:0/355691061 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0cf0010ac0 con 0x7f0d041033a0 2024-07-29T00:40:25.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.304+0000 7f0d0ddf3740 1 Processor -- start 2024-07-29T00:40:25.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.304+0000 7f0d01600640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0cec0a2d70 0x7f0cec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.304+0000 7f0d0ddf3740 1 -- start start 2024-07-29T00:40:25.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.304+0000 7f0d01600640 1 -- 172.21.15.175:0/355691061 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0cf00cf760 con 0x7f0d041033a0 2024-07-29T00:40:25.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.304+0000 7f0d09400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0cec0a2d70 0x7f0cec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fa7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fa7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fbd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557a58f0dab0 con 0x557a58fa7980 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a58ea5a80 con 0x557a58fa8ef0 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a58f09a70 con 0x557a58fbaf30 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d09400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fbd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d03e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fa7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fa7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d09400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fbd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35314/0 (socket says 172.21.15.175:35314) 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d03e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fa7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33926/0 (socket says 172.21.15.175:33926) 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d09400640 1 -- 172.21.15.175:0/2416785874 learned_addr learned my addr 172.21.15.175:0/2416785874 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.305+0000 7f0d09400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0cec0a2d70 0x7f0cec0a5230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005fd0 tx=0x7f0cfc011810 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:25.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0d08a00640 1 -- 172.21.15.175:0/2416785874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 msgr2=0x557a58fbd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0d08a00640 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fbd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0d08a00640 1 -- 172.21.15.175:0/2416785874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 msgr2=0x557a58fa7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0d08a00640 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fa7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0d08a00640 1 -- 172.21.15.175:0/2416785874 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a58c7f3c0 con 0x557a58fa8ef0 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0d08a00640 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fa7440 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0cf802aa70 tx=0x7f0cf8005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29716ddf062fa8d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0cf5c00640 1 -- 172.21.15.175:0/2416785874 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8035040 con 0x557a58fa8ef0 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0cf5c00640 1 -- 172.21.15.175:0/2416785874 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8004400 con 0x557a58fa8ef0 2024-07-29T00:40:25.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.306+0000 7f0cf5c00640 1 -- 172.21.15.175:0/2416785874 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80035f0 con 0x557a58fa8ef0 2024-07-29T00:40:25.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/2416785874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 msgr2=0x557a58fa7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fa7440 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0cf802aa70 tx=0x7f0cf8005de0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/2416785874 shutdown_connections 2024-07-29T00:40:25.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fbd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fa7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/2416785874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fa7440 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/2416785874 >> 172.21.15.175:0/2416785874 conn(0x557a58f70950 msgr2=0x557a58fa4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/2416785874 shutdown_connections 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.307+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/2416785874 wait complete. 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.308+0000 7f0d0ddf3740 1 -- start start 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.308+0000 7f0d0ddf3740 1 Processor -- start 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa7980 0x557a58fb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fa8ef0 0x557a58fb8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 0x557a58fb8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557a58fbdc70 con 0x557a58fbaf30 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a58ec8c10 con 0x557a58fa7980 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a58fbddf0 con 0x557a58fa8ef0 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d0ddf3740 10 client.0 _ll_register_callbacks cb 0x557a58e91a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 0x557a58fb8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 0x557a58fb8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33930/0 (socket says 172.21.15.175:33930) 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 -- 172.21.15.175:0/1108587609 learned_addr learned my addr 172.21.15.175:0/1108587609 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa7980 0x557a58fb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d03e00640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fa8ef0 0x557a58fb8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 -- 172.21.15.175:0/1108587609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fa8ef0 msgr2=0x557a58fb8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fa8ef0 0x557a58fb8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 -- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa7980 msgr2=0x557a58fb7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa7980 0x557a58fb7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d09400640 1 -- 172.21.15.175:0/1108587609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a58ecc1e0 con 0x557a58fbaf30 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.309+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa7980 0x557a58fb7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0d09400640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 0x557a58fb8980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004210 tx=0x7f0cfc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8561325a4adef05f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0cea000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0cea000640 1 -- 172.21.15.175:0/1108587609 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc017040 con 0x557a58fbaf30 2024-07-29T00:40:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0cea000640 1 -- 172.21.15.175:0/1108587609 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0cfc01e950 con 0x557a58fbaf30 2024-07-29T00:40:25.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf8004400 con 0x557a58fbaf30 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0cea000640 1 -- 172.21.15.175:0/1108587609 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc01eb50 con 0x557a58fbaf30 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0d0ddf3740 10 client.16068 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.310+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557a58fc9a30 con 0x557a58fbaf30 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.311+0000 7f0cea000640 1 -- 172.21.15.175:0/1108587609 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8004400 con 0x557a58fbaf30 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.311+0000 7f0cea000640 1 -- 172.21.15.175:0/1108587609 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0cfc02f3d0 con 0x557a58fbaf30 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d0ddf3740 10 client.16068 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d0ddf3740 10 client.16068 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d0ddf3740 20 client.16068 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a58fcae40 0x557a58fcd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d0ddf3740 4 client.16068 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a58f1b020 con 0x557a58fcae40 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.312+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a58fcae40 0x557a58fcd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.313+0000 7f0cea000640 10 client.16068 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.313+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a58fcae40 0x557a58fcd230 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f1db9dbb4b3af2a server_cookie=89f726955a3200c7 in_seq=0 out_seq=0 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.313+0000 7f0cea000640 1 -- 172.21.15.175:0/1108587609 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a58f1b020 con 0x557a58fcae40 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.313+0000 7f0cea000640 10 client.16068 handle_command_reply: tid=0 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.313+0000 7f0d0ddf3740 1 client.16068 shutdown 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 msgr2=0x557a58fb8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 0x557a58fb8980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004210 tx=0x7f0cfc016000 comp rx=0 tx=0).stop 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 shutdown_connections 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fbaf30 0x557a58fb8980 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fa8ef0 0x557a58fb8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a58fcae40 0x557a58fcd230 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1108587609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa7980 0x557a58fb7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 >> 172.21.15.175:0/1108587609 conn(0x557a58f70950 msgr2=0x557a58fa9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0cea000640 0 client.16068 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:40:25.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 shutdown_connections 2024-07-29T00:40:25.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1108587609 wait complete. 2024-07-29T00:40:25.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.314+0000 7f0d0ddf3740 20 client.16068 trim_cache size 0 max 16384 2024-07-29T00:40:25.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.328+0000 7f0d0ddf3740 1 Processor -- start 2024-07-29T00:40:25.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.328+0000 7f0d0ddf3740 1 -- start start 2024-07-29T00:40:25.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.328+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fb85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.328+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fd2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fd3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041ab4d0 con 0x557a58fa7980 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041ad0d0 con 0x557a58fa8ef0 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041abba0 con 0x557a58fbaf30 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fb85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fb85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33932/0 (socket says 172.21.15.175:33932) 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 -- 172.21.15.175:0/1935724603 learned_addr learned my addr 172.21.15.175:0/1935724603 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:25.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d03e00640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fd2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d09400640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fd3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 -- 172.21.15.175:0/1935724603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 msgr2=0x557a58fd3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fd3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 -- 172.21.15.175:0/1935724603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 msgr2=0x557a58fd2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fd2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.329+0000 7f0d08a00640 1 -- 172.21.15.175:0/1935724603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a58fa4d10 con 0x557a58fa7980 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.330+0000 7f0d09400640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.330+0000 7f0d03e00640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fd2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:25.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.330+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fb85d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8040000 tx=0x7f0cf8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7a247f19fc2d4c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:25.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.330+0000 7f0cf5200640 1 -- 172.21.15.175:0/1935724603 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8035040 con 0x557a58fa7980 2024-07-29T00:40:25.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.330+0000 7f0cf5200640 1 -- 172.21.15.175:0/1935724603 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8004480 con 0x557a58fa7980 2024-07-29T00:40:25.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.330+0000 7f0cf5200640 1 -- 172.21.15.175:0/1935724603 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8003850 con 0x557a58fa7980 2024-07-29T00:40:25.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1935724603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 msgr2=0x557a58fb85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fb85d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8040000 tx=0x7f0cf8000bc0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1935724603 shutdown_connections 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fd2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1935724603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fb85d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1935724603 >> 172.21.15.175:0/1935724603 conn(0x557a58f70950 msgr2=0x557a58fb7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1935724603 shutdown_connections 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.331+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1935724603 wait complete. 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 -- start start 2024-07-29T00:40:25.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 Processor -- start 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fc3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fc35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fb4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041ac940 con 0x557a58fa7980 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041ac270 con 0x557a58fa8ef0 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041a94d0 con 0x557a58fbaf30 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.332+0000 7f0d0ddf3740 10 client.0 _ll_register_callbacks cb 0x7f0cfc003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fc35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fc3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fc35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41400/0 (socket says 172.21.15.175:41400) 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 -- 172.21.15.175:0/1663472116 learned_addr learned my addr 172.21.15.175:0/1663472116 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d09400640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fb4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 -- 172.21.15.175:0/1663472116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 msgr2=0x557a58fb4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fb4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 -- 172.21.15.175:0/1663472116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 msgr2=0x557a58fc3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fc3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 -- 172.21.15.175:0/1663472116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a58fb4a90 con 0x557a58fa8ef0 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d09400640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fb4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0d03e00640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fc35c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0cf009bc40 tx=0x7f0cf00cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59cb27a26a02ff2d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0cea000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.333+0000 7f0cea000640 1 -- 172.21.15.175:0/1663472116 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00b9060 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0cea000640 1 -- 172.21.15.175:0/1663472116 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0cf009b310 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf8004480 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0cea000640 1 -- 172.21.15.175:0/1663472116 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0cf0018b20 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 10 client.26201 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557a59035310 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0cea000640 1 -- 172.21.15.175:0/1663472116 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8004480 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0cea000640 1 -- 172.21.15.175:0/1663472116 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0cf009ac90 con 0x557a58fa8ef0 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 10 client.26201 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 10 client.26201 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 20 client.26201 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a59035a70 0x557a59037e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 4 client.26201 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.334+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a58f15530 con 0x557a59035a70 2024-07-29T00:40:25.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.335+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a59035a70 0x557a59037e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:25.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.335+0000 7f0d08a00640 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a59035a70 0x557a59037e90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c65eb7f1cfa1934 server_cookie=200cda4ad76bdd35 in_seq=0 out_seq=0 2024-07-29T00:40:25.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.335+0000 7f0cea000640 10 client.26201 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.335+0000 7f0cea000640 1 -- 172.21.15.175:0/1663472116 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a58f15530 con 0x557a59035a70 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.335+0000 7f0cea000640 10 client.26201 handle_command_reply: tid=0 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.335+0000 7f0d0ddf3740 1 client.26201 shutdown 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 msgr2=0x557a58fc35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fc35c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0cf009bc40 tx=0x7f0cf00cf040 comp rx=0 tx=0).stop 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 shutdown_connections 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a58fbaf30 0x557a58fb4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x557a59035a70 0x557a59037e90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a58fa8ef0 0x557a58fc35c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 --2- 172.21.15.175:0/1663472116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557a58fa7980 0x557a58fc3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0cea000640 0 client.26201 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 >> 172.21.15.175:0/1663472116 conn(0x557a58f70950 msgr2=0x557a58fa2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 shutdown_connections 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 1 -- 172.21.15.175:0/1663472116 wait complete. 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.336+0000 7f0d0ddf3740 20 client.26201 trim_cache size 0 max 16384 2024-07-29T00:40:25.342 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0cec0a2d70 msgr2=0x7f0cec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0cec0a2d70 0x7f0cec0a5230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005fd0 tx=0x7f0cfc011810 comp rx=0 tx=0).stop 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 msgr2=0x7f0d0419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 0x7f0d0419a870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000e9f0 tx=0x7f0cf000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 shutdown_connections 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0cec0a2d70 0x7f0cec0a5230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0410be60 0x7f0d0419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d041033a0 0x7f0d0419a870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 --2- 172.21.15.175:0/355691061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d04102a80 0x7f0d0419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:25.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 >> 172.21.15.175:0/355691061 conn(0x7f0d040fc480 msgr2=0x7f0d04100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:25.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 shutdown_connections 2024-07-29T00:40:25.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:25.337+0000 7f0d0b400640 1 -- 172.21.15.175:0/355691061 wait complete. 2024-07-29T00:40:25.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:25 smithi181 ceph-mon[36706]: pgmap v120: 97 pgs: 97 active+clean; 72 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 964 KiB/s wr, 45 op/s 2024-07-29T00:40:25.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:25 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2072764801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:25.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:25 smithi204 ceph-mon[37368]: pgmap v120: 97 pgs: 97 active+clean; 72 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 964 KiB/s wr, 45 op/s 2024-07-29T00:40:25.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:25 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2072764801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:25.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 16 sec latency 18736.591 ms 2024-07-29T00:40:26.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:40:26.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id 3b49758e --release --await' 2024-07-29T00:40:26.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db --set-id 3b49758e --release --await 2024-07-29T00:40:26.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 17 sec latency 19736.663 ms 2024-07-29T00:40:26.951 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:27.367 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:27 smithi175 ceph-mon[32138]: pgmap v121: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2024-07-29T00:40:27.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:27 smithi181 ceph-mon[36706]: pgmap v121: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2024-07-29T00:40:27.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:27 smithi204 ceph-mon[37368]: pgmap v121: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 -- 172.21.15.175:0/2979509814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f0102060 msgr2=0x7f92f010e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 --2- 172.21.15.175:0/2979509814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f0102060 0x7f92f010e590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009be0 tx=0x7f92e802a6c0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 -- 172.21.15.175:0/2979509814 shutdown_connections 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 --2- 172.21.15.175:0/2979509814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f0102060 0x7f92f010e590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 --2- 172.21.15.175:0/2979509814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92f01016c0 0x7f92f0101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 --2- 172.21.15.175:0/2979509814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f0107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 -- 172.21.15.175:0/2979509814 >> 172.21.15.175:0/2979509814 conn(0x7f92f00fd400 msgr2=0x7f92f00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 -- 172.21.15.175:0/2979509814 shutdown_connections 2024-07-29T00:40:27.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.646+0000 7f92f6a00640 1 -- 172.21.15.175:0/2979509814 wait complete. 2024-07-29T00:40:27.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.647+0000 7f92f6a00640 1 Processor -- start 2024-07-29T00:40:27.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.647+0000 7f92f6a00640 1 -- start start 2024-07-29T00:40:27.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f01016c0 0x7f92f019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92f0102060 0x7f92f019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0111ac0 con 0x7f92f01076c0 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0111940 con 0x7f92f01016c0 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0111c40 con 0x7f92f0102060 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33956/0 (socket says 172.21.15.175:33956) 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f6000640 1 -- 172.21.15.175:0/3407598166 learned_addr learned my addr 172.21.15.175:0/3407598166 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f01016c0 0x7f92f019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.648+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92f0102060 0x7f92f019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6000640 1 -- 172.21.15.175:0/3407598166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92f0102060 msgr2=0x7f92f019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6000640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92f0102060 0x7f92f019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6000640 1 -- 172.21.15.175:0/3407598166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f01016c0 msgr2=0x7f92f019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6000640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f01016c0 0x7f92f019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6000640 1 -- 172.21.15.175:0/3407598166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f019f130 con 0x7f92f01076c0 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f5600640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f01016c0 0x7f92f019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6000640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f019ea50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92e8034040 tx=0x7f92e8030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92ee000640 1 -- 172.21.15.175:0/3407598166 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8040020 con 0x7f92f01076c0 2024-07-29T00:40:27.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f019f3c0 con 0x7f92f01076c0 2024-07-29T00:40:27.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92ee000640 1 -- 172.21.15.175:0/3407598166 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e8030e80 con 0x7f92f01076c0 2024-07-29T00:40:27.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.649+0000 7f92ee000640 1 -- 172.21.15.175:0/3407598166 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8036040 con 0x7f92f01076c0 2024-07-29T00:40:27.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.650+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a6d00 con 0x7f92f01076c0 2024-07-29T00:40:27.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.653+0000 7f92ee000640 1 -- 172.21.15.175:0/3407598166 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f92e8043020 con 0x7f92f01076c0 2024-07-29T00:40:27.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.653+0000 7f92faa69740 1 Processor -- start 2024-07-29T00:40:27.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.653+0000 7f92faa69740 1 -- start start 2024-07-29T00:40:27.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92faa69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccae9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92faa69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccae9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92faa69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92faa69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5561cca4fab0 con 0x5561ccae9b70 2024-07-29T00:40:27.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92faa69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5561cc9e7a80 con 0x5561ccaf4910 2024-07-29T00:40:27.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92faa69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5561cca4ba70 con 0x5561ccaeb0e0 2024-07-29T00:40:27.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92f4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccae9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92f4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92f4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41420/0 (socket says 172.21.15.175:41420) 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92f4c00640 1 -- 172.21.15.175:0/614056351 learned_addr learned my addr 172.21.15.175:0/614056351 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.654+0000 7f92f5600640 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccae9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f4c00640 1 -- 172.21.15.175:0/614056351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 msgr2=0x5561ccae9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f4c00640 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccae9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f4c00640 1 -- 172.21.15.175:0/614056351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 msgr2=0x5561ccae9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f4c00640 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccae9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f4c00640 1 -- 172.21.15.175:0/614056351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cc7c13c0 con 0x5561ccaf4910 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f5600640 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccae9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92f4c00640 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf6d00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f92dc010970 tx=0x7f92dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f12525bdb71441e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92e2600640 1 -- 172.21.15.175:0/614056351 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc016070 con 0x5561ccaf4910 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92e2600640 1 -- 172.21.15.175:0/614056351 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc01e3b0 con 0x5561ccaf4910 2024-07-29T00:40:27.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92e2600640 1 -- 172.21.15.175:0/614056351 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc019040 con 0x5561ccaf4910 2024-07-29T00:40:27.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.655+0000 7f92ee000640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92d80a2cf0 0x7f92d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92ee000640 1 -- 172.21.15.175:0/3407598166 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f92e80f0090 con 0x7f92f01076c0 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92f5600640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92d80a2cf0 0x7f92d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 -- 172.21.15.175:0/614056351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 msgr2=0x5561ccaf6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf6d00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f92dc010970 tx=0x7f92dc005f90 comp rx=0 tx=0).stop 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 -- 172.21.15.175:0/614056351 shutdown_connections 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf6d00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccae9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 --2- 172.21.15.175:0/614056351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccae9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.656+0000 7f92faa69740 1 -- 172.21.15.175:0/614056351 >> 172.21.15.175:0/614056351 conn(0x5561ccab2950 msgr2=0x5561ccae6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.657+0000 7f92faa69740 1 -- 172.21.15.175:0/614056351 shutdown_connections 2024-07-29T00:40:27.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.657+0000 7f92faa69740 1 -- 172.21.15.175:0/614056351 wait complete. 2024-07-29T00:40:27.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.657+0000 7f92f5600640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92d80a2cf0 0x7f92d80a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5561ccae7060 tx=0x7f92e4009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:27.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.657+0000 7f92faa69740 1 -- start start 2024-07-29T00:40:27.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 Processor -- start 2024-07-29T00:40:27.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccaf1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccaf1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5561ccaf7700 con 0x5561ccae9b70 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5561cca0ac10 con 0x5561ccaf4910 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5561ccaf7880 con 0x5561ccaeb0e0 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.658+0000 7f92faa69740 10 client.0 _ll_register_callbacks cb 0x5561cc9d3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccaf1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccaf1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33974/0 (socket says 172.21.15.175:33974) 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f5600640 1 -- 172.21.15.175:0/3027156269 learned_addr learned my addr 172.21.15.175:0/3027156269 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccaf1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f4c00640 1 -- 172.21.15.175:0/3027156269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 msgr2=0x5561ccaf2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f4c00640 1 -- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 msgr2=0x5561ccaf1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccaf1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f4c00640 1 -- 172.21.15.175:0/3027156269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cca0e1e0 con 0x5561ccaeb0e0 2024-07-29T00:40:27.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f5600640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccaf1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:27.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.659+0000 7f92f6000640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:27.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccaf1dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f92dc001eb0 tx=0x7f92dc01ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336bc93e8523df9c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:27.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:27.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92d6a00640 1 -- 172.21.15.175:0/3027156269 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc016070 con 0x5561ccaeb0e0 2024-07-29T00:40:27.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92dc01e3b0 con 0x5561ccaeb0e0 2024-07-29T00:40:27.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92d6a00640 1 -- 172.21.15.175:0/3027156269 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f92dc00f790 con 0x5561ccaeb0e0 2024-07-29T00:40:27.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92d6a00640 1 -- 172.21.15.175:0/3027156269 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92dc00f990 con 0x5561ccaeb0e0 2024-07-29T00:40:27.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92faa69740 10 client.26170 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:27.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.660+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561ccb03470 con 0x5561ccaeb0e0 2024-07-29T00:40:27.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92d6a00640 1 -- 172.21.15.175:0/3027156269 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc01e3b0 con 0x5561ccaeb0e0 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92d6a00640 1 -- 172.21.15.175:0/3027156269 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f92dc0052e0 con 0x5561ccaeb0e0 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92faa69740 10 client.26170 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92faa69740 10 client.26170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92faa69740 20 client.26170 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92faa69740 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb04820 0x5561ccb06c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92faa69740 4 client.26170 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.661+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561cca5d020 con 0x5561ccb04820 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.662+0000 7f92f5600640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb04820 0x5561ccb06c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.662+0000 7f92f5600640 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb04820 0x5561ccb06c10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1479a4d1aa812fb6 server_cookie=9c604a05f9b8e3ae in_seq=0 out_seq=0 2024-07-29T00:40:27.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.662+0000 7f92d6a00640 10 client.26170 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:40:27.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.663+0000 7f92d6a00640 1 -- 172.21.15.175:0/3027156269 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561cca5d020 con 0x5561ccb04820 2024-07-29T00:40:27.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.663+0000 7f92d6a00640 10 client.26170 handle_command_reply: tid=0 2024-07-29T00:40:27.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.663+0000 7f92faa69740 1 client.26170 shutdown 2024-07-29T00:40:27.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 msgr2=0x5561ccaf1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccaf1dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f92dc001eb0 tx=0x7f92dc01ea40 comp rx=0 tx=0).stop 2024-07-29T00:40:27.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 shutdown_connections 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb04820 0x5561ccb06c10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccaf2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaeb0e0 0x5561ccaf1dd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 --2- 172.21.15.175:0/3027156269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccae9b70 0x5561ccaf1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 >> 172.21.15.175:0/3027156269 conn(0x5561ccab2950 msgr2=0x5561ccaebf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92d6a00640 0 client.26170 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 shutdown_connections 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 1 -- 172.21.15.175:0/3027156269 wait complete. 2024-07-29T00:40:27.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.664+0000 7f92faa69740 20 client.26170 trim_cache size 0 max 16384 2024-07-29T00:40:27.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.679+0000 7f92faa69740 1 Processor -- start 2024-07-29T00:40:27.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 -- start start 2024-07-29T00:40:27.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 0x5561ccb00220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccaeeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaf4910 0x5561ccaef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01af780 con 0x5561ccaeb0e0 2024-07-29T00:40:27.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b1380 con 0x5561ccae9b70 2024-07-29T00:40:27.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.680+0000 7f92faa69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01afe50 con 0x5561ccaf4910 2024-07-29T00:40:27.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 0x5561ccb00220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 0x5561ccb00220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41428/0 (socket says 172.21.15.175:41428) 2024-07-29T00:40:27.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 -- 172.21.15.175:0/3566543279 learned_addr learned my addr 172.21.15.175:0/3566543279 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:27.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f6000640 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaf4910 0x5561ccaef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccaeeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 -- 172.21.15.175:0/3566543279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaf4910 msgr2=0x5561ccaef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaf4910 0x5561ccaef070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 -- 172.21.15.175:0/3566543279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 msgr2=0x5561ccaeeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccaeeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f5600640 1 -- 172.21.15.175:0/3566543279 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ccaec6c0 con 0x5561ccae9b70 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.681+0000 7f92f4c00640 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccaeeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.682+0000 7f92f5600640 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 0x5561ccb00220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92e4005fd0 tx=0x7f92e4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13d484e77b98320 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.682+0000 7f92e1c00640 1 -- 172.21.15.175:0/3566543279 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400fe10 con 0x5561ccae9b70 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.682+0000 7f92e1c00640 1 -- 172.21.15.175:0/3566543279 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e4005020 con 0x5561ccae9b70 2024-07-29T00:40:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.682+0000 7f92e1c00640 1 -- 172.21.15.175:0/3566543279 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4004b20 con 0x5561ccae9b70 2024-07-29T00:40:27.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 -- 172.21.15.175:0/3566543279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 msgr2=0x5561ccb00220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 0x5561ccb00220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92e4005fd0 tx=0x7f92e4018000 comp rx=0 tx=0).stop 2024-07-29T00:40:27.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 -- 172.21.15.175:0/3566543279 shutdown_connections 2024-07-29T00:40:27.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccaf4910 0x5561ccaef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccaeeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 --2- 172.21.15.175:0/3566543279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccae9b70 0x5561ccb00220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 -- 172.21.15.175:0/3566543279 >> 172.21.15.175:0/3566543279 conn(0x5561ccab2950 msgr2=0x5561ccaffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 -- 172.21.15.175:0/3566543279 shutdown_connections 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.683+0000 7f92faa69740 1 -- 172.21.15.175:0/3566543279 wait complete. 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.684+0000 7f92faa69740 1 -- start start 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.684+0000 7f92faa69740 1 Processor -- start 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.684+0000 7f92faa69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccae9b70 0x5561ccb06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92faa69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccb06d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92faa69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccb072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92faa69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b0bf0 con 0x5561ccaeb0e0 2024-07-29T00:40:27.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92faa69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b0520 con 0x5561ccaf4910 2024-07-29T00:40:27.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92faa69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ad760 con 0x5561ccae9b70 2024-07-29T00:40:27.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92faa69740 10 client.0 _ll_register_callbacks cb 0x7f92dc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:27.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92f5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccae9b70 0x5561ccb06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.685+0000 7f92f4c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccb06d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccae9b70 0x5561ccb06830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:35414/0 (socket says 172.21.15.175:35414) 2024-07-29T00:40:27.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f5600640 1 -- 172.21.15.175:0/299932050 learned_addr learned my addr 172.21.15.175:0/299932050 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccb072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 -- 172.21.15.175:0/299932050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccae9b70 msgr2=0x5561ccb06830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccae9b70 0x5561ccb06830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 -- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 msgr2=0x5561ccb06d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccb06d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 -- 172.21.15.175:0/299932050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ccb6d1d0 con 0x5561ccaf4910 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f4c00640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccb06d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92f6000640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccb072b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92e80ed760 tx=0x7f92e80bc130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f7f16e7839ad7b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.686+0000 7f92d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92d6a00640 1 -- 172.21.15.175:0/299932050 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80da060 con 0x5561ccaf4910 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92d6a00640 1 -- 172.21.15.175:0/299932050 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f92e80c4f10 con 0x5561ccaf4910 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e4005020 con 0x5561ccaf4910 2024-07-29T00:40:27.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92d6a00640 1 -- 172.21.15.175:0/299932050 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92e80a2050 con 0x5561ccaf4910 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 10 client.26219 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561ccb6e510 con 0x5561ccaf4910 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92d6a00640 1 -- 172.21.15.175:0/299932050 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e4005020 con 0x5561ccaf4910 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92d6a00640 1 -- 172.21.15.175:0/299932050 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f92e8004b40 con 0x5561ccaf4910 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 10 client.26219 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 10 client.26219 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 20 client.26219 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb6ec30 0x5561ccb71020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 4 client.26219 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:40:27.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.687+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561cca57530 con 0x5561ccb6ec30 2024-07-29T00:40:27.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.688+0000 7f92f5600640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb6ec30 0x5561ccb71020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:27.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.688+0000 7f92f5600640 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb6ec30 0x5561ccb71020 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4b05e547b434875 server_cookie=725040669c28cea1 in_seq=0 out_seq=0 2024-07-29T00:40:27.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.688+0000 7f92d6a00640 10 client.26219 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92d6a00640 1 -- 172.21.15.175:0/299932050 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561cca57530 con 0x5561ccb6ec30 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92d6a00640 10 client.26219 handle_command_reply: tid=0 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 client.26219 shutdown 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 msgr2=0x5561ccb072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccb072b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92e80ed760 tx=0x7f92e80bc130 comp rx=0 tx=0).stop 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 shutdown_connections 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5561ccb6ec30 0x5561ccb71020 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561ccaf4910 0x5561ccb072b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561ccaeb0e0 0x5561ccb06d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 --2- 172.21.15.175:0/299932050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561ccae9b70 0x5561ccb06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 >> 172.21.15.175:0/299932050 conn(0x5561ccab2950 msgr2=0x5561ccae5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.689+0000 7f92d6a00640 0 client.26219 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:40:27.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 shutdown_connections 2024-07-29T00:40:27.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92faa69740 1 -- 172.21.15.175:0/299932050 wait complete. 2024-07-29T00:40:27.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92faa69740 20 client.26219 trim_cache size 0 max 16384 2024-07-29T00:40:27.692 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92d80a2cf0 msgr2=0x7f92d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92f6a00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92d80a2cf0 0x7f92d80a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5561ccae7060 tx=0x7f92e4009260 comp rx=0 tx=0).stop 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 msgr2=0x7f92f019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.690+0000 7f92f6a00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f019ea50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92e8034040 tx=0x7f92e8030c50 comp rx=0 tx=0).stop 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 shutdown_connections 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92d80a2cf0 0x7f92d80a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f01076c0 0x7f92f019ea50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92f0102060 0x7f92f019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 --2- 172.21.15.175:0/3407598166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92f01016c0 0x7f92f019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 >> 172.21.15.175:0/3407598166 conn(0x7f92f00fd400 msgr2=0x7f92f010cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 shutdown_connections 2024-07-29T00:40:27.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:27.691+0000 7f92f6a00640 1 -- 172.21.15.175:0/3407598166 wait complete. 2024-07-29T00:40:27.712 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:27.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 18 sec latency 20736.770 ms 2024-07-29T00:40:28.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:40:28.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103633 0.00 MB/sec execute 19 sec latency 21736.844 ms 2024-07-29T00:40:29.459 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:29 smithi175 ceph-mon[32138]: pgmap v122: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-07-29T00:40:29.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:29 smithi181 ceph-mon[36706]: pgmap v122: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-07-29T00:40:29.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:29 smithi204 ceph-mon[37368]: pgmap v122: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-07-29T00:40:29.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 -- 172.21.15.175:0/2249059010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 msgr2=0x7fd94c1121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 --2- 172.21.15.175:0/2249059010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 0x7fd94c1121e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd938009900 tx=0x7fd93802a3f0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 -- 172.21.15.175:0/2249059010 shutdown_connections 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 --2- 172.21.15.175:0/2249059010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 0x7fd94c114b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 --2- 172.21.15.175:0/2249059010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 0x7fd94c1121e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 --2- 172.21.15.175:0/2249059010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 -- 172.21.15.175:0/2249059010 >> 172.21.15.175:0/2249059010 conn(0x7fd94c0ff780 msgr2=0x7fd94c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 -- 172.21.15.175:0/2249059010 shutdown_connections 2024-07-29T00:40:29.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.753+0000 7fd951200640 1 -- 172.21.15.175:0/2249059010 wait complete. 2024-07-29T00:40:29.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.754+0000 7fd951200640 1 Processor -- start 2024-07-29T00:40:29.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 -- start start 2024-07-29T00:40:29.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 0x7fd94c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 0x7fd94c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1182f0 con 0x7fd94c076d30 2024-07-29T00:40:29.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c118170 con 0x7fd94c076410 2024-07-29T00:40:29.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.755+0000 7fd951200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c118470 con 0x7fd94c112720 2024-07-29T00:40:29.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c1a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56266/0 (socket says 172.21.15.175:56266) 2024-07-29T00:40:29.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 0x7fd94c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 -- 172.21.15.175:0/3430508934 learned_addr learned my addr 172.21.15.175:0/3430508934 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd94a800640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 0x7fd94c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 -- 172.21.15.175:0/3430508934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 msgr2=0x7fd94c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 0x7fd94c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 -- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 msgr2=0x7fd94c1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 0x7fd94c1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 -- 172.21.15.175:0/3430508934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c19ece0 con 0x7fd94c076410 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd94a800640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 0x7fd94c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:29.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.756+0000 7fd949e00640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c1a43f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9f0 tx=0x7fd93c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:29.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.757+0000 7fd942a00640 1 -- 172.21.15.175:0/3430508934 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00cd80 con 0x7fd94c076410 2024-07-29T00:40:29.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.757+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c19efd0 con 0x7fd94c076410 2024-07-29T00:40:29.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.757+0000 7fd942a00640 1 -- 172.21.15.175:0/3430508934 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c010910 con 0x7fd94c076410 2024-07-29T00:40:29.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.757+0000 7fd942a00640 1 -- 172.21.15.175:0/3430508934 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c018990 con 0x7fd94c076410 2024-07-29T00:40:29.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.757+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1ab2c0 con 0x7fd94c076410 2024-07-29T00:40:29.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.760+0000 7fd942a00640 1 -- 172.21.15.175:0/3430508934 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd93c010ac0 con 0x7fd94c076410 2024-07-29T00:40:29.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.761+0000 7fd953c8b740 1 Processor -- start 2024-07-29T00:40:29.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.761+0000 7fd953c8b740 1 -- start start 2024-07-29T00:40:29.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.761+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc1f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd953c8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b9db7acc0 con 0x558b9dc1fde0 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd953c8b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b9db12bc0 con 0x558b9dc15520 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd953c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b9db76c80 con 0x558b9dc14c00 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd949e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc1f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc14fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55352/0 (socket says 172.21.15.175:55352) 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 -- 172.21.15.175:0/2537181529 learned_addr learned my addr 172.21.15.175:0/2537181529 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd949400640 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 -- 172.21.15.175:0/2537181529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 msgr2=0x558b9dc1f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc1f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd942a00640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd9300a2cf0 0x7fd9300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 -- 172.21.15.175:0/2537181529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 msgr2=0x558b9dc221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.762+0000 7fd94a800640 1 -- 172.21.15.175:0/2537181529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9d88a580 con 0x558b9dc14c00 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.763+0000 7fd942a00640 1 -- 172.21.15.175:0/3430508934 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd93c0cf890 con 0x7fd94c076410 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.763+0000 7fd94a800640 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc14fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd944009f80 tx=0x7fd944010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31af1ddb78e6aa36 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.763+0000 7fd94a800640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd9300a2cf0 0x7fd9300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.763+0000 7fd937400640 1 -- 172.21.15.175:0/2537181529 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944011760 con 0x558b9dc14c00 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.763+0000 7fd937400640 1 -- 172.21.15.175:0/2537181529 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd944011de0 con 0x558b9dc14c00 2024-07-29T00:40:29.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.763+0000 7fd937400640 1 -- 172.21.15.175:0/2537181529 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94401a5e0 con 0x558b9dc14c00 2024-07-29T00:40:29.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 -- 172.21.15.175:0/2537181529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 msgr2=0x558b9dc14fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd94a800640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd9300a2cf0 0x7fd9300a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd94401cfd0 tx=0x7fd94401cec0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:29.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc14fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd944009f80 tx=0x7fd944010860 comp rx=0 tx=0).stop 2024-07-29T00:40:29.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 -- 172.21.15.175:0/2537181529 shutdown_connections 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc1f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2537181529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc14fe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.764+0000 7fd953c8b740 1 -- 172.21.15.175:0/2537181529 >> 172.21.15.175:0/2537181529 conn(0x558b9dc11190 msgr2=0x558b9dc11570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.765+0000 7fd953c8b740 1 -- 172.21.15.175:0/2537181529 shutdown_connections 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.765+0000 7fd953c8b740 1 -- 172.21.15.175:0/2537181529 wait complete. 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.765+0000 7fd953c8b740 1 -- start start 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.765+0000 7fd953c8b740 1 Processor -- start 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 0x558b9dc1cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc1d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b9dc229b0 con 0x558b9dc1fde0 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b9db35ca0 con 0x558b9dc14c00 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b9dc22b30 con 0x558b9dc15520 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd953c8b740 10 client.0 _ll_register_callbacks cb 0x558b9db21e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd949e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 0x558b9dc1cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd949e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 0x558b9dc1cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56284/0 (socket says 172.21.15.175:56284) 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd949e00640 1 -- 172.21.15.175:0/2906086995 learned_addr learned my addr 172.21.15.175:0/2906086995 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd94a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.766+0000 7fd949400640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc1d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd949e00640 1 -- 172.21.15.175:0/2906086995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 msgr2=0x558b9dc1d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd949e00640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc1d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd949e00640 1 -- 172.21.15.175:0/2906086995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 msgr2=0x558b9dc1d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd949e00640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd949e00640 1 -- 172.21.15.175:0/2906086995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9d954960 con 0x558b9dc14c00 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd94a800640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd949e00640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 0x558b9dc1cd40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0ccfc0 tx=0x7fd93c09bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3351292d5dc110a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd92be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.767+0000 7fd92be00640 1 -- 172.21.15.175:0/2906086995 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0cd3f0 con 0x558b9dc14c00 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.768+0000 7fd92be00640 1 -- 172.21.15.175:0/2906086995 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd93c09b180 con 0x558b9dc14c00 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.768+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd944011de0 con 0x558b9dc14c00 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.768+0000 7fd92be00640 1 -- 172.21.15.175:0/2906086995 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd93c09bd30 con 0x558b9dc14c00 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.768+0000 7fd953c8b740 10 client.26231 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:29.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.768+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b9dc2e6e0 con 0x558b9dc14c00 2024-07-29T00:40:29.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.768+0000 7fd92be00640 1 -- 172.21.15.175:0/2906086995 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd944011de0 con 0x558b9dc14c00 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd92be00640 1 -- 172.21.15.175:0/2906086995 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd93c097b10 con 0x558b9dc14c00 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd953c8b740 10 client.26231 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd953c8b740 10 client.26231 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd953c8b740 20 client.26231 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dc2fc40 0x558b9dc320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd953c8b740 4 client.26231 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b9db88230 con 0x558b9dc2fc40 2024-07-29T00:40:29.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.769+0000 7fd949400640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dc2fc40 0x558b9dc320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.770+0000 7fd949400640 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dc2fc40 0x558b9dc320a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df8adafcac49b74 server_cookie=6574bae30f61deee in_seq=0 out_seq=0 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.770+0000 7fd92be00640 10 client.26231 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd92be00640 1 -- 172.21.15.175:0/2906086995 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b9db88230 con 0x558b9dc2fc40 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd92be00640 10 client.26231 handle_command_reply: tid=0 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 client.26231 shutdown 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 msgr2=0x558b9dc1cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 0x558b9dc1cd40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0ccfc0 tx=0x7fd93c09bbc0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 shutdown_connections 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dc2fc40 0x558b9dc320a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc1d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd92be00640 0 client.26231 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:29.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 --2- 172.21.15.175:0/2906086995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc14c00 0x558b9dc1cd40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 >> 172.21.15.175:0/2906086995 conn(0x558b9dc11190 msgr2=0x558b9dc16ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:29.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 shutdown_connections 2024-07-29T00:40:29.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.771+0000 7fd953c8b740 1 -- 172.21.15.175:0/2906086995 wait complete. 2024-07-29T00:40:29.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.772+0000 7fd953c8b740 20 client.26231 trim_cache size 0 max 16384 2024-07-29T00:40:29.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 Processor -- start 2024-07-29T00:40:29.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 -- start start 2024-07-29T00:40:29.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 0x558b9dc176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b3d00 con 0x558b9dc14c00 2024-07-29T00:40:29.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b5900 con 0x558b9dc1fde0 2024-07-29T00:40:29.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd953c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b43d0 con 0x558b9dc15520 2024-07-29T00:40:29.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd949e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 0x558b9dc176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd949400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd949e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 0x558b9dc176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38940/0 (socket says 172.21.15.175:38940) 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.799+0000 7fd949e00640 1 -- 172.21.15.175:0/4020330713 learned_addr learned my addr 172.21.15.175:0/4020330713 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd94a800640 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949400640 1 -- 172.21.15.175:0/4020330713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc1fde0 msgr2=0x558b9dc1d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949400640 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949400640 1 -- 172.21.15.175:0/4020330713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 msgr2=0x558b9dc176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949400640 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 0x558b9dc176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949400640 1 -- 172.21.15.175:0/4020330713 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9dc1ec80 con 0x558b9dc15520 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949e00640 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 0x558b9dc176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.800+0000 7fd949400640 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc182f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd93802a9e0 tx=0x7fd938005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496fd9f9f6e14231 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd936a00640 1 -- 172.21.15.175:0/4020330713 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938035040 con 0x558b9dc15520 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd936a00640 1 -- 172.21.15.175:0/4020330713 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93804b070 con 0x558b9dc15520 2024-07-29T00:40:29.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd936a00640 1 -- 172.21.15.175:0/4020330713 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380315b0 con 0x558b9dc15520 2024-07-29T00:40:29.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd953c8b740 1 -- 172.21.15.175:0/4020330713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 msgr2=0x558b9dc182f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd953c8b740 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc182f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd93802a9e0 tx=0x7fd938005b50 comp rx=0 tx=0).stop 2024-07-29T00:40:29.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd953c8b740 1 -- 172.21.15.175:0/4020330713 shutdown_connections 2024-07-29T00:40:29.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd953c8b740 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc1fde0 0x558b9dc1d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.801+0000 7fd953c8b740 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc15520 0x558b9dc182f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.802+0000 7fd953c8b740 1 --2- 172.21.15.175:0/4020330713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc14c00 0x558b9dc176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.802+0000 7fd953c8b740 1 -- 172.21.15.175:0/4020330713 >> 172.21.15.175:0/4020330713 conn(0x558b9dc11190 msgr2=0x558b9dc1cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:29.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.802+0000 7fd953c8b740 1 -- 172.21.15.175:0/4020330713 shutdown_connections 2024-07-29T00:40:29.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.802+0000 7fd953c8b740 1 -- 172.21.15.175:0/4020330713 wait complete. 2024-07-29T00:40:29.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.802+0000 7fd953c8b740 1 -- start start 2024-07-29T00:40:29.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.802+0000 7fd953c8b740 1 Processor -- start 2024-07-29T00:40:29.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc27d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc24d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b5170 con 0x558b9dc1fde0 2024-07-29T00:40:29.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b4aa0 con 0x558b9dc15520 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b1ce0 con 0x558b9dc14c00 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd953c8b740 10 client.0 _ll_register_callbacks cb 0x7fd93c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd949e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc27d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd949e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc27d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55378/0 (socket says 172.21.15.175:55378) 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.803+0000 7fd949e00640 1 -- 172.21.15.175:0/1262434654 learned_addr learned my addr 172.21.15.175:0/1262434654 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949400640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd94a800640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc24d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949e00640 1 -- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 msgr2=0x558b9dc282c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949e00640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc282c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949e00640 1 -- 172.21.15.175:0/1262434654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 msgr2=0x558b9dc24d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:29.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949e00640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc24d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949e00640 1 -- 172.21.15.175:0/1262434654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9dc1ac50 con 0x558b9dc14c00 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.804+0000 7fd949e00640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc27d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0d5610 tx=0x7fd93c09b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72bfb757463e83be server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd92be00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd92be00640 1 -- 172.21.15.175:0/1262434654 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c09b4a0 con 0x558b9dc14c00 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd92be00640 1 -- 172.21.15.175:0/1262434654 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd93c018b30 con 0x558b9dc14c00 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd93804b070 con 0x558b9dc14c00 2024-07-29T00:40:29.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd92be00640 1 -- 172.21.15.175:0/1262434654 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd93c0a49a0 con 0x558b9dc14c00 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 10 client.26194 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b9dcabe80 con 0x558b9dc14c00 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd92be00640 1 -- 172.21.15.175:0/1262434654 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93804b070 con 0x558b9dc14c00 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd92be00640 1 -- 172.21.15.175:0/1262434654 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd93c0a1400 con 0x558b9dc14c00 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 10 client.26194 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 10 client.26194 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 20 client.26194 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dcac660 0x558b9dcaea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 4 client.26194 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3b49758e", "release": true, "await": true}] 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.805+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3b49758e", "release": true, "await": true}) -- 0x558b9db82740 con 0x558b9dcac660 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.806+0000 7fd949400640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dcac660 0x558b9dcaea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.806+0000 7fd949400640 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dcac660 0x558b9dcaea80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffbc7db760f772f server_cookie=643c40974e799fb5 in_seq=0 out_seq=0 2024-07-29T00:40:29.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.806+0000 7fd92be00640 10 client.26194 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:29.815 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:29.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.816+0000 7fd92be00640 1 -- 172.21.15.175:0/1262434654 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558b9db82740 con 0x558b9dcac660 2024-07-29T00:40:29.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.816+0000 7fd92be00640 10 client.26194 handle_command_reply: tid=0 2024-07-29T00:40:29.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.816+0000 7fd953c8b740 1 client.26194 shutdown 2024-07-29T00:40:29.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.816+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 msgr2=0x558b9dc27d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.816+0000 7fd953c8b740 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc27d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0d5610 tx=0x7fd93c09b3b0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 shutdown_connections 2024-07-29T00:40:29.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b9dcac660 0x558b9dcaea80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b9dc1fde0 0x558b9dc24d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b9dc15520 0x558b9dc282c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 --2- 172.21.15.175:0/1262434654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b9dc14c00 0x558b9dc27d80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 >> 172.21.15.175:0/1262434654 conn(0x558b9dc11190 msgr2=0x558b9dc1d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd92be00640 0 client.26194 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 shutdown_connections 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 1 -- 172.21.15.175:0/1262434654 wait complete. 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.817+0000 7fd953c8b740 20 client.26194 trim_cache size 0 max 16384 2024-07-29T00:40:29.818 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 31, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 12, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "3b49758e": { 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 12, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 139.8, 2024-07-29T00:40:29.819 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:29.820 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.818+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd9300a2cf0 msgr2=0x7fd9300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.818+0000 7fd951200640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd9300a2cf0 0x7fd9300a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd94401cfd0 tx=0x7fd94401cec0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.818+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 msgr2=0x7fd94c1a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.818+0000 7fd951200640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c1a43f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9f0 tx=0x7fd93c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 shutdown_connections 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd94c112720 0x7fd94c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd9300a2cf0 0x7fd9300a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd94c076d30 0x7fd94c1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 --2- 172.21.15.175:0/3430508934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94c076410 0x7fd94c1a43f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 >> 172.21.15.175:0/3430508934 conn(0x7fd94c0ff780 msgr2=0x7fd94c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 shutdown_connections 2024-07-29T00:40:29.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:29.819+0000 7fd951200640 1 -- 172.21.15.175:0/3430508934 wait complete. 2024-07-29T00:40:29.880 INFO:tasks.workunit.client.0.smithi175.stdout: 1 103699 0.01 MB/sec execute 20 sec latency 22674.331 ms 2024-07-29T00:40:30.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 105493 0.37 MB/sec execute 21 sec latency 13.828 ms 2024-07-29T00:40:31.054 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3b49758e', total seconds elapsed: 28.9 2024-07-29T00:40:31.054 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.8 2024-07-29T00:40:31.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:40:31.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:31 smithi175 ceph-mon[32138]: pgmap v123: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 -- 172.21.15.175:0/2520471041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 msgr2=0x7f8850110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 --2- 172.21.15.175:0/2520471041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f8850110430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8848009ca0 tx=0x7f884802a8b0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 -- 172.21.15.175:0/2520471041 shutdown_connections 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 --2- 172.21.15.175:0/2520471041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f8850110430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 --2- 172.21.15.175:0/2520471041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 0x7f885010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 --2- 172.21.15.175:0/2520471041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f8850100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 -- 172.21.15.175:0/2520471041 >> 172.21.15.175:0/2520471041 conn(0x7f88500fc300 msgr2=0x7f88500fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 -- 172.21.15.175:0/2520471041 shutdown_connections 2024-07-29T00:40:31.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.303+0000 7f8855c00640 1 -- 172.21.15.175:0/2520471041 wait complete. 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.304+0000 7f8855c00640 1 Processor -- start 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 -- start start 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f885019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 0x7f885019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f885019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113cc0 con 0x7f8850100490 2024-07-29T00:40:31.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113b40 con 0x7f8850100e40 2024-07-29T00:40:31.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113e40 con 0x7f885010e040 2024-07-29T00:40:31.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f884fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f885019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f8855200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f885019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f884fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f885019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38964/0 (socket says 172.21.15.175:38964) 2024-07-29T00:40:31.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f884fe00640 1 -- 172.21.15.175:0/1468827214 learned_addr learned my addr 172.21.15.175:0/1468827214 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:31.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.305+0000 7f884f400640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 0x7f885019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884fe00640 1 -- 172.21.15.175:0/1468827214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 msgr2=0x7f885019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884fe00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f885019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884fe00640 1 -- 172.21.15.175:0/1468827214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 msgr2=0x7f885019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884fe00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 0x7f885019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884fe00640 1 -- 172.21.15.175:0/1468827214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885019f110 con 0x7f8850100490 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f8855200640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f885019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884f400640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 0x7f885019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884fe00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f885019a160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f884000e9f0 tx=0x7f884000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f884cc00640 1 -- 172.21.15.175:0/1468827214 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884000cd80 con 0x7f8850100490 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.306+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885019f400 con 0x7f8850100490 2024-07-29T00:40:31.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.307+0000 7f884cc00640 1 -- 172.21.15.175:0/1468827214 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8840010910 con 0x7f8850100490 2024-07-29T00:40:31.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.307+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501a6ce0 con 0x7f8850100490 2024-07-29T00:40:31.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.307+0000 7f884cc00640 1 -- 172.21.15.175:0/1468827214 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840018910 con 0x7f8850100490 2024-07-29T00:40:31.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.310+0000 7f884cc00640 1 -- 172.21.15.175:0/1468827214 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8840010ac0 con 0x7f8850100490 2024-07-29T00:40:31.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.310+0000 7f8859e13740 1 Processor -- start 2024-07-29T00:40:31.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 -- start start 2024-07-29T00:40:31.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb6095d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb6095dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb60968910 0x55fb6096ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb608c3ab0 con 0x55fb6095db70 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb6085ba80 con 0x55fb6095f0e0 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8859e13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb608bfa70 con 0x55fb60968910 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8855200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb6095d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8855200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb6095d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56318/0 (socket says 172.21.15.175:56318) 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f8855200640 1 -- 172.21.15.175:0/788258014 learned_addr learned my addr 172.21.15.175:0/788258014 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.311+0000 7f884f400640 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb60968910 0x55fb6096ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f884fe00640 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb6095dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f884cc00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88340a2cf0 0x7f88340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f8855200640 1 -- 172.21.15.175:0/788258014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb60968910 msgr2=0x55fb6096ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f8855200640 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb60968910 0x55fb6096ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f8855200640 1 -- 172.21.15.175:0/788258014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 msgr2=0x55fb6095dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f8855200640 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb6095dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f8855200640 1 -- 172.21.15.175:0/788258014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb606353c0 con 0x55fb6095f0e0 2024-07-29T00:40:31.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.312+0000 7f884cc00640 1 -- 172.21.15.175:0/1468827214 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f88400cf900 con 0x7f8850100490 2024-07-29T00:40:31.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.313+0000 7f884fe00640 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb6095dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:31.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.313+0000 7f8855200640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88340a2cf0 0x7f88340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.313+0000 7f8855200640 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb6095d630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8848035000 tx=0x7f8848005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867b4f81f956d711 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:31.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.314+0000 7f8845600640 1 -- 172.21.15.175:0/788258014 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848003b70 con 0x55fb6095f0e0 2024-07-29T00:40:31.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.314+0000 7f8845600640 1 -- 172.21.15.175:0/788258014 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848036040 con 0x55fb6095f0e0 2024-07-29T00:40:31.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.314+0000 7f8845600640 1 -- 172.21.15.175:0/788258014 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848031680 con 0x55fb6095f0e0 2024-07-29T00:40:31.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 -- 172.21.15.175:0/788258014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 msgr2=0x55fb6095d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb6095d630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8848035000 tx=0x7f8848005ef0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8855200640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88340a2cf0 0x7f88340a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f884804c000 tx=0x7f884804d000 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:31.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 -- 172.21.15.175:0/788258014 shutdown_connections 2024-07-29T00:40:31.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb60968910 0x55fb6096ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb6095dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 --2- 172.21.15.175:0/788258014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb6095d630 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 -- 172.21.15.175:0/788258014 >> 172.21.15.175:0/788258014 conn(0x55fb60926950 msgr2=0x55fb6095a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:31.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 -- 172.21.15.175:0/788258014 shutdown_connections 2024-07-29T00:40:31.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 -- 172.21.15.175:0/788258014 wait complete. 2024-07-29T00:40:31.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.315+0000 7f8859e13740 1 -- start start 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 Processor -- start 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb609658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60965df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60966330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb6096b6b0 con 0x55fb60968910 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb6087ec10 con 0x55fb6095f0e0 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb6096b830 con 0x55fb6095db70 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f8859e13740 10 client.0 _ll_register_callbacks cb 0x55fb60847a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f884fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb609658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f884fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb609658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55412/0 (socket says 172.21.15.175:55412) 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.316+0000 7f884fe00640 1 -- 172.21.15.175:0/904624353 learned_addr learned my addr 172.21.15.175:0/904624353 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884f400640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60965df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f8855200640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60966330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884fe00640 1 -- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 msgr2=0x55fb60965df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884fe00640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60965df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884fe00640 1 -- 172.21.15.175:0/904624353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 msgr2=0x55fb60966330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884fe00640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60966330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884fe00640 1 -- 172.21.15.175:0/904624353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb608821e0 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884f400640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60965df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f8855200640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60966330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f884fe00640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb609658b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f884000e9c0 tx=0x7f8840098100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f471ce06e57a951 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.317+0000 7f883a000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f883a000640 1 -- 172.21.15.175:0/904624353 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884009f020 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f883a000640 1 -- 172.21.15.175:0/904624353 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f884009b1f0 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848036040 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f883a000640 1 -- 172.21.15.175:0/904624353 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88400cd450 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f8859e13740 10 client.26206 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fb60977490 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f883a000640 1 -- 172.21.15.175:0/904624353 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848036040 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f883a000640 1 -- 172.21.15.175:0/904624353 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8840097cf0 con 0x55fb6095db70 2024-07-29T00:40:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f8859e13740 10 client.26206 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.318+0000 7f8859e13740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f8859e13740 20 client.26206 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f8859e13740 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb60978940 0x55fb6097ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f8859e13740 4 client.26206 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb608d1020 con 0x55fb60978940 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f884f400640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb60978940 0x55fb6097ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f884f400640 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb60978940 0x55fb6097ad50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21330d59b8974db9 server_cookie=fbf16a08dd0f162b in_seq=0 out_seq=0 2024-07-29T00:40:31.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.319+0000 7f883a000640 10 client.26206 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f883a000640 1 -- 172.21.15.175:0/904624353 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb608d1020 con 0x55fb60978940 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f883a000640 10 client.26206 handle_command_reply: tid=0 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 client.26206 shutdown 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 msgr2=0x55fb609658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb609658b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f884000e9c0 tx=0x7f8840098100 comp rx=0 tx=0).stop 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 shutdown_connections 2024-07-29T00:40:31.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb60978940 0x55fb6097ad50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60966330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60965df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 --2- 172.21.15.175:0/904624353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb609658b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f883a000640 0 client.26206 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.320+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 >> 172.21.15.175:0/904624353 conn(0x55fb60926950 msgr2=0x55fb6095fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.321+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 shutdown_connections 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.321+0000 7f8859e13740 1 -- 172.21.15.175:0/904624353 wait complete. 2024-07-29T00:40:31.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.321+0000 7f8859e13740 20 client.26206 trim_cache size 0 max 16384 2024-07-29T00:40:31.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.337+0000 7f8859e13740 1 Processor -- start 2024-07-29T00:40:31.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.337+0000 7f8859e13740 1 -- start start 2024-07-29T00:40:31.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f8859e13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb60980660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f8859e13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60980ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f8859e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60958b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f8859e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88501af7a0 con 0x55fb60968910 2024-07-29T00:40:31.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f8859e13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88501b13a0 con 0x55fb6095f0e0 2024-07-29T00:40:31.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f8859e13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88501afe70 con 0x55fb6095db70 2024-07-29T00:40:31.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f884fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb60980660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f884fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb60980660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55416/0 (socket says 172.21.15.175:55416) 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.338+0000 7f884fe00640 1 -- 172.21.15.175:0/2438605045 learned_addr learned my addr 172.21.15.175:0/2438605045 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884f400640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60980ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884fe00640 1 -- 172.21.15.175:0/2438605045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 msgr2=0x55fb60980ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884fe00640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60980ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f8855200640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60958b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884fe00640 1 -- 172.21.15.175:0/2438605045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 msgr2=0x55fb60958b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884fe00640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60958b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884fe00640 1 -- 172.21.15.175:0/2438605045 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6097b3d0 con 0x55fb6095db70 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884f400640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60980ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f8855200640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60958b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.339+0000 7f884fe00640 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb60980660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8840098200 tx=0x7f884009a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be60cfff926ef079 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:31.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.340+0000 7f8844c00640 1 -- 172.21.15.175:0/2438605045 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88400cd450 con 0x55fb6095db70 2024-07-29T00:40:31.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.340+0000 7f8844c00640 1 -- 172.21.15.175:0/2438605045 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884009b1f0 con 0x55fb6095db70 2024-07-29T00:40:31.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.340+0000 7f8844c00640 1 -- 172.21.15.175:0/2438605045 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884009afa0 con 0x55fb6095db70 2024-07-29T00:40:31.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.340+0000 7f8859e13740 1 -- 172.21.15.175:0/2438605045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 msgr2=0x55fb60980660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.340+0000 7f8859e13740 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb60980660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8840098200 tx=0x7f884009a2b0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 -- 172.21.15.175:0/2438605045 shutdown_connections 2024-07-29T00:40:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb60968910 0x55fb60958b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb6095f0e0 0x55fb60980ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 --2- 172.21.15.175:0/2438605045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095db70 0x55fb60980660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 -- 172.21.15.175:0/2438605045 >> 172.21.15.175:0/2438605045 conn(0x55fb60926950 msgr2=0x55fb60955070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 -- 172.21.15.175:0/2438605045 shutdown_connections 2024-07-29T00:40:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.341+0000 7f8859e13740 1 -- 172.21.15.175:0/2438605045 wait complete. 2024-07-29T00:40:31.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 -- start start 2024-07-29T00:40:31.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 Processor -- start 2024-07-29T00:40:31.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb60970f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 0x55fb6096c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb60968910 0x55fb6096c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88501b0c10 con 0x55fb6095db70 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88501b0540 con 0x55fb60968910 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.342+0000 7f8859e13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ad780 con 0x55fb6095f0e0 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f8859e13740 10 client.0 _ll_register_callbacks cb 0x7f884000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 0x55fb6096c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb60970f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 0x55fb6096c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55430/0 (socket says 172.21.15.175:55430) 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 -- 172.21.15.175:0/2577037444 learned_addr learned my addr 172.21.15.175:0/2577037444 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f8855200640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb60968910 0x55fb6096c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 -- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb60968910 msgr2=0x55fb6096c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb60968910 0x55fb6096c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 -- 172.21.15.175:0/2577037444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 msgr2=0x55fb60970f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb60970f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f884f400640 1 -- 172.21.15.175:0/2577037444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6096cd40 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.343+0000 7f8855200640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb60968910 0x55fb6096c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f884f400640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 0x55fb6096c2c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55fb6095e610 tx=0x7f883c0044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a571c25509d304 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f883a000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f883a000640 1 -- 172.21.15.175:0/2577037444 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c027450 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f883a000640 1 -- 172.21.15.175:0/2577037444 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f883c0275f0 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884009b1f0 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f883a000640 1 -- 172.21.15.175:0/2577037444 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f883c0277f0 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f8859e13740 10 client.26212 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.344+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fb609e3360 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f883a000640 1 -- 172.21.15.175:0/2577037444 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884009b1f0 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f883a000640 1 -- 172.21.15.175:0/2577037444 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f883c012c10 con 0x55fb6095f0e0 2024-07-29T00:40:31.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f8859e13740 10 client.26212 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:31.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f8859e13740 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:31.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f8859e13740 20 client.26212 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:31.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f8859e13740 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb609e3bd0 0x55fb609e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:31.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f8859e13740 4 client.26212 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:40:31.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb608cb530 con 0x55fb609e3bd0 2024-07-29T00:40:31.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.345+0000 7f884fe00640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb609e3bd0 0x55fb609e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:31.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f883a000640 10 client.26212 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:31.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f884fe00640 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb609e3bd0 0x55fb609e5ff0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68d4a89f1ffe884 server_cookie=41a0bb31b841bab9 in_seq=0 out_seq=0 2024-07-29T00:40:31.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f883a000640 1 -- 172.21.15.175:0/2577037444 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb608cb530 con 0x55fb609e3bd0 2024-07-29T00:40:31.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f883a000640 10 client.26212 handle_command_reply: tid=0 2024-07-29T00:40:31.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f8859e13740 1 client.26212 shutdown 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 msgr2=0x55fb6096c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.346+0000 7f8859e13740 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 0x55fb6096c2c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55fb6095e610 tx=0x7f883c0044d0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 shutdown_connections 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55fb609e3bd0 0x55fb609e5ff0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb60968910 0x55fb6096c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb6095f0e0 0x55fb6096c2c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 --2- 172.21.15.175:0/2577037444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb6095db70 0x55fb60970f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 >> 172.21.15.175:0/2577037444 conn(0x55fb60926950 msgr2=0x55fb60958b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f883a000640 0 client.26212 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 shutdown_connections 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 1 -- 172.21.15.175:0/2577037444 wait complete. 2024-07-29T00:40:31.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.347+0000 7f8859e13740 20 client.26212 trim_cache size 0 max 16384 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88340a2cf0 msgr2=0x7f88340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88340a2cf0 0x7f88340a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f884804c000 tx=0x7f884804d000 comp rx=0 tx=0).stop 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 msgr2=0x7f885019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f885019a160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f884000e9f0 tx=0x7f884000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 shutdown_connections 2024-07-29T00:40:31.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885010e040 0x7f885019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8850100e40 0x7f885019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f88340a2cf0 0x7f88340a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 --2- 172.21.15.175:0/1468827214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850100490 0x7f885019a160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:31.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 >> 172.21.15.175:0/1468827214 conn(0x7f88500fc300 msgr2=0x7f88500fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:31.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 shutdown_connections 2024-07-29T00:40:31.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:31.348+0000 7f8855c00640 1 -- 172.21.15.175:0/1468827214 wait complete. 2024-07-29T00:40:31.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:31 smithi181 ceph-mon[36706]: pgmap v123: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-07-29T00:40:31.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:31 smithi204 ceph-mon[37368]: pgmap v123: 97 pgs: 97 active+clean; 70 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-07-29T00:40:31.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 106783 0.84 MB/sec execute 22 sec latency 17.855 ms 2024-07-29T00:40:32.442 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:32.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 107505 0.90 MB/sec execute 23 sec latency 16.881 ms 2024-07-29T00:40:33.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 -- 172.21.15.175:0/2012670882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd288104260 msgr2=0x7fd288110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 --2- 172.21.15.175:0/2012670882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd288104260 0x7fd288110790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd280009930 tx=0x7fd28002a420 comp rx=0 tx=0).stop 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 -- 172.21.15.175:0/2012670882 shutdown_connections 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 --2- 172.21.15.175:0/2012670882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd288104260 0x7fd288110790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 --2- 172.21.15.175:0/2012670882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2881038c0 0x7fd288103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 --2- 172.21.15.175:0/2012670882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2881098c0 0x7fd288109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 -- 172.21.15.175:0/2012670882 >> 172.21.15.175:0/2012670882 conn(0x7fd2880ff600 msgr2=0x7fd288101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 -- 172.21.15.175:0/2012670882 shutdown_connections 2024-07-29T00:40:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.082+0000 7fd28dc00640 1 -- 172.21.15.175:0/2012670882 wait complete. 2024-07-29T00:40:33.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.083+0000 7fd28dc00640 1 Processor -- start 2024-07-29T00:40:33.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.083+0000 7fd28dc00640 1 -- start start 2024-07-29T00:40:33.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2881038c0 0x7fd28819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:33.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 0x7fd28819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:33.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2881098c0 0x7fd28819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd288113d10 con 0x7fd2881098c0 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28dc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd288113b90 con 0x7fd288104260 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd288113e90 con 0x7fd2881038c0 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 0x7fd28819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 0x7fd28819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56374/0 (socket says 172.21.15.175:56374) 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 -- 172.21.15.175:0/4149135695 learned_addr learned my addr 172.21.15.175:0/4149135695 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287e00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2881038c0 0x7fd28819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:33.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd28d200640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2881098c0 0x7fd28819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:33.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 -- 172.21.15.175:0/4149135695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2881038c0 msgr2=0x7fd28819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:33.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2881038c0 0x7fd28819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 -- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2881098c0 msgr2=0x7fd28819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:33.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2881098c0 0x7fd28819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.084+0000 7fd287400640 1 -- 172.21.15.175:0/4149135695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28819f140 con 0x7fd288104260 2024-07-29T00:40:33.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.085+0000 7fd287400640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 0x7fd28819a6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd27000ebc0 tx=0x7fd27000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:33.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.085+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270010630 con 0x7fd288104260 2024-07-29T00:40:33.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.085+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd270010cb0 con 0x7fd288104260 2024-07-29T00:40:33.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.085+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270018bd0 con 0x7fd288104260 2024-07-29T00:40:33.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.085+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28819f430 con 0x7fd288104260 2024-07-29T00:40:33.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.085+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a6d10 con 0x7fd288104260 2024-07-29T00:40:33.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.086+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd288105000 con 0x7fd288104260 2024-07-29T00:40:33.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.089+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd2700107d0 con 0x7fd288104260 2024-07-29T00:40:33.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.090+0000 7fd284c00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd2680a2d70 0x7fd2680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:33.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.091+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd2700cfdc0 con 0x7fd288104260 2024-07-29T00:40:33.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.091+0000 7fd287e00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd2680a2d70 0x7fd2680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:33.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.091+0000 7fd287e00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd2680a2d70 0x7fd2680a5230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd278009b50 tx=0x7fd278005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:33.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.094+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd27009c960 con 0x7fd288104260 2024-07-29T00:40:33.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:33 smithi175 ceph-mon[32138]: pgmap v124: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-07-29T00:40:33.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.353+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2881a6f40 con 0x7fd288104260 2024-07-29T00:40:33.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.354+0000 7fd284c00640 1 -- 172.21.15.175:0/4149135695 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12001 (secure 0 0 0) 0x7fd27009c260 con 0x7fd288104260 2024-07-29T00:40:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:40:33.355 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":31,"btime":"2024-07-29T00:38:32:652621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:38:32.432546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:40:33.356 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 31 2024-07-29T00:40:33.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.357+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd2680a2d70 msgr2=0x7fd2680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:33.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.357+0000 7fd28dc00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd2680a2d70 0x7fd2680a5230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd278009b50 tx=0x7fd278005f90 comp rx=0 tx=0).stop 2024-07-29T00:40:33.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.357+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 msgr2=0x7fd28819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:33.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.357+0000 7fd28dc00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 0x7fd28819a6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd27000ebc0 tx=0x7fd27000c6a0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 shutdown_connections 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd2680a2d70 0x7fd2680a5230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2881098c0 0x7fd28819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288104260 0x7fd28819a6d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 --2- 172.21.15.175:0/4149135695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2881038c0 0x7fd28819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 >> 172.21.15.175:0/4149135695 conn(0x7fd2880ff600 msgr2=0x7fd2880ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 shutdown_connections 2024-07-29T00:40:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:33.358+0000 7fd28dc00640 1 -- 172.21.15.175:0/4149135695 wait complete. 2024-07-29T00:40:33.427 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:40:33.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:40:33.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:33 smithi181 ceph-mon[36706]: pgmap v124: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-07-29T00:40:33.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:33 smithi204 ceph-mon[37368]: pgmap v124: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-07-29T00:40:33.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 109202 1.24 MB/sec execute 24 sec latency 117.789 ms 2024-07-29T00:40:34.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:34 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/4149135695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:34.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:34 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/4149135695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:34.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:34 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/4149135695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:34.760 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 90 --expiration 139.8 --await-for 45.0' 2024-07-29T00:40:34.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db / --timeout 90 --expiration 139.8 --await-for 45.0 2024-07-29T00:40:34.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 110212 1.51 MB/sec execute 25 sec latency 321.624 ms 2024-07-29T00:40:34.883 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:35.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:35 smithi175 ceph-mon[32138]: pgmap v125: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 980 KiB/s wr, 65 op/s 2024-07-29T00:40:35.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:35 smithi181 ceph-mon[36706]: pgmap v125: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 980 KiB/s wr, 65 op/s 2024-07-29T00:40:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 -- 172.21.15.175:0/668324468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a0102a20 msgr2=0x7f06a0102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 --2- 172.21.15.175:0/668324468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a0102a20 0x7f06a0102e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0694009900 tx=0x7f069402a400 comp rx=0 tx=0).stop 2024-07-29T00:40:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 -- 172.21.15.175:0/668324468 shutdown_connections 2024-07-29T00:40:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 --2- 172.21.15.175:0/668324468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0107f90 0x7f06a010e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 --2- 172.21.15.175:0/668324468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a01033d0 0x7f06a0107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 --2- 172.21.15.175:0/668324468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a0102a20 0x7f06a0102e00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 -- 172.21.15.175:0/668324468 >> 172.21.15.175:0/668324468 conn(0x7f06a00fc4c0 msgr2=0x7f06a00fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.555+0000 7f06a7e00640 1 -- 172.21.15.175:0/668324468 shutdown_connections 2024-07-29T00:40:35.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.556+0000 7f06a7e00640 1 -- 172.21.15.175:0/668324468 wait complete. 2024-07-29T00:40:35.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.557+0000 7f06a7e00640 1 Processor -- start 2024-07-29T00:40:35.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.557+0000 7f06a7e00640 1 -- start start 2024-07-29T00:40:35.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.557+0000 7f06a7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 0x7f06a019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.557+0000 7f06a7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a01033d0 0x7f06a019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a0107f90 0x7f06a019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0111de0 con 0x7f06a0107f90 2024-07-29T00:40:35.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0111c60 con 0x7f06a0102a20 2024-07-29T00:40:35.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0111f60 con 0x7f06a01033d0 2024-07-29T00:40:35.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 0x7f06a019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 0x7f06a019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56398/0 (socket says 172.21.15.175:56398) 2024-07-29T00:40:35.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 -- 172.21.15.175:0/372623367 learned_addr learned my addr 172.21.15.175:0/372623367 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:35.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a0107f90 0x7f06a019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a4a00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a01033d0 0x7f06a019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 -- 172.21.15.175:0/372623367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a01033d0 msgr2=0x7f06a019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a01033d0 0x7f06a019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 -- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a0107f90 msgr2=0x7f06a019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a0107f90 0x7f06a019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5400640 1 -- 172.21.15.175:0/372623367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a019f410 con 0x7f06a0102a20 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a4a00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a01033d0 0x7f06a019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.558+0000 7f06a5e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a0107f90 0x7f06a019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:35.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.559+0000 7f06a5400640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 0x7f06a019a460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f069402a9f0 tx=0x7f0694003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:35.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.559+0000 7f069e000640 1 -- 172.21.15.175:0/372623367 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069403f070 con 0x7f06a0102a20 2024-07-29T00:40:35.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.559+0000 7f069e000640 1 -- 172.21.15.175:0/372623367 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694031c30 con 0x7f06a0102a20 2024-07-29T00:40:35.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.559+0000 7f069e000640 1 -- 172.21.15.175:0/372623367 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694035030 con 0x7f06a0102a20 2024-07-29T00:40:35.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.559+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a019f6a0 con 0x7f06a0102a20 2024-07-29T00:40:35.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.560+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a019fb30 con 0x7f06a0102a20 2024-07-29T00:40:35.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.563+0000 7f069e000640 1 -- 172.21.15.175:0/372623367 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0694042070 con 0x7f06a0102a20 2024-07-29T00:40:35.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.564+0000 7f06aa8d9740 1 Processor -- start 2024-07-29T00:40:35.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.564+0000 7f06aa8d9740 1 -- start start 2024-07-29T00:40:35.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.564+0000 7f069e000640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f06880a2d70 0x7f06880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c7cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92c877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92c8a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a92be28a0 con 0x555a92c7cb40 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f069e000640 1 -- 172.21.15.175:0/372623367 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f06940efb60 con 0x7f06a0102a20 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555a92b7a710 con 0x555a92c7d460 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a92bde860 con 0x555a92c87d20 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c7cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c7cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39018/0 (socket says 172.21.15.175:39018) 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06a5400640 1 -- 172.21.15.175:0/670077058 learned_addr learned my addr 172.21.15.175:0/670077058 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06a5e00640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92c8a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06a4a00640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92c877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.565+0000 7f06a5e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f06880a2d70 0x7f06880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5400640 1 -- 172.21.15.175:0/670077058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 msgr2=0x555a92c8a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5400640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92c8a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5400640 1 -- 172.21.15.175:0/670077058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 msgr2=0x555a92c877e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5400640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92c877e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5400640 1 -- 172.21.15.175:0/670077058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a929546a0 con 0x555a92c7cb40 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a4a00640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92c877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5e00640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92c8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f06880a2d70 0x7f06880a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0698009fd0 tx=0x7f0698009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f06a5400640 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c7cf20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06940b89d0 tx=0x7f06940bbc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5694b1661dc3d8e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.566+0000 7f0692600640 1 -- 172.21.15.175:0/670077058 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694049040 con 0x555a92c7cb40 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.567+0000 7f0692600640 1 -- 172.21.15.175:0/670077058 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069403b460 con 0x555a92c7cb40 2024-07-29T00:40:35.568 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.567+0000 7f0692600640 1 -- 172.21.15.175:0/670077058 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940bc000 con 0x555a92c7cb40 2024-07-29T00:40:35.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.567+0000 7f06aa8d9740 1 -- 172.21.15.175:0/670077058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 msgr2=0x555a92c7cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.567+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c7cf20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06940b89d0 tx=0x7f06940bbc00 comp rx=0 tx=0).stop 2024-07-29T00:40:35.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 -- 172.21.15.175:0/670077058 shutdown_connections 2024-07-29T00:40:35.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92c8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92c877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/670077058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c7cf20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 -- 172.21.15.175:0/670077058 >> 172.21.15.175:0/670077058 conn(0x555a92c790d0 msgr2=0x555a92c794b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:35.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 -- 172.21.15.175:0/670077058 shutdown_connections 2024-07-29T00:40:35.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.568+0000 7f06aa8d9740 1 -- 172.21.15.175:0/670077058 wait complete. 2024-07-29T00:40:35.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.569+0000 7f06aa8d9740 1 -- start start 2024-07-29T00:40:35.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.569+0000 7f06aa8d9740 1 Processor -- start 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c98be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c99120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a92c8a650 con 0x555a92c7cb40 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555a92b9d8c0 con 0x555a92c87d20 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a92c8a7d0 con 0x555a92c7d460 2024-07-29T00:40:35.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c99120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06aa8d9740 10 client.0 _ll_register_callbacks cb 0x555a92b89a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c99120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56420/0 (socket says 172.21.15.175:56420) 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 -- 172.21.15.175:0/1657767261 learned_addr learned my addr 172.21.15.175:0/1657767261 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a4a00640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c98be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5400640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 -- 172.21.15.175:0/1657767261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 msgr2=0x555a92c98be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c98be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 -- 172.21.15.175:0/1657767261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 msgr2=0x555a92c986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.570+0000 7f06a5e00640 1 -- 172.21.15.175:0/1657767261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a92b80bc0 con 0x555a92c87d20 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f06a5e00640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c99120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06980122f0 tx=0x7f0698004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24d8f07ba9f3466 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f0686a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f0686a00640 1 -- 172.21.15.175:0/1657767261 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698012e50 con 0x555a92c87d20 2024-07-29T00:40:35.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f0686a00640 1 -- 172.21.15.175:0/1657767261 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0698011dd0 con 0x555a92c87d20 2024-07-29T00:40:35.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a92ba2c90 con 0x555a92c87d20 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f0686a00640 1 -- 172.21.15.175:0/1657767261 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0698004b20 con 0x555a92c87d20 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f06aa8d9740 10 client.26270 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.571+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555a92c9ef60 con 0x555a92c87d20 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f0686a00640 1 -- 172.21.15.175:0/1657767261 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555a92ba2c90 con 0x555a92c87d20 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f0686a00640 1 -- 172.21.15.175:0/1657767261 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f069800f410 con 0x555a92c87d20 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f06aa8d9740 10 client.26270 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f06aa8d9740 10 client.26270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f06aa8d9740 20 client.26270 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:35.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92ca0480 0x555a92ca2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f06aa8d9740 4 client.26270 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:35.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.572+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a92befe10 con 0x555a92ca0480 2024-07-29T00:40:35.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.573+0000 7f06a5400640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92ca0480 0x555a92ca2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.573+0000 7f0686a00640 10 client.26270 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:35.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.573+0000 7f06a5400640 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92ca0480 0x555a92ca2900 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0003b5ade79509 server_cookie=f26c8172199e8bc1 in_seq=0 out_seq=0 2024-07-29T00:40:35.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f0686a00640 1 -- 172.21.15.175:0/1657767261 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a92befe10 con 0x555a92ca0480 2024-07-29T00:40:35.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f0686a00640 10 client.26270 handle_command_reply: tid=0 2024-07-29T00:40:35.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f06aa8d9740 1 client.26270 shutdown 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 msgr2=0x555a92c99120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c99120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06980122f0 tx=0x7f0698004ab0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 shutdown_connections 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92ca0480 0x555a92ca2900 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.574+0000 7f0686a00640 0 client.26270 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.575+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c99120 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.575+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c98be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.575+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1657767261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.575+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 >> 172.21.15.175:0/1657767261 conn(0x555a92c790d0 msgr2=0x555a92c7ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.575+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 shutdown_connections 2024-07-29T00:40:35.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.575+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1657767261 wait complete. 2024-07-29T00:40:35.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.576+0000 7f06aa8d9740 20 client.26270 trim_cache size 0 max 16384 2024-07-29T00:40:35.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:35 smithi204 ceph-mon[37368]: pgmap v125: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 980 KiB/s wr, 65 op/s 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.596+0000 7f06aa8d9740 1 Processor -- start 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.596+0000 7f06aa8d9740 1 -- start start 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.596+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c98f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.596+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92ca81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92ca8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06a006fd40 con 0x555a92c7cb40 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0071940 con 0x555a92c7d460 2024-07-29T00:40:35.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0070410 con 0x555a92c87d20 2024-07-29T00:40:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c98f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92ca8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c98f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39030/0 (socket says 172.21.15.175:39030) 2024-07-29T00:40:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5400640 1 -- 172.21.15.175:0/178338832 learned_addr learned my addr 172.21.15.175:0/178338832 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:35.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a4a00640 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92ca81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5e00640 1 -- 172.21.15.175:0/178338832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 msgr2=0x555a92ca81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5e00640 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92ca81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5e00640 1 -- 172.21.15.175:0/178338832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 msgr2=0x555a92c98f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.597+0000 7f06a5e00640 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c98f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.598+0000 7f06a5e00640 1 -- 172.21.15.175:0/178338832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a92c78f00 con 0x555a92c87d20 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.598+0000 7f06a5e00640 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92ca8710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f069800f980 tx=0x7f06980123e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ba7d08fde7817c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.598+0000 7f0691c00640 1 -- 172.21.15.175:0/178338832 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698012e30 con 0x555a92c87d20 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.598+0000 7f0691c00640 1 -- 172.21.15.175:0/178338832 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0698011ca0 con 0x555a92c87d20 2024-07-29T00:40:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.598+0000 7f0691c00640 1 -- 172.21.15.175:0/178338832 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698028bc0 con 0x555a92c87d20 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 -- 172.21.15.175:0/178338832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 msgr2=0x555a92ca8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92ca8710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f069800f980 tx=0x7f06980123e0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 -- 172.21.15.175:0/178338832 shutdown_connections 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c87d20 0x555a92ca8710 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c7d460 0x555a92ca81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/178338832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c98f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 -- 172.21.15.175:0/178338832 >> 172.21.15.175:0/178338832 conn(0x555a92c790d0 msgr2=0x555a92c9a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:35.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.599+0000 7f06aa8d9740 1 -- 172.21.15.175:0/178338832 shutdown_connections 2024-07-29T00:40:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.600+0000 7f06aa8d9740 1 -- 172.21.15.175:0/178338832 wait complete. 2024-07-29T00:40:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.600+0000 7f06aa8d9740 1 -- start start 2024-07-29T00:40:35.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.600+0000 7f06aa8d9740 1 Processor -- start 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00711b0 con 0x555a92c7cb40 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0070ae0 con 0x555a92c87d20 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06a006dd20 con 0x555a92c7d460 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06aa8d9740 10 client.0 _ll_register_callbacks cb 0x7f0698012d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:35.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c85d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39036/0 (socket says 172.21.15.175:39036) 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.601+0000 7f06a5400640 1 -- 172.21.15.175:0/1117294068 learned_addr learned my addr 172.21.15.175:0/1117294068 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5e00640 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5400640 1 -- 172.21.15.175:0/1117294068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 msgr2=0x555a92c86260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5400640 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c86260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5400640 1 -- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 msgr2=0x555a92c812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5400640 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5400640 1 -- 172.21.15.175:0/1117294068 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a92c81aa0 con 0x555a92c7cb40 2024-07-29T00:40:35.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f06a5400640 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c85d20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0694006010 tx=0x7f06940b7ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de6dd89481acd2a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:35.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f0686a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:40:35.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f0686a00640 1 -- 172.21.15.175:0/1117294068 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694049040 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.602+0000 7f0686a00640 1 -- 172.21.15.175:0/1117294068 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f069403e170 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.603+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0698011ca0 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.603+0000 7f0686a00640 1 -- 172.21.15.175:0/1117294068 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0694031de0 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.603+0000 7f06aa8d9740 10 client.16146 fetch_fsmap learned FSMap version 31 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.603+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555a92d0ae60 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.603+0000 7f0686a00640 1 -- 172.21.15.175:0/1117294068 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0698011ca0 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.603+0000 7f0686a00640 1 -- 172.21.15.175:0/1117294068 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f06940ed4b0 con 0x555a92c7cb40 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06aa8d9740 10 client.16146 fetch_fsmap finished waiting for FSMap version 31 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06aa8d9740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06aa8d9740 20 client.16146 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92d0b3f0 0x555a92d0d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06aa8d9740 4 client.16146 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a92c942c0 con 0x555a92d0b3f0 2024-07-29T00:40:35.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.604+0000 7f06a4a00640 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92d0b3f0 0x555a92d0d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.605+0000 7f06a4a00640 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92d0b3f0 0x555a92d0d810 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee075f0593c50e62 server_cookie=b88fc7deecd9adf7 in_seq=0 out_seq=0 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.605+0000 7f0686a00640 10 client.16146 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f0686a00640 1 -- 172.21.15.175:0/1117294068 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a92c942c0 con 0x555a92d0b3f0 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f0686a00640 10 client.16146 handle_command_reply: tid=0 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 client.16146 shutdown 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 msgr2=0x555a92c85d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c85d20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0694006010 tx=0x7f06940b7ca0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 shutdown_connections 2024-07-29T00:40:35.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a92d0b3f0 0x555a92d0d810 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a92c87d20 0x555a92c812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a92c7d460 0x555a92c86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 --2- 172.21.15.175:0/1117294068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a92c7cb40 0x555a92c85d20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 >> 172.21.15.175:0/1117294068 conn(0x555a92c790d0 msgr2=0x555a92c99f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.606+0000 7f0686a00640 0 client.16146 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.607+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 shutdown_connections 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.607+0000 7f06aa8d9740 1 -- 172.21.15.175:0/1117294068 wait complete. 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.607+0000 7f06aa8d9740 20 client.16146 trim_cache size 0 max 16384 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "12c04211-19f2-45f4-a7e7-ed8f89409b4c", 2024-07-29T00:40:35.608 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f06880a2d70 msgr2=0x7f06880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f06880a2d70 0x7f06880a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0698009fd0 tx=0x7f0698009500 comp rx=0 tx=0).stop 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 msgr2=0x7f06a019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 0x7f06a019a460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f069402a9f0 tx=0x7f0694003c10 comp rx=0 tx=0).stop 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 shutdown_connections 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f06880a2d70 0x7f06880a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06a0107f90 0x7f06a019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06a01033d0 0x7f06a019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 --2- 172.21.15.175:0/372623367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a0102a20 0x7f06a019a460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:35.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 >> 172.21.15.175:0/372623367 conn(0x7f06a00fc4c0 msgr2=0x7f06a00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:35.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 shutdown_connections 2024-07-29T00:40:35.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:35.608+0000 7f06a7e00640 1 -- 172.21.15.175:0/372623367 wait complete. 2024-07-29T00:40:35.628 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:35.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 111072 1.54 MB/sec execute 26 sec latency 18.667 ms 2024-07-29T00:40:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:36 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:40:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:36 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:40:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:36 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:36 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:36 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:40:36.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:36 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:36 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:40:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:36 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:40:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:36 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:36 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:36 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:40:36.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:36 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:36 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:40:36.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:36 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:40:36.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:36 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:36 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:36 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:40:36.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:36 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:36.881 INFO:tasks.workunit.client.0.smithi175.stdout: 1 112898 1.85 MB/sec execute 27 sec latency 21.818 ms 2024-07-29T00:40:36.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:40:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:40:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:40:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:40:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:40:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: pgmap v126: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-07-29T00:40:37.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:37 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:37.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 -- 172.21.15.175:0/3111825642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 msgr2=0x7f55b4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 --2- 172.21.15.175:0/3111825642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b4107aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009890 tx=0x7f55a802a360 comp rx=0 tx=0).stop 2024-07-29T00:40:37.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 -- 172.21.15.175:0/3111825642 shutdown_connections 2024-07-29T00:40:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 --2- 172.21.15.175:0/3111825642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55b4102060 0x7f55b410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 --2- 172.21.15.175:0/3111825642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b4101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 --2- 172.21.15.175:0/3111825642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b4107aa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.327+0000 7f55b9e00640 1 -- 172.21.15.175:0/3111825642 >> 172.21.15.175:0/3111825642 conn(0x7f55b40fd400 msgr2=0x7f55b40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.328+0000 7f55b9e00640 1 -- 172.21.15.175:0/3111825642 shutdown_connections 2024-07-29T00:40:37.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.328+0000 7f55b9e00640 1 -- 172.21.15.175:0/3111825642 wait complete. 2024-07-29T00:40:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 Processor -- start 2024-07-29T00:40:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 -- start start 2024-07-29T00:40:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55b4102060 0x7f55b419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4111a80 con 0x7f55b41016c0 2024-07-29T00:40:37.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4111900 con 0x7f55b4102060 2024-07-29T00:40:37.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b9e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4111c00 con 0x7f55b41076c0 2024-07-29T00:40:37.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.329+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39054/0 (socket says 172.21.15.175:39054) 2024-07-29T00:40:37.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b8a00640 1 -- 172.21.15.175:0/2753337974 learned_addr learned my addr 172.21.15.175:0/2753337974 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55b4102060 0x7f55b419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 -- 172.21.15.175:0/2753337974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55b4102060 msgr2=0x7f55b419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55b4102060 0x7f55b419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 -- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 msgr2=0x7f55b419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 -- 172.21.15.175:0/2753337974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b419f140 con 0x7f55b41076c0 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.330+0000 7f55b9400640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b419ea60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00da10 tx=0x7f55ac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.331+0000 7f55b1600640 1 -- 172.21.15.175:0/2753337974 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00bd10 con 0x7f55b41076c0 2024-07-29T00:40:37.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.331+0000 7f55b1600640 1 -- 172.21.15.175:0/2753337974 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ac003e50 con 0x7f55b41076c0 2024-07-29T00:40:37.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.331+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b419f430 con 0x7f55b41076c0 2024-07-29T00:40:37.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.331+0000 7f55b1600640 1 -- 172.21.15.175:0/2753337974 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0106c0 con 0x7f55b41076c0 2024-07-29T00:40:37.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.331+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b41a6d10 con 0x7f55b41076c0 2024-07-29T00:40:37.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.334+0000 7f55be090740 1 Processor -- start 2024-07-29T00:40:37.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.335+0000 7f55b1600640 1 -- 172.21.15.175:0/2753337974 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f55ac003970 con 0x7f55b41076c0 2024-07-29T00:40:37.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 -- start start 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c29830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c36a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556c67b8ea80 con 0x556c67c34630 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556c67b26080 con 0x556c67c29450 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55be090740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c67b8aa40 con 0x556c67c29d70 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c340f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55530/0 (socket says 172.21.15.175:55530) 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.336+0000 7f55b8a00640 1 -- 172.21.15.175:0/2542661772 learned_addr learned my addr 172.21.15.175:0/2542661772 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:37.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c29830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c36a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b8a00640 1 -- 172.21.15.175:0/2542661772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 msgr2=0x556c67c29830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c29830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b1600640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f559c0a2cf0 0x7f559c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b8a00640 1 -- 172.21.15.175:0/2542661772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 msgr2=0x556c67c36a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c36a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b8a00640 1 -- 172.21.15.175:0/2542661772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c67900b90 con 0x556c67c29d70 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c36a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b1600640 1 -- 172.21.15.175:0/2753337974 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f55ac0cf5e0 con 0x7f55b41076c0 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c340f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c00 tx=0x7f55a80095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391a9e84a0d09f52 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.337+0000 7f55b9400640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f559c0a2cf0 0x7f559c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.338+0000 7f55a5c00640 1 -- 172.21.15.175:0/2542661772 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8041020 con 0x556c67c29d70 2024-07-29T00:40:37.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.338+0000 7f55a5c00640 1 -- 172.21.15.175:0/2542661772 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a8004240 con 0x556c67c29d70 2024-07-29T00:40:37.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.338+0000 7f55a5c00640 1 -- 172.21.15.175:0/2542661772 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8037040 con 0x556c67c29d70 2024-07-29T00:40:37.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 -- 172.21.15.175:0/2542661772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 msgr2=0x556c67c340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55b9400640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f559c0a2cf0 0x7f559c0a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55ac09b7e0 tx=0x7f55ac005f10 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:37.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c340f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c00 tx=0x7f55a80095a0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 -- 172.21.15.175:0/2542661772 shutdown_connections 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c36a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c340f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 --2- 172.21.15.175:0/2542661772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c29830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 -- 172.21.15.175:0/2542661772 >> 172.21.15.175:0/2542661772 conn(0x556c67c259e0 msgr2=0x556c67c25dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 -- 172.21.15.175:0/2542661772 shutdown_connections 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.339+0000 7f55be090740 1 -- 172.21.15.175:0/2542661772 wait complete. 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.340+0000 7f55be090740 1 -- start start 2024-07-29T00:40:37.341 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.340+0000 7f55be090740 1 Processor -- start 2024-07-29T00:40:37.343 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c31500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c31a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c31f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556c67c37050 con 0x556c67c34630 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556c67b49750 con 0x556c67c29450 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c67c371d0 con 0x556c67c29d70 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55be090740 10 client.0 _ll_register_callbacks cb 0x556c6789e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c31a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c31500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c31a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55546/0 (socket says 172.21.15.175:55546) 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55b8a00640 1 -- 172.21.15.175:0/3098030406 learned_addr learned my addr 172.21.15.175:0/3098030406 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.341+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c31f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b8a00640 1 -- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 msgr2=0x556c67c31500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b8a00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c31500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b8a00640 1 -- 172.21.15.175:0/3098030406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 msgr2=0x556c67c31f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b8a00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c31f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b8a00640 1 -- 172.21.15.175:0/3098030406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c67c406e0 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c31500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55b8a00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c31a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f55a8035040 tx=0x7f55a8030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09e0f84afa4b047 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f559a000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f559a000640 1 -- 172.21.15.175:0/3098030406 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a804a030 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f559a000640 1 -- 172.21.15.175:0/3098030406 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f55a80035f0 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.342+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a8004240 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f559a000640 1 -- 172.21.15.175:0/3098030406 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55a8041070 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f55be090740 10 client.26245 fetch_fsmap learned FSMap version 32 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c67c37050 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f559a000640 1 -- 172.21.15.175:0/3098030406 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a8004240 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f559a000640 1 -- 172.21.15.175:0/3098030406 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f55a803d8d0 con 0x556c67c29d70 2024-07-29T00:40:37.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f55be090740 10 client.26245 fetch_fsmap finished waiting for FSMap version 32 2024-07-29T00:40:37.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f55be090740 10 client.26245 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:37.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.343+0000 7f55be090740 20 client.26245 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:37.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.344+0000 7f55be090740 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67c441c0 0x556c67c465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.344+0000 7f55be090740 4 client.26245 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:37.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.344+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c67b9c1b0 con 0x556c67c441c0 2024-07-29T00:40:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.344+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67c441c0 0x556c67c465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.345+0000 7f559a000640 10 client.26245 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.345+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67c441c0 0x556c67c465b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167fb7c25bf3fe74 server_cookie=dae6a0194a9ac4e6 in_seq=0 out_seq=0 2024-07-29T00:40:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.345+0000 7f559a000640 1 -- 172.21.15.175:0/3098030406 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556c67b9c1b0 con 0x556c67c441c0 2024-07-29T00:40:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.345+0000 7f559a000640 10 client.26245 handle_command_reply: tid=0 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 client.26245 shutdown 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 msgr2=0x556c67c31a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c31a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f55a8035040 tx=0x7f55a8030a30 comp rx=0 tx=0).stop 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 shutdown_connections 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67c441c0 0x556c67c465b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c34630 0x556c67c31f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c29d70 0x556c67c31a40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 --2- 172.21.15.175:0/3098030406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c31500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 >> 172.21.15.175:0/3098030406 conn(0x556c67c259e0 msgr2=0x556c67c24c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f559a000640 0 client.26245 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 shutdown_connections 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.346+0000 7f55be090740 1 -- 172.21.15.175:0/3098030406 wait complete. 2024-07-29T00:40:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.347+0000 7f55be090740 20 client.26245 trim_cache size 0 max 16384 2024-07-29T00:40:37.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.369+0000 7f55be090740 1 Processor -- start 2024-07-29T00:40:37.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.369+0000 7f55be090740 1 -- start start 2024-07-29T00:40:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55be090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c4be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55be090740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c4c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55be090740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c2aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55be090740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f559400e850 con 0x556c67c29d70 2024-07-29T00:40:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55be090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5594010480 con 0x556c67c29450 2024-07-29T00:40:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55be090740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f559400ef20 con 0x556c67c34630 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c2aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c2aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55548/0 (socket says 172.21.15.175:55548) 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55b3e00640 1 -- 172.21.15.175:0/3927368858 learned_addr learned my addr 172.21.15.175:0/3927368858 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55b8a00640 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c4c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.370+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c4be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b3e00640 1 -- 172.21.15.175:0/3927368858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 msgr2=0x556c67c4be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c4be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b3e00640 1 -- 172.21.15.175:0/3927368858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 msgr2=0x556c67c4c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c4c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b3e00640 1 -- 172.21.15.175:0/3927368858 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c67c25480 con 0x556c67c34630 2024-07-29T00:40:37.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b8a00640 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c4c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:37.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55b3e00640 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c2aea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009690 tx=0x7f55a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e773e5d8b621f9a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:37.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.371+0000 7f55a5200640 1 -- 172.21.15.175:0/3927368858 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0004030 con 0x556c67c34630 2024-07-29T00:40:37.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.372+0000 7f55a5200640 1 -- 172.21.15.175:0/3927368858 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a0005500 con 0x556c67c34630 2024-07-29T00:40:37.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.372+0000 7f55a5200640 1 -- 172.21.15.175:0/3927368858 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0004b00 con 0x556c67c34630 2024-07-29T00:40:37.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.372+0000 7f55be090740 1 -- 172.21.15.175:0/3927368858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 msgr2=0x556c67c2aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.372+0000 7f55be090740 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c2aea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009690 tx=0x7f55a0009260 comp rx=0 tx=0).stop 2024-07-29T00:40:37.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 -- 172.21.15.175:0/3927368858 shutdown_connections 2024-07-29T00:40:37.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c2aea0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c4c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 --2- 172.21.15.175:0/3927368858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c4be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 -- 172.21.15.175:0/3927368858 >> 172.21.15.175:0/3927368858 conn(0x556c67c259e0 msgr2=0x556c67c24c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:37.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 -- 172.21.15.175:0/3927368858 shutdown_connections 2024-07-29T00:40:37.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.373+0000 7f55be090740 1 -- 172.21.15.175:0/3927368858 wait complete. 2024-07-29T00:40:37.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.374+0000 7f55be090740 1 -- start start 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.374+0000 7f55be090740 1 Processor -- start 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c3cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c37f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f559400fcf0 con 0x556c67c29d70 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f559400f5f0 con 0x556c67c29450 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f559400c830 con 0x556c67c34630 2024-07-29T00:40:37.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55be090740 10 client.0 _ll_register_callbacks cb 0x7f55a0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:37.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c37f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55550/0 (socket says 172.21.15.175:55550) 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55b3e00640 1 -- 172.21.15.175:0/2707044518 learned_addr learned my addr 172.21.15.175:0/2707044518 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.375+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c3cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b8a00640 1 -- 172.21.15.175:0/2707044518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 msgr2=0x556c67c384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b8a00640 1 -- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 msgr2=0x556c67c3cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c3cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b8a00640 1 -- 172.21.15.175:0/2707044518 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c67c38bb0 con 0x556c67c29d70 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f55b8a00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c37f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55a80099b0 tx=0x7f55a8003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc735affc0ecc09e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f559a000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.376+0000 7f559a000640 1 -- 172.21.15.175:0/2707044518 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a804a070 con 0x556c67c29d70 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.377+0000 7f559a000640 1 -- 172.21.15.175:0/2707044518 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f55a8004100 con 0x556c67c29d70 2024-07-29T00:40:37.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.377+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a0005500 con 0x556c67c29d70 2024-07-29T00:40:37.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.377+0000 7f559a000640 1 -- 172.21.15.175:0/2707044518 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55a8037040 con 0x556c67c29d70 2024-07-29T00:40:37.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 10 client.16164 fetch_fsmap learned FSMap version 32 2024-07-29T00:40:37.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c67caeca0 con 0x556c67c29d70 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f559a000640 1 -- 172.21.15.175:0/2707044518 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a0005500 con 0x556c67c29d70 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f559a000640 1 -- 172.21.15.175:0/2707044518 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f55a803d480 con 0x556c67c29d70 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 10 client.16164 fetch_fsmap finished waiting for FSMap version 32 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 10 client.16164 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 20 client.16164 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67caf4c0 0x556c67cb18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 4 client.16164 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.8, "await_for": 45.0}] 2024-07-29T00:40:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.378+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.8, "await_for": 45.0}) -- 0x556c67c26590 con 0x556c67caf4c0 2024-07-29T00:40:37.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.379+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67caf4c0 0x556c67cb18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:37.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.379+0000 7f55b3e00640 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67caf4c0 0x556c67cb18e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7c8506fe11cb64 server_cookie=cc46cd3e47816ff2 in_seq=0 out_seq=0 2024-07-29T00:40:37.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.379+0000 7f559a000640 10 client.16164 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:37.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f559a000640 1 -- 172.21.15.175:0/2707044518 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556c67c26590 con 0x556c67caf4c0 2024-07-29T00:40:37.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f559a000640 10 client.16164 handle_command_reply: tid=0 2024-07-29T00:40:37.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 client.16164 shutdown 2024-07-29T00:40:37.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 msgr2=0x556c67c37f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c37f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55a80099b0 tx=0x7f55a8003e40 comp rx=0 tx=0).stop 2024-07-29T00:40:37.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 shutdown_connections 2024-07-29T00:40:37.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x556c67caf4c0 0x556c67cb18e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c67c34630 0x556c67c384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c67c29d70 0x556c67c37f80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 --2- 172.21.15.175:0/2707044518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c67c29450 0x556c67c3cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 >> 172.21.15.175:0/2707044518 conn(0x556c67c259e0 msgr2=0x556c67c2aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f559a000640 0 client.16164 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 shutdown_connections 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.463+0000 7f55be090740 1 -- 172.21.15.175:0/2707044518 wait complete. 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.464+0000 7f55be090740 20 client.16164 trim_cache size 0 max 16384 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 32, 2024-07-29T00:40:37.465 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 14, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "2642d471": { 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 14, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 139.8, 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:40:37.466 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:37.467 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.464+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f559c0a2cf0 msgr2=0x7f559c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.464+0000 7f55b9e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f559c0a2cf0 0x7f559c0a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55ac09b7e0 tx=0x7f55ac005f10 comp rx=0 tx=0).stop 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.464+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 msgr2=0x7f55b419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.464+0000 7f55b9e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b419ea60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00da10 tx=0x7f55ac00de10 comp rx=0 tx=0).stop 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 shutdown_connections 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55b41076c0 0x7f55b419ea60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55b4102060 0x7f55b419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f559c0a2cf0 0x7f559c0a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 --2- 172.21.15.175:0/2753337974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b41016c0 0x7f55b419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 >> 172.21.15.175:0/2753337974 conn(0x7f55b40fd400 msgr2=0x7f55b410cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 shutdown_connections 2024-07-29T00:40:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:37.465+0000 7f55b9e00640 1 -- 172.21.15.175:0/2753337974 wait complete. 2024-07-29T00:40:37.470 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: pgmap v126: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-07-29T00:40:37.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:37 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:40:37.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: pgmap v126: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-07-29T00:40:37.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:37 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:37.882 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.98 MB/sec execute 28 sec latency 466.614 ms 2024-07-29T00:40:38.768 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2642d471', quiesce took 7.7 seconds. Will release after: 12.100000000000001 2024-07-29T00:40:38.882 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.91 MB/sec execute 29 sec latency 1467.528 ms 2024-07-29T00:40:39.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 -- 172.21.15.175:0/4113981703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa224108670 msgr2=0x7fa224114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4113981703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa224108670 0x7fa224114b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009860 tx=0x7fa21c02a340 comp rx=0 tx=0).stop 2024-07-29T00:40:39.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 -- 172.21.15.175:0/4113981703 shutdown_connections 2024-07-29T00:40:39.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4113981703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa224108670 0x7fa224114b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4113981703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa224107d10 0x7fa224108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4113981703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa22410dd50 0x7fa22410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 -- 172.21.15.175:0/4113981703 >> 172.21.15.175:0/4113981703 conn(0x7fa224077670 msgr2=0x7fa224079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:39.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 -- 172.21.15.175:0/4113981703 shutdown_connections 2024-07-29T00:40:39.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.454+0000 7fa22ac00640 1 -- 172.21.15.175:0/4113981703 wait complete. 2024-07-29T00:40:39.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.455+0000 7fa22ac00640 1 Processor -- start 2024-07-29T00:40:39.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.455+0000 7fa22ac00640 1 -- start start 2024-07-29T00:40:39.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.455+0000 7fa22ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 0x7fa22419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.455+0000 7fa22ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa224108670 0x7fa22419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.455+0000 7fa22ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22410dd50 0x7fa2241a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa22ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa224118070 con 0x7fa224108670 2024-07-29T00:40:39.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa22ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa224117ef0 con 0x7fa22410dd50 2024-07-29T00:40:39.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa22ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241181f0 con 0x7fa224107d10 2024-07-29T00:40:39.462 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:39 smithi175 ceph-mon[32138]: pgmap v127: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-07-29T00:40:39.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 0x7fa22419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 0x7fa22419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:48008/0 (socket says 172.21.15.175:48008) 2024-07-29T00:40:39.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 -- 172.21.15.175:0/4102873254 learned_addr learned my addr 172.21.15.175:0/4102873254 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa228e00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa224108670 0x7fa22419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa22a200640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22410dd50 0x7fa2241a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 -- 172.21.15.175:0/4102873254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22410dd50 msgr2=0x7fa2241a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22410dd50 0x7fa2241a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 -- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa224108670 msgr2=0x7fa22419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa224108670 0x7fa22419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.456+0000 7fa229800640 1 -- 172.21.15.175:0/4102873254 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2241a34c0 con 0x7fa224107d10 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.457+0000 7fa229800640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 0x7fa22419e510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa21800eb90 tx=0x7fa21800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.457+0000 7fa222000640 1 -- 172.21.15.175:0/4102873254 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218004d20 con 0x7fa224107d10 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.457+0000 7fa222000640 1 -- 172.21.15.175:0/4102873254 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa21800ee90 con 0x7fa224107d10 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.457+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2241a37b0 con 0x7fa224107d10 2024-07-29T00:40:39.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.457+0000 7fa222000640 1 -- 172.21.15.175:0/4102873254 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218010830 con 0x7fa224107d10 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.457+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2241ab090 con 0x7fa224107d10 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.460+0000 7fa222000640 1 -- 172.21.15.175:0/4102873254 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa2180109d0 con 0x7fa224107d10 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 Processor -- start 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 -- start start 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da05fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da06067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da0606cf0 0x559da06090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559da0560ad0 con 0x559da05fbb10 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559da04f8af0 con 0x559da0606cf0 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22ec42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559da055ca90 con 0x559da05fc430 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa229800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da06067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da05fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da05fbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50134/0 (socket says 172.21.15.175:50134) 2024-07-29T00:40:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa229800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da06067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:48024/0 (socket says 172.21.15.175:48024) 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.461+0000 7fa22a200640 1 -- 172.21.15.175:0/2208667662 learned_addr learned my addr 172.21.15.175:0/2208667662 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa229800640 1 -- 172.21.15.175:0/2208667662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da0606cf0 msgr2=0x559da06090e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa229800640 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da0606cf0 0x559da06090e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa229800640 1 -- 172.21.15.175:0/2208667662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 msgr2=0x559da05fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa229800640 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da05fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa229800640 1 -- 172.21.15.175:0/2208667662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da02d23c0 con 0x559da05fc430 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa22a200640 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da05fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa222000640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa20c0a2cf0 0x7fa20c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.462+0000 7fa229800640 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da06067b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa2180cd080 tx=0x7fa2180d5ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c1d8a867ea43e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.463+0000 7fa22a200640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa20c0a2cf0 0x7fa20c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.463+0000 7fa222000640 1 -- 172.21.15.175:0/4102873254 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa2180cf9b0 con 0x7fa224107d10 2024-07-29T00:40:39.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.463+0000 7fa216a00640 1 -- 172.21.15.175:0/2208667662 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180980f0 con 0x559da05fc430 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.463+0000 7fa216a00640 1 -- 172.21.15.175:0/2208667662 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa218053de0 con 0x559da05fc430 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.463+0000 7fa216a00640 1 -- 172.21.15.175:0/2208667662 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21809b7b0 con 0x559da05fc430 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 -- 172.21.15.175:0/2208667662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 msgr2=0x559da06067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da06067b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa2180cd080 tx=0x7fa2180d5ab0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22a200640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa20c0a2cf0 0x7fa20c0a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0095f0 tx=0x7fa21c005710 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 -- 172.21.15.175:0/2208667662 shutdown_connections 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da0606cf0 0x559da06090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da06067b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 --2- 172.21.15.175:0/2208667662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da05fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 -- 172.21.15.175:0/2208667662 >> 172.21.15.175:0/2208667662 conn(0x559da05f80a0 msgr2=0x559da05f8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 -- 172.21.15.175:0/2208667662 shutdown_connections 2024-07-29T00:40:39.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 -- 172.21.15.175:0/2208667662 wait complete. 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.464+0000 7fa22ec42740 1 -- start start 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 Processor -- start 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 0x559da0603ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da06040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da0604620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559da06098c0 con 0x559da0606cf0 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559da051bbc0 con 0x559da05fc430 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559da0609a40 con 0x559da05fbb10 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22ec42740 10 client.0 _ll_register_callbacks cb 0x559da0507d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa229800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 0x559da0603ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa22a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da0604620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa229800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 0x559da0603ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:48036/0 (socket says 172.21.15.175:48036) 2024-07-29T00:40:39.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.465+0000 7fa229800640 1 -- 172.21.15.175:0/1742054898 learned_addr learned my addr 172.21.15.175:0/1742054898 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa228e00640 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da06040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa229800640 1 -- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 msgr2=0x559da06040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa229800640 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da06040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa229800640 1 -- 172.21.15.175:0/1742054898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 msgr2=0x559da0604620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa229800640 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da0604620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa229800640 1 -- 172.21.15.175:0/1742054898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da0612e10 con 0x559da05fbb10 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa229800640 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 0x559da0603ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa218098820 tx=0x7fa2180986f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be63c4f396a29b04 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa20b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa20b400640 1 -- 172.21.15.175:0/1742054898 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21809b990 con 0x559da05fbb10 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa20b400640 1 -- 172.21.15.175:0/1742054898 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa21809c0b0 con 0x559da05fbb10 2024-07-29T00:40:39.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa218053de0 con 0x559da05fbb10 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa20b400640 1 -- 172.21.15.175:0/1742054898 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa21809bc20 con 0x559da05fbb10 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa22ec42740 10 client.26263 fetch_fsmap learned FSMap version 32 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.466+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559da06098c0 con 0x559da05fbb10 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa20b400640 1 -- 172.21.15.175:0/1742054898 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa218053de0 con 0x559da05fbb10 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa20b400640 1 -- 172.21.15.175:0/1742054898 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fa2180a4630 con 0x559da05fbb10 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa22ec42740 10 client.26263 fetch_fsmap finished waiting for FSMap version 32 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa22ec42740 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa22ec42740 20 client.26263 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa22ec42740 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0616920 0x559da0618d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa22ec42740 4 client.26263 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da056e040 con 0x559da0616920 2024-07-29T00:40:39.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.467+0000 7fa228e00640 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0616920 0x559da0618d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.468+0000 7fa228e00640 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0616920 0x559da0618d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee14d05978a5ccb8 server_cookie=1b571608f773574c in_seq=0 out_seq=0 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.468+0000 7fa20b400640 10 client.26263 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa20b400640 1 -- 172.21.15.175:0/1742054898 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559da056e040 con 0x559da0616920 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa20b400640 10 client.26263 handle_command_reply: tid=0 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 client.26263 shutdown 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 msgr2=0x559da0603ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 0x559da0603ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa218098820 tx=0x7fa2180986f0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 shutdown_connections 2024-07-29T00:40:39.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0616920 0x559da0618d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da0604620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da06040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 --2- 172.21.15.175:0/1742054898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fbb10 0x559da0603ba0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa20b400640 0 client.26263 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 >> 172.21.15.175:0/1742054898 conn(0x559da05f80a0 msgr2=0x559da05fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 shutdown_connections 2024-07-29T00:40:39.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.469+0000 7fa22ec42740 1 -- 172.21.15.175:0/1742054898 wait complete. 2024-07-29T00:40:39.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.470+0000 7fa22ec42740 20 client.26263 trim_cache size 0 max 16384 2024-07-29T00:40:39.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.487+0000 7fa22ec42740 1 Processor -- start 2024-07-29T00:40:39.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.487+0000 7fa22ec42740 1 -- start start 2024-07-29T00:40:39.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 0x559da0604470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da061e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da061eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22ec42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20400e800 con 0x559da0606cf0 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22ec42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa204010430 con 0x559da05fbb10 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22ec42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa20400eed0 con 0x559da05fc430 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa22a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da061eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa229800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 0x559da0604470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa229800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 0x559da0604470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:47252/0 (socket says 172.21.15.175:47252) 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa229800640 1 -- 172.21.15.175:0/4189942057 learned_addr learned my addr 172.21.15.175:0/4189942057 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:39.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.488+0000 7fa228e00640 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da061e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa229800640 1 -- 172.21.15.175:0/4189942057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 msgr2=0x559da061e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa229800640 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da061e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa229800640 1 -- 172.21.15.175:0/4189942057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 msgr2=0x559da061eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa229800640 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da061eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa229800640 1 -- 172.21.15.175:0/4189942057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da05f9210 con 0x559da05fbb10 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa228e00640 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da061e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa229800640 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 0x559da0604470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa21809a100 tx=0x7fa2180d5ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec01dbbbabfe3aeb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:39.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa216000640 1 -- 172.21.15.175:0/4189942057 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21809b990 con 0x559da05fbb10 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa216000640 1 -- 172.21.15.175:0/4189942057 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa21809e030 con 0x559da05fbb10 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.489+0000 7fa216000640 1 -- 172.21.15.175:0/4189942057 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180cd510 con 0x559da05fbb10 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 -- 172.21.15.175:0/4189942057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 msgr2=0x559da0604470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 0x559da0604470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa21809a100 tx=0x7fa2180d5ab0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 -- 172.21.15.175:0/4189942057 shutdown_connections 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da0606cf0 0x559da061eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da05fc430 0x559da061e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 --2- 172.21.15.175:0/4189942057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fbb10 0x559da0604470 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 -- 172.21.15.175:0/4189942057 >> 172.21.15.175:0/4189942057 conn(0x559da05f80a0 msgr2=0x559da0605c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:39.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 -- 172.21.15.175:0/4189942057 shutdown_connections 2024-07-29T00:40:39.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.490+0000 7fa22ec42740 1 -- 172.21.15.175:0/4189942057 wait complete. 2024-07-29T00:40:39.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 -- start start 2024-07-29T00:40:39.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 Processor -- start 2024-07-29T00:40:39.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da060f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da060a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da0606cf0 0x559da060af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20400fca0 con 0x559da05fbb10 2024-07-29T00:40:39.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.491+0000 7fa22ec42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa20400f5a0 con 0x559da05fc430 2024-07-29T00:40:39.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa22ec42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa20400c7e0 con 0x559da0606cf0 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa22ec42740 10 client.0 _ll_register_callbacks cb 0x7fa2180d5870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da060f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da060f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50174/0 (socket says 172.21.15.175:50174) 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 -- 172.21.15.175:0/3631801395 learned_addr learned my addr 172.21.15.175:0/3631801395 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa228e00640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da060a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa22a200640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da0606cf0 0x559da060af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 -- 172.21.15.175:0/3631801395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da0606cf0 msgr2=0x559da060af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da0606cf0 0x559da060af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 -- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 msgr2=0x559da060a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da060a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa229800640 1 -- 172.21.15.175:0/3631801395 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da060b760 con 0x559da05fbb10 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.492+0000 7fa228e00640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da060a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:40:39.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa229800640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da060f5a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa2180049d0 tx=0x7fa2180d5fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e5fe442f9aa1a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa20b400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa20b400640 1 -- 172.21.15.175:0/3631801395 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180cd510 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa20b400640 1 -- 172.21.15.175:0/3631801395 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa21809c0b0 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa21809e030 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa20b400640 1 -- 172.21.15.175:0/3631801395 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa21809b990 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa22ec42740 10 client.16182 fetch_fsmap learned FSMap version 32 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559da060b760 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.493+0000 7fa20b400640 1 -- 172.21.15.175:0/3631801395 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa21809e030 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa20b400640 1 -- 172.21.15.175:0/3631801395 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fa21809b2a0 con 0x559da05fbb10 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa22ec42740 10 client.16182 fetch_fsmap finished waiting for FSMap version 32 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa22ec42740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa22ec42740 20 client.16182 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:39.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa22ec42740 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0681940 0x559da0683d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa22ec42740 4 client.16182 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559da0568550 con 0x559da0681940 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.494+0000 7fa228e00640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0681940 0x559da0683d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa228e00640 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0681940 0x559da0683d60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79b4944896404d3 server_cookie=6f5c5d8d74942de0 in_seq=0 out_seq=0 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa20b400640 10 client.16182 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa20b400640 1 -- 172.21.15.175:0/3631801395 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559da0568550 con 0x559da0681940 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa20b400640 10 client.16182 handle_command_reply: tid=0 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 client.16182 shutdown 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 msgr2=0x559da060f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da060f5a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa2180049d0 tx=0x7fa2180d5fd0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 shutdown_connections 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559da0681940 0x559da0683d60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559da0606cf0 0x559da060af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559da05fc430 0x559da060a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa20b400640 0 client.16182 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 --2- 172.21.15.175:0/3631801395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559da05fbb10 0x559da060f5a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 >> 172.21.15.175:0/3631801395 conn(0x559da05f80a0 msgr2=0x559da05fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 shutdown_connections 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.495+0000 7fa22ec42740 1 -- 172.21.15.175:0/3631801395 wait complete. 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ec42740 20 client.16182 trim_cache size 0 max 16384 2024-07-29T00:40:39.500 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: "12c04211-19f2-45f4-a7e7-ed8f89409b4c": { 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "12c04211-19f2-45f4-a7e7-ed8f89409b4c", 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:39.501 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:39.501 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:39 smithi181 ceph-mon[36706]: pgmap v127: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-07-29T00:40:39.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa20c0a2cf0 msgr2=0x7fa20c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa20c0a2cf0 0x7fa20c0a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0095f0 tx=0x7fa21c005710 comp rx=0 tx=0).stop 2024-07-29T00:40:39.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 msgr2=0x7fa22419e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:39.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 0x7fa22419e510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa21800eb90 tx=0x7fa21800c6a0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 shutdown_connections 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22410dd50 0x7fa2241a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa20c0a2cf0 0x7fa20c0a51b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa224108670 0x7fa22419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 --2- 172.21.15.175:0/4102873254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa224107d10 0x7fa22419e510 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.496+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 >> 172.21.15.175:0/4102873254 conn(0x7fa224077670 msgr2=0x7fa224078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.497+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 shutdown_connections 2024-07-29T00:40:39.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:39.497+0000 7fa22ac00640 1 -- 172.21.15.175:0/4102873254 wait complete. 2024-07-29T00:40:39.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:39 smithi204 ceph-mon[37368]: pgmap v127: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-07-29T00:40:39.883 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.85 MB/sec execute 30 sec latency 2467.594 ms 2024-07-29T00:40:40.510 INFO:tasks.cephfs.filesystem:scrub status for tag:12c04211-19f2-45f4-a7e7-ed8f89409b4c - {'path': '/', 'tag': '12c04211-19f2-45f4-a7e7-ed8f89409b4c', 'options': 'recursive,force'} 2024-07-29T00:40:40.883 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.79 MB/sec execute 31 sec latency 3468.067 ms 2024-07-29T00:40:41.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:41 smithi181 ceph-mon[36706]: pgmap v128: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-07-29T00:40:41.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:41 smithi175 ceph-mon[32138]: pgmap v128: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-07-29T00:40:41.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:41 smithi204 ceph-mon[37368]: pgmap v128: 97 pgs: 97 active+clean; 72 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-07-29T00:40:41.883 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.73 MB/sec execute 32 sec latency 4468.227 ms 2024-07-29T00:40:42.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:42 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:42.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:42 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:42.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:42 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:42.883 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.68 MB/sec execute 33 sec latency 5468.389 ms 2024-07-29T00:40:43.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:43 smithi181 ceph-mon[36706]: pgmap v129: 97 pgs: 97 active+clean; 75 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-07-29T00:40:43.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:43 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:40:43.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:43 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:40:43.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:43 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:40:43.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:43 smithi175 ceph-mon[32138]: pgmap v129: 97 pgs: 97 active+clean; 75 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-07-29T00:40:43.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:43 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:40:43.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:43 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:40:43.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:43 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:40:43.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:43 smithi204 ceph-mon[37368]: pgmap v129: 97 pgs: 97 active+clean; 75 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-07-29T00:40:43.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:43 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:40:43.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:43 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:40:43.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:43 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:40:43.883 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.63 MB/sec execute 34 sec latency 6468.504 ms 2024-07-29T00:40:44.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:44 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:40:44.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:44 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:40:44.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:44 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:40:44.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.58 MB/sec execute 35 sec latency 7468.671 ms 2024-07-29T00:40:45.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:45 smithi181 ceph-mon[36706]: pgmap v130: 97 pgs: 97 active+clean; 75 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-07-29T00:40:45.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:45 smithi175 ceph-mon[32138]: pgmap v130: 97 pgs: 97 active+clean; 75 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-07-29T00:40:45.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:45 smithi204 ceph-mon[37368]: pgmap v130: 97 pgs: 97 active+clean; 75 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-07-29T00:40:45.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.54 MB/sec execute 36 sec latency 8468.843 ms 2024-07-29T00:40:46.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.50 MB/sec execute 37 sec latency 9468.946 ms 2024-07-29T00:40:47.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:47 smithi175 ceph-mon[32138]: pgmap v131: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-07-29T00:40:47.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:47 smithi204 ceph-mon[37368]: pgmap v131: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-07-29T00:40:47.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:47 smithi181 ceph-mon[36706]: pgmap v131: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-07-29T00:40:47.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.46 MB/sec execute 38 sec latency 10469.060 ms 2024-07-29T00:40:48.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:48 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:40:48.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:48 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:40:48.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:48 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:40:48.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.42 MB/sec execute 39 sec latency 11469.235 ms 2024-07-29T00:40:49.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:49 smithi175 ceph-mon[32138]: pgmap v132: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 392 KiB/s wr, 48 op/s 2024-07-29T00:40:49.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:49 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:40:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:49 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:40:49.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:49 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:40:49.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:49 smithi204 ceph-mon[37368]: pgmap v132: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 392 KiB/s wr, 48 op/s 2024-07-29T00:40:49.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:49 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:40:49.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:49 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:40:49.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:49 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:40:49.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:49 smithi181 ceph-mon[36706]: pgmap v132: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 392 KiB/s wr, 48 op/s 2024-07-29T00:40:49.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:49 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:40:49.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:49 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:40:49.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:49 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:40:49.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.39 MB/sec execute 40 sec latency 12469.383 ms 2024-07-29T00:40:50.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:40:50.884 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.35 MB/sec execute 41 sec latency 13469.503 ms 2024-07-29T00:40:51.328 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:51.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:51 smithi175 ceph-mon[32138]: pgmap v133: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 392 KiB/s wr, 48 op/s 2024-07-29T00:40:51.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:51 smithi204 ceph-mon[37368]: pgmap v133: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 392 KiB/s wr, 48 op/s 2024-07-29T00:40:51.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:51 smithi181 ceph-mon[36706]: pgmap v133: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 392 KiB/s wr, 48 op/s 2024-07-29T00:40:51.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.32 MB/sec execute 42 sec latency 14469.621 ms 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 -- 172.21.15.175:0/2550267730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80104420 msgr2=0x7f0b80110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 --2- 172.21.15.175:0/2550267730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80104420 0x7f0b80110950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ca0 tx=0x7f0b7c02a8c0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 -- 172.21.15.175:0/2550267730 shutdown_connections 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 --2- 172.21.15.175:0/2550267730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80104420 0x7f0b80110950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 --2- 172.21.15.175:0/2550267730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80103a80 0x7f0b80103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 --2- 172.21.15.175:0/2550267730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b80109a80 0x7f0b80109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 -- 172.21.15.175:0/2550267730 >> 172.21.15.175:0/2550267730 conn(0x7f0b800ff760 msgr2=0x7f0b80101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:52.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 -- 172.21.15.175:0/2550267730 shutdown_connections 2024-07-29T00:40:52.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.053+0000 7f0b88800640 1 -- 172.21.15.175:0/2550267730 wait complete. 2024-07-29T00:40:52.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.054+0000 7f0b88800640 1 Processor -- start 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 -- start start 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 0x7f0b8019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80104420 0x7f0b8019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b80109a80 0x7f0b8019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80113ed0 con 0x7f0b80109a80 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80113d50 con 0x7f0b80103a80 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b88800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80114050 con 0x7f0b80104420 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b85e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 0x7f0b8019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b85e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 0x7f0b8019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58304/0 (socket says 172.21.15.175:58304) 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b85e00640 1 -- 172.21.15.175:0/1022559079 learned_addr learned my addr 172.21.15.175:0/1022559079 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b85400640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80104420 0x7f0b8019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.055+0000 7f0b86800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b80109a80 0x7f0b8019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:52.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b85e00640 1 -- 172.21.15.175:0/1022559079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80104420 msgr2=0x7f0b8019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:52.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b85e00640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80104420 0x7f0b8019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b85e00640 1 -- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b80109a80 msgr2=0x7f0b8019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b85e00640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b80109a80 0x7f0b8019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b85e00640 1 -- 172.21.15.175:0/1022559079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8019f260 con 0x7f0b80103a80 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b85e00640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 0x7f0b8019a2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00ea60 tx=0x7f0b6c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00cde0 con 0x7f0b80103a80 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c0035f0 con 0x7f0b80103a80 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.056+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c010670 con 0x7f0b80103a80 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.057+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8019f550 con 0x7f0b80103a80 2024-07-29T00:40:52.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.057+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a6e30 con 0x7f0b80103a80 2024-07-29T00:40:52.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.058+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80103ee0 con 0x7f0b80103a80 2024-07-29T00:40:52.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.060+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0b6c0037a0 con 0x7f0b80103a80 2024-07-29T00:40:52.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.062+0000 7f0b7aa00640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0b680a2d70 0x7f0b680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:52.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.062+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0b6c0cf760 con 0x7f0b80103a80 2024-07-29T00:40:52.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.062+0000 7f0b85400640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0b680a2d70 0x7f0b680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:52.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.063+0000 7f0b85400640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0b680a2d70 0x7f0b680a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0b80101cf0 tx=0x7f0b74005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:52.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.063+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0b6c09c390 con 0x7f0b80103a80 2024-07-29T00:40:52.292 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.292+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b801a7090 con 0x7f0b80103a80 2024-07-29T00:40:52.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.293+0000 7f0b7aa00640 1 -- 172.21.15.175:0/1022559079 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12004 (secure 0 0 0) 0x7f0b6c09bc90 con 0x7f0b80103a80 2024-07-29T00:40:52.294 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:40:52.294 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":33,"btime":"2024-07-29T00:40:41:209693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:40:40.662388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:40:52.295 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 33 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0b680a2d70 msgr2=0x7f0b680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0b680a2d70 0x7f0b680a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0b80101cf0 tx=0x7f0b74005f90 comp rx=0 tx=0).stop 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 msgr2=0x7f0b8019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 0x7f0b8019a2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00ea60 tx=0x7f0b6c00ef20 comp rx=0 tx=0).stop 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 shutdown_connections 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f0b680a2d70 0x7f0b680a5230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b80109a80 0x7f0b8019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80104420 0x7f0b8019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 --2- 172.21.15.175:0/1022559079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b80103a80 0x7f0b8019a2b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:52.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.296+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 >> 172.21.15.175:0/1022559079 conn(0x7f0b800ff760 msgr2=0x7f0b800ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:52.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.297+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 shutdown_connections 2024-07-29T00:40:52.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:52.297+0000 7f0b88800640 1 -- 172.21.15.175:0/1022559079 wait complete. 2024-07-29T00:40:52.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.29 MB/sec execute 43 sec latency 15469.740 ms 2024-07-29T00:40:53.290 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id 2642d471 --release --await' 2024-07-29T00:40:53.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db --set-id 2642d471 --release --await 2024-07-29T00:40:53.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:53 smithi175 ceph-mon[32138]: pgmap v134: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 401 KiB/s wr, 54 op/s 2024-07-29T00:40:53.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:53 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1022559079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:53.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:53 smithi204 ceph-mon[37368]: pgmap v134: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 401 KiB/s wr, 54 op/s 2024-07-29T00:40:53.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:53 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1022559079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:53 smithi181 ceph-mon[36706]: pgmap v134: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 401 KiB/s wr, 54 op/s 2024-07-29T00:40:53.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:53 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1022559079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:40:53.780 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:40:53.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113554 1.26 MB/sec execute 44 sec latency 16469.913 ms 2024-07-29T00:40:54.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.458+0000 7f51e1800640 1 -- 172.21.15.175:0/1122848613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc1076b0 msgr2=0x7f51dc107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.458+0000 7f51e1800640 1 --2- 172.21.15.175:0/1122848613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc1076b0 0x7f51dc107a90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009930 tx=0x7f51cc02a420 comp rx=0 tx=0).stop 2024-07-29T00:40:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 -- 172.21.15.175:0/1122848613 shutdown_connections 2024-07-29T00:40:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 --2- 172.21.15.175:0/1122848613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51dc102050 0x7f51dc10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 --2- 172.21.15.175:0/1122848613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51dc1016b0 0x7f51dc101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 --2- 172.21.15.175:0/1122848613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc1076b0 0x7f51dc107a90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 -- 172.21.15.175:0/1122848613 >> 172.21.15.175:0/1122848613 conn(0x7f51dc0fd3f0 msgr2=0x7f51dc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 -- 172.21.15.175:0/1122848613 shutdown_connections 2024-07-29T00:40:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.459+0000 7f51e1800640 1 -- 172.21.15.175:0/1122848613 wait complete. 2024-07-29T00:40:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.460+0000 7f51e1800640 1 Processor -- start 2024-07-29T00:40:54.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.460+0000 7f51e1800640 1 -- start start 2024-07-29T00:40:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.460+0000 7f51e1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51dc1016b0 0x7f51dc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51e1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 0x7f51dc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51e1800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51dc1076b0 0x7f51dc19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51e1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc111b00 con 0x7f51dc102050 2024-07-29T00:40:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51e1800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc111980 con 0x7f51dc1076b0 2024-07-29T00:40:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51e1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc111c80 con 0x7f51dc1016b0 2024-07-29T00:40:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51db400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 0x7f51dc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51db400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 0x7f51dc19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43920/0 (socket says 172.21.15.175:43920) 2024-07-29T00:40:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51dc1016b0 0x7f51dc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51db400640 1 -- 172.21.15.175:0/3153535167 learned_addr learned my addr 172.21.15.175:0/3153535167 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.461+0000 7f51e0e00640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51dc1076b0 0x7f51dc19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.462+0000 7f51db400640 1 -- 172.21.15.175:0/3153535167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51dc1016b0 msgr2=0x7f51dc19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.462+0000 7f51db400640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51dc1016b0 0x7f51dc19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.462+0000 7f51db400640 1 -- 172.21.15.175:0/3153535167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51dc1076b0 msgr2=0x7f51dc19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.462+0000 7f51db400640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51dc1076b0 0x7f51dc19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.462+0000 7f51db400640 1 -- 172.21.15.175:0/3153535167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc19f150 con 0x7f51dc102050 2024-07-29T00:40:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.462+0000 7f51db400640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 0x7f51dc19a6e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51c800e9f0 tx=0x7f51c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.463+0000 7f51d8c00640 1 -- 172.21.15.175:0/3153535167 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c800cd80 con 0x7f51dc102050 2024-07-29T00:40:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.463+0000 7f51d8c00640 1 -- 172.21.15.175:0/3153535167 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c8010910 con 0x7f51dc102050 2024-07-29T00:40:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.463+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc19f440 con 0x7f51dc102050 2024-07-29T00:40:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.463+0000 7f51d8c00640 1 -- 172.21.15.175:0/3153535167 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8018990 con 0x7f51dc102050 2024-07-29T00:40:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.463+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc1a6d20 con 0x7f51dc102050 2024-07-29T00:40:54.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.466+0000 7f51e59be740 1 Processor -- start 2024-07-29T00:40:54.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 -- start start 2024-07-29T00:40:54.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51d8c00640 1 -- 172.21.15.175:0/3153535167 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f51c8018b30 con 0x7f51dc102050 2024-07-29T00:40:54.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e29abb1cc0 con 0x55e29ac4c6f0 2024-07-29T00:40:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e29ab49bc0 con 0x55e29ac56fb0 2024-07-29T00:40:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e59be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e29abadc80 con 0x55e29ac4bdd0 2024-07-29T00:40:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52762/0 (socket says 172.21.15.175:52762) 2024-07-29T00:40:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.467+0000 7f51e0e00640 1 -- 172.21.15.175:0/2654571532 learned_addr learned my addr 172.21.15.175:0/2654571532 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51db400640 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51e0e00640 1 -- 172.21.15.175:0/2654571532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 msgr2=0x55e29ac593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51e0e00640 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51e0e00640 1 -- 172.21.15.175:0/2654571532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 msgr2=0x55e29ac56a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51e0e00640 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac56a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51e0e00640 1 -- 172.21.15.175:0/2654571532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29a8c1580 con 0x55e29ac4bdd0 2024-07-29T00:40:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51db400640 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.468+0000 7f51e0e00640 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4c1b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009f50 tx=0x7f51d40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99fbba4053174ee server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.469+0000 7f51d1600640 1 -- 172.21.15.175:0/2654571532 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4004d20 con 0x55e29ac4bdd0 2024-07-29T00:40:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.469+0000 7f51d8c00640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51c00a2d70 0x7f51c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.469+0000 7f51d1600640 1 -- 172.21.15.175:0/2654571532 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d4004050 con 0x55e29ac4bdd0 2024-07-29T00:40:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.469+0000 7f51d1600640 1 -- 172.21.15.175:0/2654571532 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4005280 con 0x55e29ac4bdd0 2024-07-29T00:40:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.469+0000 7f51d8c00640 1 -- 172.21.15.175:0/3153535167 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f51c80cf690 con 0x7f51dc102050 2024-07-29T00:40:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.469+0000 7f51dbe00640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51c00a2d70 0x7f51c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51dbe00640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51c00a2d70 0x7f51c00a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006010 tx=0x7f51cc000bc0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 -- 172.21.15.175:0/2654571532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 msgr2=0x55e29ac4c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4c1b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009f50 tx=0x7f51d40118d0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 -- 172.21.15.175:0/2654571532 shutdown_connections 2024-07-29T00:40:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac56a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 --2- 172.21.15.175:0/2654571532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4c1b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 -- 172.21.15.175:0/2654571532 >> 172.21.15.175:0/2654571532 conn(0x55e29ac48360 msgr2=0x55e29ac48740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 -- 172.21.15.175:0/2654571532 shutdown_connections 2024-07-29T00:40:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.470+0000 7f51e59be740 1 -- 172.21.15.175:0/2654571532 wait complete. 2024-07-29T00:40:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.471+0000 7f51e59be740 1 -- start start 2024-07-29T00:40:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.471+0000 7f51e59be740 1 Processor -- start 2024-07-29T00:40:54.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac53e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac56fb0 0x55e29ac54900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e29ac59b80 con 0x55e29ac4c6f0 2024-07-29T00:40:54.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e29ab6cca0 con 0x55e29ac4bdd0 2024-07-29T00:40:54.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e29ac59d00 con 0x55e29ac56fb0 2024-07-29T00:40:54.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.472+0000 7f51e59be740 10 client.0 _ll_register_callbacks cb 0x55e29ab58e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:54.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51db400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac53e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac53e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58342/0 (socket says 172.21.15.175:58342) 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51dbe00640 1 -- 172.21.15.175:0/891129297 learned_addr learned my addr 172.21.15.175:0/891129297 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac56fb0 0x55e29ac54900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 -- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 msgr2=0x55e29ac53e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac53e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 -- 172.21.15.175:0/891129297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 msgr2=0x55e29ac543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 -- 172.21.15.175:0/891129297 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29a98b960 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51e0e00640 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac56fb0 0x55e29ac54900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51d4012fd0 tx=0x7f51d40044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b18f9ecdd5ad11 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.473+0000 7f51c6000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51c6000640 1 -- 172.21.15.175:0/891129297 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4003e40 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d4004050 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51c6000640 1 -- 172.21.15.175:0/891129297 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f51d400f480 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51c6000640 1 -- 172.21.15.175:0/891129297 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51d4033050 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51e59be740 10 client.26287 fetch_fsmap learned FSMap version 33 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e29ac65870 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.474+0000 7f51c6000640 1 -- 172.21.15.175:0/891129297 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d4004050 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51c6000640 1 -- 172.21.15.175:0/891129297 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f51d402e820 con 0x55e29ac56fb0 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51e59be740 10 client.26287 fetch_fsmap finished waiting for FSMap version 33 2024-07-29T00:40:54.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51e59be740 10 client.26287 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51e59be740 20 client.26287 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51e59be740 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29ac66c70 0x55e29ac69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51e59be740 4 client.26287 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29abbf230 con 0x55e29ac66c70 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.475+0000 7f51dbe00640 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29ac66c70 0x55e29ac69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.476+0000 7f51dbe00640 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29ac66c70 0x55e29ac69060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4555b241e4fe5926 server_cookie=2a3d21a9fde667c4 in_seq=0 out_seq=0 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.476+0000 7f51c6000640 10 client.26287 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.476+0000 7f51c6000640 1 -- 172.21.15.175:0/891129297 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e29abbf230 con 0x55e29ac66c70 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.476+0000 7f51c6000640 10 client.26287 handle_command_reply: tid=0 2024-07-29T00:40:54.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 client.26287 shutdown 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac56fb0 msgr2=0x55e29ac54900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac56fb0 0x55e29ac54900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51d4012fd0 tx=0x7f51d40044b0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 shutdown_connections 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29ac66c70 0x55e29ac69060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac56fb0 0x55e29ac54900 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4c6f0 0x55e29ac543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 --2- 172.21.15.175:0/891129297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac53e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51c6000640 0 client.26287 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 >> 172.21.15.175:0/891129297 conn(0x55e29ac48360 msgr2=0x55e29ac4e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 shutdown_connections 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 1 -- 172.21.15.175:0/891129297 wait complete. 2024-07-29T00:40:54.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.477+0000 7f51e59be740 20 client.26287 trim_cache size 0 max 16384 2024-07-29T00:40:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.496+0000 7f51e59be740 1 Processor -- start 2024-07-29T00:40:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.496+0000 7f51e59be740 1 -- start start 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.496+0000 7f51e59be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.496+0000 7f51e59be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.496+0000 7f51e59be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51e59be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00e800 con 0x55e29ac4bdd0 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51e59be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc010460 con 0x55e29ac56fb0 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51e59be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00eed0 con 0x55e29ac4c6f0 2024-07-29T00:40:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac6e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52786/0 (socket says 172.21.15.175:52786) 2024-07-29T00:40:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 -- 172.21.15.175:0/2765703822 learned_addr learned my addr 172.21.15.175:0/2765703822 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51e0e00640 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 -- 172.21.15.175:0/2765703822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 msgr2=0x55e29ac6ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac6ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 -- 172.21.15.175:0/2765703822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4bdd0 msgr2=0x55e29ac4e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51db400640 1 -- 172.21.15.175:0/2765703822 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29ac552f0 con 0x55e29ac4c6f0 2024-07-29T00:40:54.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.497+0000 7f51dbe00640 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:40:54.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.498+0000 7f51db400640 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac6e910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51c80985b0 tx=0x7f51c801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29cfafed5d37d0e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:54.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.498+0000 7f51d0c00640 1 -- 172.21.15.175:0/2765703822 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c809fee0 con 0x55e29ac4c6f0 2024-07-29T00:40:54.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.498+0000 7f51d0c00640 1 -- 172.21.15.175:0/2765703822 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c80a47a0 con 0x55e29ac4c6f0 2024-07-29T00:40:54.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.498+0000 7f51d0c00640 1 -- 172.21.15.175:0/2765703822 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c809b030 con 0x55e29ac4c6f0 2024-07-29T00:40:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 -- 172.21.15.175:0/2765703822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 msgr2=0x55e29ac6e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac6e910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51c80985b0 tx=0x7f51c801f000 comp rx=0 tx=0).stop 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 -- 172.21.15.175:0/2765703822 shutdown_connections 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac56fb0 0x55e29ac6ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac6e910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 --2- 172.21.15.175:0/2765703822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac4bdd0 0x55e29ac4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 -- 172.21.15.175:0/2765703822 >> 172.21.15.175:0/2765703822 conn(0x55e29ac48360 msgr2=0x55e29ac52750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 -- 172.21.15.175:0/2765703822 shutdown_connections 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.499+0000 7f51e59be740 1 -- 172.21.15.175:0/2765703822 wait complete. 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 -- start start 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 Processor -- start 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac56fb0 0x55e29acca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00fcd0 con 0x55e29ac56fb0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00f5a0 con 0x55e29ac4bdd0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.500+0000 7f51e59be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc00c7e0 con 0x55e29ac4c6f0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51e59be740 10 client.0 _ll_register_callbacks cb 0x7f51d4004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51e0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac56fb0 0x55e29acca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51e0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac56fb0 0x55e29acca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43958/0 (socket says 172.21.15.175:43958) 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51e0e00640 1 -- 172.21.15.175:0/540154118 learned_addr learned my addr 172.21.15.175:0/540154118 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51db400640 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51dbe00640 1 -- 172.21.15.175:0/540154118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 msgr2=0x55e29ac5ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51dbe00640 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac5ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51dbe00640 1 -- 172.21.15.175:0/540154118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac56fb0 msgr2=0x55e29acca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:40:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51dbe00640 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac56fb0 0x55e29acca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.501+0000 7f51dbe00640 1 -- 172.21.15.175:0/540154118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29accaea0 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51dbe00640 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac5b550 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55e29ac69880 tx=0x7f51cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0be62f463ae289a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51c6000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51c6000640 1 -- 172.21.15.175:0/540154118 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc035040 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51c6000640 1 -- 172.21.15.175:0/540154118 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f51cc004c50 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c80a47a0 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51c6000640 1 -- 172.21.15.175:0/540154118 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51cc004e50 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51e59be740 10 client.26333 fetch_fsmap learned FSMap version 33 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e29acd22c0 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.502+0000 7f51c6000640 1 -- 172.21.15.175:0/540154118 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c80a47a0 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51c6000640 1 -- 172.21.15.175:0/540154118 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f51cc003940 con 0x55e29ac4bdd0 2024-07-29T00:40:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51e59be740 10 client.26333 fetch_fsmap finished waiting for FSMap version 33 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51e59be740 10 client.26333 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51e59be740 20 client.26333 populate_metadata read hostname 'smithi175' 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51e59be740 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29acd29f0 0x55e29acd4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51e59be740 4 client.26333 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2642d471", "release": true, "await": true}] 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2642d471", "release": true, "await": true}) -- 0x55e29abb9740 con 0x55e29acd29f0 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.503+0000 7f51db400640 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29acd29f0 0x55e29acd4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.504+0000 7f51c6000640 10 client.26333 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:54.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.504+0000 7f51db400640 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29acd29f0 0x55e29acd4e10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b35152b272dcf0c server_cookie=da5837a741fe3d60 in_seq=0 out_seq=0 2024-07-29T00:40:54.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.515+0000 7f51c6000640 1 -- 172.21.15.175:0/540154118 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e29abb9740 con 0x55e29acd29f0 2024-07-29T00:40:54.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.515+0000 7f51c6000640 10 client.26333 handle_command_reply: tid=0 2024-07-29T00:40:54.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.515+0000 7f51e59be740 1 client.26333 shutdown 2024-07-29T00:40:54.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 msgr2=0x55e29ac5b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac5b550 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55e29ac69880 tx=0x7f51cc009260 comp rx=0 tx=0).stop 2024-07-29T00:40:54.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 shutdown_connections 2024-07-29T00:40:54.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55e29acd29f0 0x55e29acd4e10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e29ac56fb0 0x55e29acca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29ac4c6f0 0x55e29ac5ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 --2- 172.21.15.175:0/540154118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e29ac4bdd0 0x55e29ac5b550 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 >> 172.21.15.175:0/540154118 conn(0x55e29ac48360 msgr2=0x55e29ac46ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:54.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51c6000640 0 client.26333 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:40:54.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.516+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 shutdown_connections 2024-07-29T00:40:54.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.517+0000 7f51e59be740 1 -- 172.21.15.175:0/540154118 wait complete. 2024-07-29T00:40:54.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.517+0000 7f51e59be740 20 client.26333 trim_cache size 0 max 16384 2024-07-29T00:40:54.520 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:40:54.520 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 33, 2024-07-29T00:40:54.520 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:40:54.521 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 16, 2024-07-29T00:40:54.521 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:40:54.521 INFO:teuthology.orchestra.run.smithi175.stdout: "2642d471": { 2024-07-29T00:40:54.521 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 16, 2024-07-29T00:40:54.521 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:40:54.521 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:54.522 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:40:54.522 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:54.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:40:54.522 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:40:54.522 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 139.8, 2024-07-29T00:40:54.523 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:40:54.523 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:40:54.523 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:40:54.523 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:40:54.523 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:40:54.523 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:40:54.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:54.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:54.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:54.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:54.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:40:54.524 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:40:54.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51c00a2d70 msgr2=0x7f51c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51c00a2d70 0x7f51c00a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006010 tx=0x7f51cc000bc0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 msgr2=0x7f51dc19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:40:54.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 0x7f51dc19a6e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51c800e9f0 tx=0x7f51c800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 shutdown_connections 2024-07-29T00:40:54.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51dc1076b0 0x7f51dc19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f51c00a2d70 0x7f51c00a5230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc102050 0x7f51dc19a6e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 --2- 172.21.15.175:0/3153535167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51dc1016b0 0x7f51dc19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:40:54.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 >> 172.21.15.175:0/3153535167 conn(0x7f51dc0fd3f0 msgr2=0x7f51dc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:40:54.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 shutdown_connections 2024-07-29T00:40:54.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:40:54.518+0000 7f51e1800640 1 -- 172.21.15.175:0/3153535167 wait complete. 2024-07-29T00:40:54.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113659 1.24 MB/sec execute 45 sec latency 17103.337 ms 2024-07-29T00:40:55.529 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:55 smithi175 ceph-mon[32138]: pgmap v135: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-07-29T00:40:55.533 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2642d471', total seconds elapsed: 24.5 2024-07-29T00:40:55.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:55 smithi204 ceph-mon[37368]: pgmap v135: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-07-29T00:40:55.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:55 smithi181 ceph-mon[36706]: pgmap v135: 97 pgs: 97 active+clean; 72 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-07-29T00:40:55.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113700 1.21 MB/sec execute 46 sec latency 1042.875 ms 2024-07-29T00:40:56.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113700 1.19 MB/sec execute 47 sec latency 1185.259 ms 2024-07-29T00:40:57.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:57 smithi175 ceph-mon[32138]: pgmap v136: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-07-29T00:40:57.551 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:57 smithi181 ceph-mon[36706]: pgmap v136: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-07-29T00:40:57.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:57 smithi204 ceph-mon[37368]: pgmap v136: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-07-29T00:40:57.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113700 1.16 MB/sec execute 48 sec latency 2185.364 ms 2024-07-29T00:40:58.885 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113700 1.14 MB/sec execute 49 sec latency 3185.462 ms 2024-07-29T00:40:59.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:59 smithi175 ceph-mon[32138]: pgmap v137: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-07-29T00:40:59.535 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:40:59 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:59.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:59 smithi204 ceph-mon[37368]: pgmap v137: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-07-29T00:40:59.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:40:59 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:59.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:59 smithi181 ceph-mon[36706]: pgmap v137: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-07-29T00:40:59.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:40:59 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:40:59.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113700 1.12 MB/sec execute 50 sec latency 4185.592 ms 2024-07-29T00:41:00.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 113773 1.11 MB/sec execute 51 sec latency 5055.718 ms 2024-07-29T00:41:01.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:01 smithi204 ceph-mon[37368]: pgmap v138: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-07-29T00:41:01.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:01 smithi181 ceph-mon[36706]: pgmap v138: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-07-29T00:41:01.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:01 smithi175 ceph-mon[32138]: pgmap v138: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-07-29T00:41:01.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 114150 1.10 MB/sec execute 52 sec latency 19.760 ms 2024-07-29T00:41:02.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 115604 1.18 MB/sec execute 53 sec latency 23.780 ms 2024-07-29T00:41:03.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:03 smithi181 ceph-mon[36706]: pgmap v139: 97 pgs: 97 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 834 KiB/s wr, 54 op/s 2024-07-29T00:41:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:03 smithi175 ceph-mon[32138]: pgmap v139: 97 pgs: 97 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 834 KiB/s wr, 54 op/s 2024-07-29T00:41:03.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:03 smithi204 ceph-mon[37368]: pgmap v139: 97 pgs: 97 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 834 KiB/s wr, 54 op/s 2024-07-29T00:41:03.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 117077 1.38 MB/sec execute 54 sec latency 13.728 ms 2024-07-29T00:41:04.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:04 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:04.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:04 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:04.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:04 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:04.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 117246 1.36 MB/sec execute 55 sec latency 828.563 ms 2024-07-29T00:41:05.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:05 smithi181 ceph-mon[36706]: pgmap v140: 97 pgs: 97 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 826 KiB/s wr, 48 op/s 2024-07-29T00:41:05.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:05 smithi175 ceph-mon[32138]: pgmap v140: 97 pgs: 97 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 826 KiB/s wr, 48 op/s 2024-07-29T00:41:05.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:05 smithi204 ceph-mon[37368]: pgmap v140: 97 pgs: 97 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 826 KiB/s wr, 48 op/s 2024-07-29T00:41:05.886 INFO:tasks.workunit.client.0.smithi175.stdout: 1 117351 1.36 MB/sec execute 56 sec latency 1583.857 ms 2024-07-29T00:41:06.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 117800 1.34 MB/sec execute 57 sec latency 20.899 ms 2024-07-29T00:41:07.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:07 smithi181 ceph-mon[36706]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-07-29T00:41:07.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:07 smithi175 ceph-mon[32138]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-07-29T00:41:07.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:07 smithi204 ceph-mon[37368]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-07-29T00:41:07.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 119508 1.44 MB/sec execute 58 sec latency 15.611 ms 2024-07-29T00:41:08.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 120793 1.59 MB/sec execute 59 sec latency 43.688 ms 2024-07-29T00:41:09.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:09 smithi181 ceph-mon[36706]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-07-29T00:41:09.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:09 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:09.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:09 smithi175 ceph-mon[32138]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-07-29T00:41:09.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:09 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:09 smithi204 ceph-mon[37368]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-07-29T00:41:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:09 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:09.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 120793 1.57 MB/sec execute 60 sec latency 1043.831 ms 2024-07-29T00:41:10.511 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:41:10.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 120921 1.56 MB/sec execute 61 sec latency 1802.441 ms 2024-07-29T00:41:11.003 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:11.404 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:11 smithi175 ceph-mon[32138]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-07-29T00:41:11.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:11 smithi181 ceph-mon[36706]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.811+0000 7fea5da00640 1 -- 172.21.15.175:0/801279755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58100f80 msgr2=0x7fea5810d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.811+0000 7fea5da00640 1 --2- 172.21.15.175:0/801279755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58100f80 0x7fea5810d490 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fea50009cd0 tx=0x7fea5002c7c0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 -- 172.21.15.175:0/801279755 shutdown_connections 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 --2- 172.21.15.175:0/801279755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58100f80 0x7fea5810d490 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 --2- 172.21.15.175:0/801279755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea581005c0 0x7fea58100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 --2- 172.21.15.175:0/801279755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea58106650 0x7fea58106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 -- 172.21.15.175:0/801279755 >> 172.21.15.175:0/801279755 conn(0x7fea580fc300 msgr2=0x7fea580fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:11.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 -- 172.21.15.175:0/801279755 shutdown_connections 2024-07-29T00:41:11.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.812+0000 7fea5da00640 1 -- 172.21.15.175:0/801279755 wait complete. 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.813+0000 7fea5da00640 1 Processor -- start 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.813+0000 7fea5da00640 1 -- start start 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 0x7fea58073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea58100f80 0x7fea58070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea58106650 0x7fea58070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810f8b0 con 0x7fea58106650 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5da00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810f730 con 0x7fea58100f80 2024-07-29T00:41:11.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810fa30 con 0x7fea581005c0 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 0x7fea58073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea58100f80 0x7fea58070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 0x7fea58073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34522/0 (socket says 172.21.15.175:34522) 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 -- 172.21.15.175:0/2005581464 learned_addr learned my addr 172.21.15.175:0/2005581464 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea5d000640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea58106650 0x7fea58070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 -- 172.21.15.175:0/2005581464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea58100f80 msgr2=0x7fea58070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea58100f80 0x7fea58070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 -- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea58106650 msgr2=0x7fea58070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea58106650 0x7fea58070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57e00640 1 -- 172.21.15.175:0/2005581464 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea580718b0 con 0x7fea581005c0 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.814+0000 7fea57400640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea58100f80 0x7fea58070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.815+0000 7fea57e00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 0x7fea58073a00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d950 tx=0x7fea4000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:11.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.815+0000 7fea54c00640 1 -- 172.21.15.175:0/2005581464 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40017030 con 0x7fea581005c0 2024-07-29T00:41:11.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.815+0000 7fea54c00640 1 -- 172.21.15.175:0/2005581464 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40004020 con 0x7fea581005c0 2024-07-29T00:41:11.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.815+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581aee70 con 0x7fea581005c0 2024-07-29T00:41:11.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.815+0000 7fea54c00640 1 -- 172.21.15.175:0/2005581464 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400047a0 con 0x7fea581005c0 2024-07-29T00:41:11.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.815+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581af320 con 0x7fea581005c0 2024-07-29T00:41:11.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.819+0000 7fea54c00640 1 -- 172.21.15.175:0/2005581464 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fea400041d0 con 0x7fea581005c0 2024-07-29T00:41:11.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.819+0000 7fea61c13740 1 Processor -- start 2024-07-29T00:41:11.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.819+0000 7fea61c13740 1 -- start start 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.819+0000 7fea61c13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x56130721fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.819+0000 7fea61c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561307220430 0x56130722a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea61c13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130722acf0 0x56130722d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea61c13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561307184ad0 con 0x56130721fb10 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea61c13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56130711caf0 con 0x56130722acf0 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea61c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561307180a90 con 0x561307220430 2024-07-29T00:41:11.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea5d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x56130721fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130722acf0 0x56130722d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea5d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x56130721fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45586/0 (socket says 172.21.15.175:45586) 2024-07-29T00:41:11.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea5d000640 1 -- 172.21.15.175:0/1207696247 learned_addr learned my addr 172.21.15.175:0/1207696247 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:11.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57e00640 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561307220430 0x56130722a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57400640 1 -- 172.21.15.175:0/1207696247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561307220430 msgr2=0x56130722a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57400640 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561307220430 0x56130722a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57400640 1 -- 172.21.15.175:0/1207696247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 msgr2=0x56130721fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57400640 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x56130721fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.820+0000 7fea57400640 1 -- 172.21.15.175:0/1207696247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561306ef63c0 con 0x56130722acf0 2024-07-29T00:41:11.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea57400640 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130722acf0 0x56130722d0e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea58072300 tx=0x7fea480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b118b880d7921b23 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea54c00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fea3c0a2d70 0x7fea3c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea4d600640 1 -- 172.21.15.175:0/1207696247 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48004310 con 0x56130722acf0 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea4d600640 1 -- 172.21.15.175:0/1207696247 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea480044b0 con 0x56130722acf0 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea4d600640 1 -- 172.21.15.175:0/1207696247 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48003650 con 0x56130722acf0 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea54c00640 1 -- 172.21.15.175:0/2005581464 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fea400ced40 con 0x7fea581005c0 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.821+0000 7fea5d000640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fea3c0a2d70 0x7fea3c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 -- 172.21.15.175:0/1207696247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130722acf0 msgr2=0x56130722d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130722acf0 0x56130722d0e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea58072300 tx=0x7fea480118d0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 -- 172.21.15.175:0/1207696247 shutdown_connections 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130722acf0 0x56130722d0e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561307220430 0x56130722a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 --2- 172.21.15.175:0/1207696247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x56130721fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 -- 172.21.15.175:0/1207696247 >> 172.21.15.175:0/1207696247 conn(0x56130721c0a0 msgr2=0x56130721c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:11.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea5d000640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fea3c0a2d70 0x7fea3c0a5230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fea50006010 tx=0x7fea50032340 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:11.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 -- 172.21.15.175:0/1207696247 shutdown_connections 2024-07-29T00:41:11.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.822+0000 7fea61c13740 1 -- 172.21.15.175:0/1207696247 wait complete. 2024-07-29T00:41:11.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.823+0000 7fea61c13740 1 -- start start 2024-07-29T00:41:11.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.823+0000 7fea61c13740 1 Processor -- start 2024-07-29T00:41:11.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 0x561307227b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561307220430 0x5613072280d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x561307228610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56130722d8c0 con 0x561307220430 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56130713fbc0 con 0x56130721fb10 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56130722da40 con 0x56130722acf0 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea61c13740 10 client.0 _ll_register_callbacks cb 0x56130712bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea57e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 0x561307227b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea57400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561307220430 0x5613072280d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea57e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 0x561307227b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48444/0 (socket says 172.21.15.175:48444) 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea57e00640 1 -- 172.21.15.175:0/3256678046 learned_addr learned my addr 172.21.15.175:0/3256678046 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea5d000640 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x561307228610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.824+0000 7fea57e00640 1 -- 172.21.15.175:0/3256678046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 msgr2=0x561307228610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea57e00640 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x561307228610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea57e00640 1 -- 172.21.15.175:0/3256678046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561307220430 msgr2=0x5613072280d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea57e00640 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561307220430 0x5613072280d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea57e00640 1 -- 172.21.15.175:0/3256678046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561307236d80 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea57e00640 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 0x561307227b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fea40009fd0 tx=0x7fea400051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8994dc31f1b7951f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea46000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea46000640 1 -- 172.21.15.175:0/3256678046 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4009ab40 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea46000640 1 -- 172.21.15.175:0/3256678046 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fea4009f1e0 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea480044b0 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea46000640 1 -- 172.21.15.175:0/3256678046 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fea400cc890 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea61c13740 10 client.26345 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.825+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5613072396f0 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea46000640 1 -- 172.21.15.175:0/3256678046 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea480044b0 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea46000640 1 -- 172.21.15.175:0/3256678046 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fea400a39c0 con 0x56130721fb10 2024-07-29T00:41:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea61c13740 10 client.26345 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:11.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea61c13740 10 client.26345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:11.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea61c13740 20 client.26345 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:11.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea61c13740 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56130723abb0 0x56130723d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea61c13740 4 client.26345 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:11.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561307192040 con 0x56130723abb0 2024-07-29T00:41:11.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.826+0000 7fea57400640 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56130723abb0 0x56130723d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.827+0000 7fea57400640 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56130723abb0 0x56130723d020 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3640f936f08c4a server_cookie=1ef0c179a499bca8 in_seq=0 out_seq=0 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.827+0000 7fea46000640 10 client.26345 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea46000640 1 -- 172.21.15.175:0/3256678046 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561307192040 con 0x56130723abb0 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea46000640 10 client.26345 handle_command_reply: tid=0 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 client.26345 shutdown 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 msgr2=0x561307227b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 0x561307227b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fea40009fd0 tx=0x7fea400051f0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 shutdown_connections 2024-07-29T00:41:11.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56130723abb0 0x56130723d020 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x561307228610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561307220430 0x5613072280d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea46000640 0 client.26345 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 --2- 172.21.15.175:0/3256678046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56130721fb10 0x561307227b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 >> 172.21.15.175:0/3256678046 conn(0x56130721c0a0 msgr2=0x561307221dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 shutdown_connections 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 1 -- 172.21.15.175:0/3256678046 wait complete. 2024-07-29T00:41:11.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.828+0000 7fea61c13740 20 client.26345 trim_cache size 0 max 16384 2024-07-29T00:41:11.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:11 smithi204 ceph-mon[37368]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-07-29T00:41:11.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 Processor -- start 2024-07-29T00:41:11.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 -- start start 2024-07-29T00:41:11.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 0x561307229a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x561307228430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130722acf0 0x561307242910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b7e90 con 0x56130722acf0 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b9a30 con 0x561307220430 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.849+0000 7fea61c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b8560 con 0x56130721fb10 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 0x561307229a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x561307228430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 0x561307229a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34544/0 (socket says 172.21.15.175:34544) 2024-07-29T00:41:11.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x561307228430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48454/0 (socket says 172.21.15.175:48454) 2024-07-29T00:41:11.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 -- 172.21.15.175:0/2504089147 learned_addr learned my addr 172.21.15.175:0/2504089147 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:11.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea5d000640 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130722acf0 0x561307242910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 -- 172.21.15.175:0/2504089147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 msgr2=0x561307228430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x561307228430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 -- 172.21.15.175:0/2504089147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130722acf0 msgr2=0x561307242910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130722acf0 0x561307242910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 -- 172.21.15.175:0/2504089147 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130721d230 con 0x56130721fb10 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57400640 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x561307228430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea5d000640 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130722acf0 0x561307242910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.850+0000 7fea57e00640 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 0x561307229a60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea400045f0 tx=0x7fea40009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce69f25761a63803 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.851+0000 7fea4cc00640 1 -- 172.21.15.175:0/2504089147 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40014070 con 0x56130721fb10 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.851+0000 7fea4cc00640 1 -- 172.21.15.175:0/2504089147 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4009b8e0 con 0x56130721fb10 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.851+0000 7fea4cc00640 1 -- 172.21.15.175:0/2504089147 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400cc690 con 0x56130721fb10 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 -- 172.21.15.175:0/2504089147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 msgr2=0x561307229a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 0x561307229a60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea400045f0 tx=0x7fea40009740 comp rx=0 tx=0).stop 2024-07-29T00:41:11.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 -- 172.21.15.175:0/2504089147 shutdown_connections 2024-07-29T00:41:11.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130722acf0 0x561307242910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x561307228430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 --2- 172.21.15.175:0/2504089147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130721fb10 0x561307229a60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 -- 172.21.15.175:0/2504089147 >> 172.21.15.175:0/2504089147 conn(0x56130721c0a0 msgr2=0x56130721b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:11.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 -- 172.21.15.175:0/2504089147 shutdown_connections 2024-07-29T00:41:11.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.852+0000 7fea61c13740 1 -- 172.21.15.175:0/2504089147 wait complete. 2024-07-29T00:41:11.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.853+0000 7fea61c13740 1 -- start start 2024-07-29T00:41:11.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.853+0000 7fea61c13740 1 Processor -- start 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x5613072335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x56130722e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x56130722f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b9360 con 0x56130721fb10 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b8c30 con 0x561307220430 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b5e70 con 0x56130722acf0 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea61c13740 10 client.0 _ll_register_callbacks cb 0x7fea400d24e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x56130722e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x56130722e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48462/0 (socket says 172.21.15.175:48462) 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 -- 172.21.15.175:0/981840178 learned_addr learned my addr 172.21.15.175:0/981840178 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:11.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea5d000640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x56130722f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57e00640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x5613072335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 -- 172.21.15.175:0/981840178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 msgr2=0x56130722f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x56130722f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 -- 172.21.15.175:0/981840178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 msgr2=0x5613072335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x5613072335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.854+0000 7fea57400640 1 -- 172.21.15.175:0/981840178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613072a48e0 con 0x561307220430 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea57400640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x56130722e8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea480058c0 tx=0x7fea48005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b1e74350c673f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea46000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea46000640 1 -- 172.21.15.175:0/981840178 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48032070 con 0x561307220430 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea46000640 1 -- 172.21.15.175:0/981840178 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fea48005410 con 0x561307220430 2024-07-29T00:41:11.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4009b8e0 con 0x561307220430 2024-07-29T00:41:11.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea46000640 1 -- 172.21.15.175:0/981840178 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fea48004d40 con 0x561307220430 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea61c13740 10 client.26351 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.855+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5613072a5b00 con 0x561307220430 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea46000640 1 -- 172.21.15.175:0/981840178 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4009b8e0 con 0x561307220430 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea46000640 1 -- 172.21.15.175:0/981840178 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fea48035020 con 0x561307220430 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea61c13740 10 client.26351 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea61c13740 10 client.26351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea61c13740 20 client.26351 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea61c13740 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5613072a6130 0x5613072a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea61c13740 4 client.26351 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56130718c550 con 0x5613072a6130 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.856+0000 7fea57e00640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5613072a6130 0x5613072a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea57e00640 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5613072a6130 0x5613072a8550 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72bbbe603cb22754 server_cookie=5204c5eca6aa9697 in_seq=0 out_seq=0 2024-07-29T00:41:11.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea46000640 10 client.26351 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea46000640 1 -- 172.21.15.175:0/981840178 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56130718c550 con 0x5613072a6130 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea46000640 10 client.26351 handle_command_reply: tid=0 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea61c13740 1 client.26351 shutdown 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 msgr2=0x56130722e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.857+0000 7fea61c13740 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x56130722e8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea480058c0 tx=0x7fea48005dc0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 shutdown_connections 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5613072a6130 0x5613072a8550 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56130722acf0 0x56130722f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561307220430 0x56130722e8f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 --2- 172.21.15.175:0/981840178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56130721fb10 0x5613072335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 >> 172.21.15.175:0/981840178 conn(0x56130721c0a0 msgr2=0x561307235b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:11.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea46000640 0 client.26351 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:11.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 shutdown_connections 2024-07-29T00:41:11.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 1 -- 172.21.15.175:0/981840178 wait complete. 2024-07-29T00:41:11.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.858+0000 7fea61c13740 20 client.26351 trim_cache size 0 max 16384 2024-07-29T00:41:11.860 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:41:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:41:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:41:11.861 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:41:11.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fea3c0a2d70 msgr2=0x7fea3c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fea3c0a2d70 0x7fea3c0a5230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fea50006010 tx=0x7fea50032340 comp rx=0 tx=0).stop 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 msgr2=0x7fea58073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 0x7fea58073a00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d950 tx=0x7fea4000de10 comp rx=0 tx=0).stop 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 shutdown_connections 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fea3c0a2d70 0x7fea3c0a5230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea58106650 0x7fea58070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea58100f80 0x7fea58070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 --2- 172.21.15.175:0/2005581464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea581005c0 0x7fea58073a00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 >> 172.21.15.175:0/2005581464 conn(0x7fea580fc300 msgr2=0x7fea580fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 shutdown_connections 2024-07-29T00:41:11.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:11.859+0000 7fea5da00640 1 -- 172.21.15.175:0/2005581464 wait complete. 2024-07-29T00:41:11.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 121692 1.57 MB/sec execute 62 sec latency 17.604 ms 2024-07-29T00:41:12.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 123562 1.71 MB/sec execute 63 sec latency 16.683 ms 2024-07-29T00:41:13.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:41:13.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:41:13.538 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:13.689 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:13 smithi175 ceph-mon[32138]: pgmap v144: 97 pgs: 97 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-07-29T00:41:13.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:13 smithi181 ceph-mon[36706]: pgmap v144: 97 pgs: 97 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-07-29T00:41:13.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:13 smithi204 ceph-mon[37368]: pgmap v144: 97 pgs: 97 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-07-29T00:41:13.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 124629 1.79 MB/sec execute 64 sec latency 12.836 ms 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 -- 172.21.15.175:0/1389525997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d010be60 msgr2=0x7f45d010e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 --2- 172.21.15.175:0/1389525997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d010be60 0x7f45d010e300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009be0 tx=0x7f45c802a750 comp rx=0 tx=0).stop 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 -- 172.21.15.175:0/1389525997 shutdown_connections 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 --2- 172.21.15.175:0/1389525997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d010be60 0x7f45d010e300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 --2- 172.21.15.175:0/1389525997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45d0103350 0x7f45d010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 --2- 172.21.15.175:0/1389525997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d0102a30 0x7f45d0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 -- 172.21.15.175:0/1389525997 >> 172.21.15.175:0/1389525997 conn(0x7f45d00fc490 msgr2=0x7f45d00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.192+0000 7f45d7400640 1 -- 172.21.15.175:0/1389525997 shutdown_connections 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.193+0000 7f45d7400640 1 -- 172.21.15.175:0/1389525997 wait complete. 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 Processor -- start 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 -- start start 2024-07-29T00:41:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45d0102a30 0x7f45d019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 0x7f45d019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d010be60 0x7f45d019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0111c20 con 0x7f45d0102a30 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0111aa0 con 0x7f45d0103350 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.194+0000 7f45d7400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0111da0 con 0x7f45d010be60 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 0x7f45d019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45d0102a30 0x7f45d019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 0x7f45d019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48480/0 (socket says 172.21.15.175:48480) 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 -- 172.21.15.175:0/2947040176 learned_addr learned my addr 172.21.15.175:0/2947040176 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45d5400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d010be60 0x7f45d019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 -- 172.21.15.175:0/2947040176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d010be60 msgr2=0x7f45d019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d010be60 0x7f45d019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 -- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45d0102a30 msgr2=0x7f45d019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45d0102a30 0x7f45d019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 -- 172.21.15.175:0/2947040176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d019f270 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.195+0000 7f45cfe00640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 0x7f45d019a800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00e9f0 tx=0x7f45bc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.196+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00cab0 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.196+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d019f560 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.196+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45bc00cc50 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.196+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0187e0 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.196+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a6e40 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.197+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d0102e10 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.200+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f45bc018b90 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.201+0000 7f45cd600640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45b80a2d70 0x7f45b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.201+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f45bc0cfba0 con 0x7f45d0103350 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.201+0000 7f45d4a00640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45b80a2d70 0x7f45b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.202+0000 7f45d4a00640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45b80a2d70 0x7f45b80a5230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45c4005fd0 tx=0x7f45c4009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.202+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f45bc09c740 con 0x7f45d0103350 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.441+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45d01a70e0 con 0x7f45d0103350 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.442+0000 7f45cd600640 1 -- 172.21.15.175:0/2947040176 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11997 (secure 0 0 0) 0x7f45bc09c040 con 0x7f45d0103350 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":36,"btime":"2024-07-29T00:41:08:370423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:41:07.438279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 36 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45b80a2d70 msgr2=0x7f45b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45b80a2d70 0x7f45b80a5230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45c4005fd0 tx=0x7f45c4009260 comp rx=0 tx=0).stop 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 msgr2=0x7f45d019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 0x7f45d019a800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00e9f0 tx=0x7f45bc00edf0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 shutdown_connections 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d010be60 0x7f45d019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d0103350 0x7f45d019a800 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45b80a2d70 0x7f45b80a5230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 --2- 172.21.15.175:0/2947040176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45d0102a30 0x7f45d019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 >> 172.21.15.175:0/2947040176 conn(0x7f45d00fc490 msgr2=0x7f45d01005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 shutdown_connections 2024-07-29T00:41:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:14.446+0000 7f45d7400640 1 -- 172.21.15.175:0/2947040176 wait complete. 2024-07-29T00:41:14.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 124775 1.76 MB/sec execute 65 sec latency 820.318 ms 2024-07-29T00:41:15.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:41:15.720 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:15 smithi175 ceph-mon[32138]: pgmap v145: 97 pgs: 97 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 KiB/s wr, 103 op/s 2024-07-29T00:41:15.720 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:15 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2947040176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:41:15.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:15 smithi181 ceph-mon[36706]: pgmap v145: 97 pgs: 97 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 KiB/s wr, 103 op/s 2024-07-29T00:41:15.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:15 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2947040176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:41:15.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:15 smithi204 ceph-mon[37368]: pgmap v145: 97 pgs: 97 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 KiB/s wr, 103 op/s 2024-07-29T00:41:15.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:15 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2947040176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:41:15.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 124775 1.73 MB/sec execute 66 sec latency 1820.437 ms 2024-07-29T00:41:16.102 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:16.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 -- 172.21.15.175:0/3433982020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6088108140 msgr2=0x7f6088108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 --2- 172.21.15.175:0/3433982020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6088108140 0x7f6088108520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6078009900 tx=0x7f607802a410 comp rx=0 tx=0).stop 2024-07-29T00:41:16.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 -- 172.21.15.175:0/3433982020 shutdown_connections 2024-07-29T00:41:16.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 --2- 172.21.15.175:0/3433982020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f608810d220 0x7f6088114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 --2- 172.21.15.175:0/3433982020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108af0 0x7f608810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 --2- 172.21.15.175:0/3433982020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6088108140 0x7f6088108520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 -- 172.21.15.175:0/3433982020 >> 172.21.15.175:0/3433982020 conn(0x7f6088077890 msgr2=0x7f6088079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 -- 172.21.15.175:0/3433982020 shutdown_connections 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.778+0000 7f608dc00640 1 -- 172.21.15.175:0/3433982020 wait complete. 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.779+0000 7f608dc00640 1 Processor -- start 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.779+0000 7f608dc00640 1 -- start start 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f608dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108140 0x7f608819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f608dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6088108af0 0x7f608819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f608dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 0x7f60881a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f608dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6088118270 con 0x7f6088108af0 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f608dc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60881180f0 con 0x7f6088108140 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f608dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60881183f0 con 0x7f608810d220 2024-07-29T00:41:16.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6086a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108140 0x7f608819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f607fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6088108af0 0x7f608819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 0x7f60881a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 0x7f60881a2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34586/0 (socket says 172.21.15.175:34586) 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 -- 172.21.15.175:0/2839816257 learned_addr learned my addr 172.21.15.175:0/2839816257 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6086a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108140 0x7f608819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48488/0 (socket says 172.21.15.175:48488) 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 -- 172.21.15.175:0/2839816257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108140 msgr2=0x7f608819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108140 0x7f608819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 -- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6088108af0 msgr2=0x7f608819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6088108af0 0x7f608819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.780+0000 7f6087400640 1 -- 172.21.15.175:0/2839816257 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60881a36a0 con 0x7f608810d220 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.781+0000 7f6087400640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 0x7f60881a2fc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f608000ea60 tx=0x7f608000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.781+0000 7f607f400640 1 -- 172.21.15.175:0/2839816257 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608000ce40 con 0x7f608810d220 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.781+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60881a3990 con 0x7f608810d220 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.781+0000 7f607f400640 1 -- 172.21.15.175:0/2839816257 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6080005020 con 0x7f608810d220 2024-07-29T00:41:16.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.781+0000 7f607f400640 1 -- 172.21.15.175:0/2839816257 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080010690 con 0x7f608810d220 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.781+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60881ab270 con 0x7f608810d220 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f607f400640 1 -- 172.21.15.175:0/2839816257 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f60800051d0 con 0x7f608810d220 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 Processor -- start 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 -- start start 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581264ef0 0x55c581263440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581263dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581270b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5811c9ab0 con 0x55c58126e720 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c581161a80 con 0x55c581263980 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.785+0000 7f6090872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c5811c5a70 con 0x55c581264ef0 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581263dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f607fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581264ef0 0x55c581263440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581263dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48502/0 (socket says 172.21.15.175:48502) 2024-07-29T00:41:16.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f607fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581264ef0 0x55c581263440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34588/0 (socket says 172.21.15.175:34588) 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 -- 172.21.15.175:0/2754177353 learned_addr learned my addr 172.21.15.175:0/2754177353 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f607fe00640 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581270b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 -- 172.21.15.175:0/2754177353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581264ef0 msgr2=0x55c581263440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581264ef0 0x55c581263440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 -- 172.21.15.175:0/2754177353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 msgr2=0x55c581270b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581270b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 -- 172.21.15.175:0/2754177353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c580f3b3c0 con 0x55c581263980 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f607f400640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f606c0a2d70 0x7f606c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f6086a00640 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581263dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6078009900 tx=0x7f6078004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a9349286d04324d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.786+0000 7f607f400640 1 -- 172.21.15.175:0/2839816257 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f60800cf910 con 0x7f608810d220 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f607fe00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f606c0a2d70 0x7f606c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f6077e00640 1 -- 172.21.15.175:0/2754177353 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078035040 con 0x55c581263980 2024-07-29T00:41:16.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f6077e00640 1 -- 172.21.15.175:0/2754177353 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6078005520 con 0x55c581263980 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f6077e00640 1 -- 172.21.15.175:0/2754177353 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078003850 con 0x55c581263980 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f6090872740 1 -- 172.21.15.175:0/2754177353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 msgr2=0x55c581263dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f607fe00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f606c0a2d70 0x7f606c0a5230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c581264420 tx=0x7f607000b390 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.787+0000 7f6090872740 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581263dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6078009900 tx=0x7f6078004040 comp rx=0 tx=0).stop 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 -- 172.21.15.175:0/2754177353 shutdown_connections 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581270b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581263dc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 --2- 172.21.15.175:0/2754177353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581264ef0 0x55c581263440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 -- 172.21.15.175:0/2754177353 >> 172.21.15.175:0/2754177353 conn(0x55c58122c950 msgr2=0x55c581260010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 -- 172.21.15.175:0/2754177353 shutdown_connections 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 -- 172.21.15.175:0/2754177353 wait complete. 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 -- start start 2024-07-29T00:41:16.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.788+0000 7f6090872740 1 Processor -- start 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c58126bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c58126e720 0x55c58126c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c581271460 con 0x55c581264ef0 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c581184c10 con 0x55c58126e720 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c5812715e0 con 0x55c581263980 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6090872740 10 client.0 _ll_register_callbacks cb 0x55c58114da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6086a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f607fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c58126bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6086a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34604/0 (socket says 172.21.15.175:34604) 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6086a00640 1 -- 172.21.15.175:0/3617689877 learned_addr learned my addr 172.21.15.175:0/3617689877 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.789+0000 7f6087400640 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c58126e720 0x55c58126c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6086a00640 1 -- 172.21.15.175:0/3617689877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c58126e720 msgr2=0x55c58126c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6086a00640 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c58126e720 0x55c58126c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6086a00640 1 -- 172.21.15.175:0/3617689877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 msgr2=0x55c58126bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6086a00640 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c58126bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6086a00640 1 -- 172.21.15.175:0/3617689877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5811881e0 con 0x55c581263980 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6086a00640 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126b680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6078004970 tx=0x7f607802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb2908f2b99a442 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6074c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.790+0000 7f6074c00640 1 -- 172.21.15.175:0/3617689877 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078053060 con 0x55c581263980 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6078005520 con 0x55c581263980 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6074c00640 1 -- 172.21.15.175:0/3617689877 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6078004e60 con 0x55c581263980 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6074c00640 1 -- 172.21.15.175:0/3617689877 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6078035050 con 0x55c581263980 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6090872740 10 client.26329 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c581271460 con 0x55c581263980 2024-07-29T00:41:16.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6074c00640 1 -- 172.21.15.175:0/3617689877 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6078005520 con 0x55c581263980 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6074c00640 1 -- 172.21.15.175:0/3617689877 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f6078031a90 con 0x55c581263980 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6090872740 10 client.26329 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6090872740 10 client.26329 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.791+0000 7f6090872740 20 client.26329 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.792+0000 7f6090872740 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c58127e380 0x55c581280770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.792+0000 7f6090872740 4 client.26329 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.792+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5811d7020 con 0x55c58127e380 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.792+0000 7f607fe00640 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c58127e380 0x55c581280770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.793+0000 7f607fe00640 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c58127e380 0x55c581280770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d544b4c24e5f32f9 server_cookie=990bcd55477e008 in_seq=0 out_seq=0 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.793+0000 7f6074c00640 10 client.26329 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.793+0000 7f6074c00640 1 -- 172.21.15.175:0/3617689877 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c5811d7020 con 0x55c58127e380 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.793+0000 7f6074c00640 10 client.26329 handle_command_reply: tid=0 2024-07-29T00:41:16.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.793+0000 7f6090872740 1 client.26329 shutdown 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 msgr2=0x55c58126b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126b680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6078004970 tx=0x7f607802a410 comp rx=0 tx=0).stop 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 shutdown_connections 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c58127e380 0x55c581280770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c58126e720 0x55c58126c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c58126bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6074c00640 0 client.26329 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 --2- 172.21.15.175:0/3617689877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126b680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 >> 172.21.15.175:0/3617689877 conn(0x55c58122c950 msgr2=0x55c581265d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 shutdown_connections 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 1 -- 172.21.15.175:0/3617689877 wait complete. 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.794+0000 7f6090872740 20 client.26329 trim_cache size 0 max 16384 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.808+0000 7f6090872740 1 Processor -- start 2024-07-29T00:41:16.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.808+0000 7f6090872740 1 -- start start 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6090872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6090872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581264ef0 0x55c581270050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6090872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581269f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6090872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b3c50 con 0x55c58126e720 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6090872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b57f0 con 0x55c581264ef0 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6090872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b4320 con 0x55c581263980 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581269f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581269f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45692/0 (socket says 172.21.15.175:45692) 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 -- 172.21.15.175:0/1671204626 learned_addr learned my addr 172.21.15.175:0/1671204626 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6086a00640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f607fe00640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581264ef0 0x55c581270050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 -- 172.21.15.175:0/1671204626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 msgr2=0x55c58126fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 -- 172.21.15.175:0/1671204626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581264ef0 msgr2=0x55c581270050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581264ef0 0x55c581270050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6087400640 1 -- 172.21.15.175:0/1671204626 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c581280cd0 con 0x55c58126e720 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f6086a00640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.809+0000 7f607fe00640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581264ef0 0x55c581270050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.810+0000 7f6087400640 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581269f50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6080009a30 tx=0x7f608009be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5477f229cd8b60 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.810+0000 7f6077400640 1 -- 172.21.15.175:0/1671204626 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800cd260 con 0x55c58126e720 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.810+0000 7f6077400640 1 -- 172.21.15.175:0/1671204626 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6080098420 con 0x55c58126e720 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.810+0000 7f6077400640 1 -- 172.21.15.175:0/1671204626 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608009b200 con 0x55c58126e720 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.810+0000 7f6090872740 1 -- 172.21.15.175:0/1671204626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 msgr2=0x55c581269f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.810+0000 7f6090872740 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581269f50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6080009a30 tx=0x7f608009be70 comp rx=0 tx=0).stop 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 -- 172.21.15.175:0/1671204626 shutdown_connections 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c58126e720 0x55c581269f50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581264ef0 0x55c581270050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 --2- 172.21.15.175:0/1671204626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c581263980 0x55c58126fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 -- 172.21.15.175:0/1671204626 >> 172.21.15.175:0/1671204626 conn(0x55c58122c950 msgr2=0x55c581266a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 -- 172.21.15.175:0/1671204626 shutdown_connections 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 -- 172.21.15.175:0/1671204626 wait complete. 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 -- start start 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.811+0000 7f6090872740 1 Processor -- start 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581276c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c581277170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c58126e720 0x55c581268b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b5120 con 0x55c581264ef0 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b49f0 con 0x55c581263980 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60881b1c30 con 0x55c58126e720 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6090872740 10 client.0 _ll_register_callbacks cb 0x7f607000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:16.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f607fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c581277170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6086a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581276c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f607fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c581277170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45694/0 (socket says 172.21.15.175:45694) 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f607fe00640 1 -- 172.21.15.175:0/3373485211 learned_addr learned my addr 172.21.15.175:0/3373485211 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.812+0000 7f6087400640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c58126e720 0x55c581268b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6087400640 1 -- 172.21.15.175:0/3373485211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 msgr2=0x55c581276c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6087400640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581276c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6087400640 1 -- 172.21.15.175:0/3373485211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 msgr2=0x55c581277170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6087400640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c581277170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6087400640 1 -- 172.21.15.175:0/3373485211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5812664d0 con 0x55c58126e720 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6086a00640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581276c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6087400640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c58126e720 0x55c581268b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f60800049d0 tx=0x7f60800096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471b7c1b1e750057 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6074c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:16.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6074c00640 1 -- 172.21.15.175:0/3373485211 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608007e070 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6074c00640 1 -- 172.21.15.175:0/3373485211 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f60800a4590 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6080098420 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6074c00640 1 -- 172.21.15.175:0/3373485211 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f60800a4790 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6090872740 10 client.26335 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.813+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c581285f20 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6074c00640 1 -- 172.21.15.175:0/3373485211 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6080098420 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6074c00640 1 -- 172.21.15.175:0/3373485211 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f6080081020 con 0x55c58126e720 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6090872740 10 client.26335 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6090872740 10 client.26335 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6090872740 20 client.26335 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6090872740 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5812e9030 0x55c5812eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6090872740 4 client.26335 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5811d1530 con 0x55c5812e9030 2024-07-29T00:41:16.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.814+0000 7f6086a00640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5812e9030 0x55c5812eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6086a00640 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5812e9030 0x55c5812eb420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=145f94ab4cb512a0 server_cookie=32729e59bc2a8b10 in_seq=0 out_seq=0 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6074c00640 10 client.26335 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6074c00640 1 -- 172.21.15.175:0/3373485211 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5811d1530 con 0x55c5812e9030 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6074c00640 10 client.26335 handle_command_reply: tid=0 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 client.26335 shutdown 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:41:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c58126e720 msgr2=0x55c581268b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c58126e720 0x55c581268b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f60800049d0 tx=0x7f60800096c0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 shutdown_connections 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c58126e720 0x55c581268b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c581264ef0 0x55c581277170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5812e9030 0x55c5812eb420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 --2- 172.21.15.175:0/3373485211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c581263980 0x55c581276c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6074c00640 0 client.26335 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 >> 172.21.15.175:0/3373485211 conn(0x55c58122c950 msgr2=0x55c581269f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 shutdown_connections 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 1 -- 172.21.15.175:0/3373485211 wait complete. 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.815+0000 7f6090872740 20 client.26335 trim_cache size 0 max 16384 2024-07-29T00:41:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f606c0a2d70 msgr2=0x7f606c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f606c0a2d70 0x7f606c0a5230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c581264420 tx=0x7f607000b390 comp rx=0 tx=0).stop 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 msgr2=0x7f60881a2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 0x7f60881a2fc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f608000ea60 tx=0x7f608000ef20 comp rx=0 tx=0).stop 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 shutdown_connections 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f608810d220 0x7f60881a2fc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f606c0a2d70 0x7f606c0a5230 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6088108af0 0x7f608819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 --2- 172.21.15.175:0/2839816257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6088108140 0x7f608819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 >> 172.21.15.175:0/2839816257 conn(0x7f6088077890 msgr2=0x7f6088078b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 shutdown_connections 2024-07-29T00:41:16.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:16.816+0000 7f608dc00640 1 -- 172.21.15.175:0/2839816257 wait complete. 2024-07-29T00:41:16.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 124775 1.71 MB/sec execute 67 sec latency 2820.571 ms 2024-07-29T00:41:17.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:17 smithi181 ceph-mon[36706]: pgmap v146: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 KiB/s wr, 152 op/s 2024-07-29T00:41:17.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:17 smithi175 ceph-mon[32138]: pgmap v146: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 KiB/s wr, 152 op/s 2024-07-29T00:41:17.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:41:17.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:17 smithi204 ceph-mon[37368]: pgmap v146: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 KiB/s wr, 152 op/s 2024-07-29T00:41:17.887 INFO:tasks.workunit.client.0.smithi175.stdout: 1 125163 1.70 MB/sec execute 68 sec latency 3432.164 ms 2024-07-29T00:41:18.295 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:18.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 126904 1.82 MB/sec execute 69 sec latency 15.134 ms 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.025+0000 7f3b7fe00640 1 -- 172.21.15.175:0/342386303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b80108670 msgr2=0x7f3b80114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.025+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/342386303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b80108670 0x7f3b80114b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009cd0 tx=0x7f3b7802a8f0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 -- 172.21.15.175:0/342386303 shutdown_connections 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/342386303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b80108670 0x7f3b80114b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/342386303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80107d10 0x7f3b80108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/342386303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b8010dd50 0x7f3b8010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 -- 172.21.15.175:0/342386303 >> 172.21.15.175:0/342386303 conn(0x7f3b80077670 msgr2=0x7f3b80079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:19.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 -- 172.21.15.175:0/342386303 shutdown_connections 2024-07-29T00:41:19.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.026+0000 7f3b7fe00640 1 -- 172.21.15.175:0/342386303 wait complete. 2024-07-29T00:41:19.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.027+0000 7f3b7fe00640 1 Processor -- start 2024-07-29T00:41:19.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.027+0000 7f3b7fe00640 1 -- start start 2024-07-29T00:41:19.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b80107d10 0x7f3b8019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 0x7f3b8019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b8010dd50 0x7f3b801a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801180b0 con 0x7f3b80107d10 2024-07-29T00:41:19.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b80117f30 con 0x7f3b80108670 2024-07-29T00:41:19.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b80118230 con 0x7f3b8010dd50 2024-07-29T00:41:19.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 0x7f3b8019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b80107d10 0x7f3b8019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b8010dd50 0x7f3b801a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 0x7f3b8019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57114/0 (socket says 172.21.15.175:57114) 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b8010dd50 0x7f3b801a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49538/0 (socket says 172.21.15.175:49538) 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.028+0000 7f3b7e000640 1 -- 172.21.15.175:0/2531398312 learned_addr learned my addr 172.21.15.175:0/2531398312 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7e000640 1 -- 172.21.15.175:0/2531398312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b8010dd50 msgr2=0x7f3b801a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7e000640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b8010dd50 0x7f3b801a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7e000640 1 -- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b80107d10 msgr2=0x7f3b8019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7e000640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b80107d10 0x7f3b8019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7e000640 1 -- 172.21.15.175:0/2531398312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b801a34d0 con 0x7f3b80108670 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7e000640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 0x7f3b8019ea60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000e9f0 tx=0x7f3b7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b77400640 1 -- 172.21.15.175:0/2531398312 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000cd80 con 0x7f3b80108670 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b77400640 1 -- 172.21.15.175:0/2531398312 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b70010910 con 0x7f3b80108670 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.029+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b801a37c0 con 0x7f3b80108670 2024-07-29T00:41:19.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.030+0000 7f3b77400640 1 -- 172.21.15.175:0/2531398312 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70018990 con 0x7f3b80108670 2024-07-29T00:41:19.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.030+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b801ab0a0 con 0x7f3b80108670 2024-07-29T00:41:19.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.033+0000 7f3b77400640 1 -- 172.21.15.175:0/2531398312 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3b70010ac0 con 0x7f3b80108670 2024-07-29T00:41:19.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 Processor -- start 2024-07-29T00:41:19.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 -- start start 2024-07-29T00:41:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dab70 0x556f725dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725e5910 0x556f725e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f72540ab0 con 0x556f725dab70 2024-07-29T00:41:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f724d8a80 con 0x556f725e5910 2024-07-29T00:41:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b88228740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556f7253ca70 con 0x556f725dc0e0 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725da630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49548/0 (socket says 172.21.15.175:49548) 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b7f400640 1 -- 172.21.15.175:0/412987017 learned_addr learned my addr 172.21.15.175:0/412987017 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.034+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dab70 0x556f725dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7e000640 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725e5910 0x556f725e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7f400640 1 -- 172.21.15.175:0/412987017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725e5910 msgr2=0x556f725e7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7f400640 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725e5910 0x556f725e7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7f400640 1 -- 172.21.15.175:0/412987017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dab70 msgr2=0x556f725dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7f400640 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dab70 0x556f725dafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7f400640 1 -- 172.21.15.175:0/412987017 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f722b23c0 con 0x556f725dc0e0 2024-07-29T00:41:19.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b77400640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3b640a2d70 0x7f3b640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b7f400640 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725da630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009cd0 tx=0x7f3b78005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d17fe65901dc8ec server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:19.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.035+0000 7f3b77400640 1 -- 172.21.15.175:0/2531398312 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3b700cf740 con 0x7f3b80108670 2024-07-29T00:41:19.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.036+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3b640a2d70 0x7f3b640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.036+0000 7f3b6be00640 1 -- 172.21.15.175:0/412987017 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78003970 con 0x556f725dc0e0 2024-07-29T00:41:19.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.036+0000 7f3b6be00640 1 -- 172.21.15.175:0/412987017 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b78036030 con 0x556f725dc0e0 2024-07-29T00:41:19.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.036+0000 7f3b6be00640 1 -- 172.21.15.175:0/412987017 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78004500 con 0x556f725dc0e0 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 -- 172.21.15.175:0/412987017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 msgr2=0x556f725da630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725da630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009cd0 tx=0x7f3b78005fa0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3b640a2d70 0x7f3b640a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x556f725d8060 tx=0x7f3b6c00b390 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 -- 172.21.15.175:0/412987017 shutdown_connections 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725e5910 0x556f725e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dab70 0x556f725dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 --2- 172.21.15.175:0/412987017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725da630 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 -- 172.21.15.175:0/412987017 >> 172.21.15.175:0/412987017 conn(0x556f725a3950 msgr2=0x556f725d7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 -- 172.21.15.175:0/412987017 shutdown_connections 2024-07-29T00:41:19.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 -- 172.21.15.175:0/412987017 wait complete. 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.037+0000 7f3b88228740 1 -- start start 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 Processor -- start 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 0x556f725e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dc0e0 0x556f725e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725e3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f725e8650 con 0x556f725e5910 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f724fbc10 con 0x556f725dc0e0 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556f725e87d0 con 0x556f725dab70 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b88228740 10 client.0 _ll_register_callbacks cb 0x556f724c4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:19.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.038+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 0x556f725e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 0x556f725e28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49562/0 (socket says 172.21.15.175:49562) 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 -- 172.21.15.175:0/471977137 learned_addr learned my addr 172.21.15.175:0/471977137 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7f400640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7e000640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dc0e0 0x556f725e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 -- 172.21.15.175:0/471977137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dc0e0 msgr2=0x556f725e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dc0e0 0x556f725e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 -- 172.21.15.175:0/471977137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 msgr2=0x556f725e3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725e3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 -- 172.21.15.175:0/471977137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f724ff1e0 con 0x556f725dab70 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7e000640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dc0e0 0x556f725e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7f400640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 0x556f725e28b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c003cb0 tx=0x7f3b6c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624fa6e22d6d906a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.039+0000 7f3b68c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:19.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.040+0000 7f3b68c00640 1 -- 172.21.15.175:0/471977137 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c02c060 con 0x556f725dab70 2024-07-29T00:41:19.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.040+0000 7f3b68c00640 1 -- 172.21.15.175:0/471977137 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3b6c017030 con 0x556f725dab70 2024-07-29T00:41:19.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.040+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b78036030 con 0x556f725dab70 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.040+0000 7f3b68c00640 1 -- 172.21.15.175:0/471977137 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b6c027410 con 0x556f725dab70 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 10 client.26347 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556f725f4470 con 0x556f725dab70 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b68c00640 1 -- 172.21.15.175:0/471977137 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b78036030 con 0x556f725dab70 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b68c00640 1 -- 172.21.15.175:0/471977137 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f3b6c011600 con 0x556f725dab70 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 10 client.26347 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 10 client.26347 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:41:19.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 20 client.26347 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f725f5930 0x556f725f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 4 client.26347 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.041+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f7254e020 con 0x556f725f5930 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.042+0000 7f3b7e000640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f725f5930 0x556f725f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.042+0000 7f3b7e000640 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f725f5930 0x556f725f7d20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b824c29f0a3a6909 server_cookie=980bdd5d85b73d08 in_seq=0 out_seq=0 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.042+0000 7f3b68c00640 10 client.26347 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.043+0000 7f3b68c00640 1 -- 172.21.15.175:0/471977137 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556f7254e020 con 0x556f725f5930 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.043+0000 7f3b68c00640 10 client.26347 handle_command_reply: tid=0 2024-07-29T00:41:19.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.043+0000 7f3b88228740 1 client.26347 shutdown 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.043+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 msgr2=0x556f725e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.043+0000 7f3b88228740 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 0x556f725e28b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c003cb0 tx=0x7f3b6c004730 comp rx=0 tx=0).stop 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 shutdown_connections 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f725f5930 0x556f725f7d20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dc0e0 0x556f725e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 --2- 172.21.15.175:0/471977137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dab70 0x556f725e28b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b68c00640 0 client.26347 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 >> 172.21.15.175:0/471977137 conn(0x556f725a3950 msgr2=0x556f725dced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 shutdown_connections 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 1 -- 172.21.15.175:0/471977137 wait complete. 2024-07-29T00:41:19.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.044+0000 7f3b88228740 20 client.26347 trim_cache size 0 max 16384 2024-07-29T00:41:19.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.059+0000 7f3b88228740 1 Processor -- start 2024-07-29T00:41:19.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.059+0000 7f3b88228740 1 -- start start 2024-07-29T00:41:19.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b88228740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b88228740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b88228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b88228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801b3a50 con 0x556f725e5910 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b88228740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801b55f0 con 0x556f725dab70 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b88228740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801b4120 con 0x556f725dc0e0 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57134/0 (socket says 172.21.15.175:57134) 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 -- 172.21.15.175:0/2920396932 learned_addr learned my addr 172.21.15.175:0/2920396932 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 -- 172.21.15.175:0/2920396932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 msgr2=0x556f725fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 -- 172.21.15.175:0/2920396932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 msgr2=0x556f725fdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725fdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7ea00640 1 -- 172.21.15.175:0/2920396932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f725db940 con 0x556f725dab70 2024-07-29T00:41:19.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.060+0000 7f3b7f400640 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:19.061 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.061+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e4820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0273e0 tx=0x7f3b6c0043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28098d84f62ea3d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.061+0000 7f3b6b400640 1 -- 172.21.15.175:0/2920396932 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c011c90 con 0x556f725dab70 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.061+0000 7f3b6b400640 1 -- 172.21.15.175:0/2920396932 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c011e30 con 0x556f725dab70 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.061+0000 7f3b6b400640 1 -- 172.21.15.175:0/2920396932 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c004a00 con 0x556f725dab70 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 -- 172.21.15.175:0/2920396932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 msgr2=0x556f725e4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e4820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0273e0 tx=0x7f3b6c0043d0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 -- 172.21.15.175:0/2920396932 shutdown_connections 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725e5910 0x556f725fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725dc0e0 0x556f725fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 --2- 172.21.15.175:0/2920396932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e4820 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 -- 172.21.15.175:0/2920396932 >> 172.21.15.175:0/2920396932 conn(0x556f725a3950 msgr2=0x556f725e4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:19.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 -- 172.21.15.175:0/2920396932 shutdown_connections 2024-07-29T00:41:19.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 -- 172.21.15.175:0/2920396932 wait complete. 2024-07-29T00:41:19.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.062+0000 7f3b88228740 1 -- start start 2024-07-29T00:41:19.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 Processor -- start 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dc0e0 0x556f725e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725e5910 0x556f725ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801b4f20 con 0x556f725dc0e0 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801b47f0 con 0x556f725dab70 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801b1a30 con 0x556f725e5910 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b88228740 10 client.0 _ll_register_callbacks cb 0x7f3b6c004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:19.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.063+0000 7f3b7ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57136/0 (socket says 172.21.15.175:57136) 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 -- 172.21.15.175:0/1018831913 learned_addr learned my addr 172.21.15.175:0/1018831913 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7f400640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725e5910 0x556f725ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7e000640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dc0e0 0x556f725e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 -- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725e5910 msgr2=0x556f725ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725e5910 0x556f725ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 -- 172.21.15.175:0/1018831913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dc0e0 msgr2=0x556f725e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dc0e0 0x556f725e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 -- 172.21.15.175:0/1018831913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7265f620 con 0x556f725dab70 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7e000640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dc0e0 0x556f725e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b7ea00640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e98b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c011b40 tx=0x7f3b6c003640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe2e87fcc943773 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.064+0000 7f3b68c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:19.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b68c00640 1 -- 172.21.15.175:0/1018831913 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0275b0 con 0x556f725dab70 2024-07-29T00:41:19.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b68c00640 1 -- 172.21.15.175:0/1018831913 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3b6c027750 con 0x556f725dab70 2024-07-29T00:41:19.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6c011e30 con 0x556f725dab70 2024-07-29T00:41:19.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b68c00640 1 -- 172.21.15.175:0/1018831913 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b6c027950 con 0x556f725dab70 2024-07-29T00:41:19.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b88228740 10 client.26393 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:19.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556f72660900 con 0x556f725dab70 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.065+0000 7f3b68c00640 1 -- 172.21.15.175:0/1018831913 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c011e30 con 0x556f725dab70 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b68c00640 1 -- 172.21.15.175:0/1018831913 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f3b6c031c40 con 0x556f725dab70 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b88228740 10 client.26393 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b88228740 10 client.26393 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b88228740 20 client.26393 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b88228740 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f72660fb0 0x556f726633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b88228740 4 client.26393 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f72548530 con 0x556f72660fb0 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.066+0000 7f3b7e000640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f72660fb0 0x556f726633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b68c00640 10 client.26393 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b7e000640 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f72660fb0 0x556f726633a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da220b9df9cca5e9 server_cookie=2e05e7895330fcc1 in_seq=0 out_seq=0 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b68c00640 1 -- 172.21.15.175:0/1018831913 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f72548530 con 0x556f72660fb0 2024-07-29T00:41:19.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b68c00640 10 client.26393 handle_command_reply: tid=0 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 client.26393 shutdown 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 msgr2=0x556f725e98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e98b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c011b40 tx=0x7f3b6c003640 comp rx=0 tx=0).stop 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 shutdown_connections 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x556f72660fb0 0x556f726633a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f725e5910 0x556f725ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f725dc0e0 0x556f725e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b88228740 1 --2- 172.21.15.175:0/1018831913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f725dab70 0x556f725e98b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.067+0000 7f3b68c00640 0 client.26393 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.068+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 >> 172.21.15.175:0/1018831913 conn(0x556f725a3950 msgr2=0x556f725d6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.068+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 shutdown_connections 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.068+0000 7f3b88228740 1 -- 172.21.15.175:0/1018831913 wait complete. 2024-07-29T00:41:19.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.068+0000 7f3b88228740 20 client.26393 trim_cache size 0 max 16384 2024-07-29T00:41:19.070 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:41:19.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3b640a2d70 msgr2=0x7f3b640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3b640a2d70 0x7f3b640a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x556f725d8060 tx=0x7f3b6c00b390 comp rx=0 tx=0).stop 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 msgr2=0x7f3b8019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 0x7f3b8019ea60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000e9f0 tx=0x7f3b7000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 shutdown_connections 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b8010dd50 0x7f3b801a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80108670 0x7f3b8019ea60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3b640a2d70 0x7f3b640a5230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 --2- 172.21.15.175:0/2531398312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b80107d10 0x7f3b8019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 >> 172.21.15.175:0/2531398312 conn(0x7f3b80077670 msgr2=0x7f3b80078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 shutdown_connections 2024-07-29T00:41:19.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:19.069+0000 7f3b7fe00640 1 -- 172.21.15.175:0/2531398312 wait complete. 2024-07-29T00:41:19.737 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:19 smithi175 ceph-mon[32138]: pgmap v147: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 545 KiB/s wr, 96 op/s 2024-07-29T00:41:19.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:19 smithi181 ceph-mon[36706]: pgmap v147: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 545 KiB/s wr, 96 op/s 2024-07-29T00:41:19.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:19 smithi204 ceph-mon[37368]: pgmap v147: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 545 KiB/s wr, 96 op/s 2024-07-29T00:41:19.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 128039 1.90 MB/sec execute 70 sec latency 26.517 ms 2024-07-29T00:41:20.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:41:20.816 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:20.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 128720 1.89 MB/sec execute 71 sec latency 19.021 ms 2024-07-29T00:41:21.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:21 smithi175 ceph-mon[32138]: pgmap v148: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 545 KiB/s wr, 96 op/s 2024-07-29T00:41:21.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 -- 172.21.15.175:0/3052822073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098104440 msgr2=0x7fd098110970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 --2- 172.21.15.175:0/3052822073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098104440 0x7fd098110970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd090009c10 tx=0x7fd09002a6e0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 -- 172.21.15.175:0/3052822073 shutdown_connections 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 --2- 172.21.15.175:0/3052822073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098104440 0x7fd098110970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 --2- 172.21.15.175:0/3052822073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd098103aa0 0x7fd098103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 --2- 172.21.15.175:0/3052822073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098109aa0 0x7fd098109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 -- 172.21.15.175:0/3052822073 >> 172.21.15.175:0/3052822073 conn(0x7fd0980ff740 msgr2=0x7fd098101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 -- 172.21.15.175:0/3052822073 shutdown_connections 2024-07-29T00:41:21.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.499+0000 7fd09ce00640 1 -- 172.21.15.175:0/3052822073 wait complete. 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.500+0000 7fd09ce00640 1 Processor -- start 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.500+0000 7fd09ce00640 1 -- start start 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd09ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd098103aa0 0x7fd09819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd09ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 0x7fd09819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd09ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098109aa0 0x7fd09819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd09ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd098113ea0 con 0x7fd098109aa0 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd09ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd098113d20 con 0x7fd098103aa0 2024-07-29T00:41:21.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd09ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd098114020 con 0x7fd098104440 2024-07-29T00:41:21.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd095400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 0x7fd09819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd095400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 0x7fd09819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49586/0 (socket says 172.21.15.175:49586) 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd095400640 1 -- 172.21.15.175:0/1751120924 learned_addr learned my addr 172.21.15.175:0/1751120924 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd095e00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd098103aa0 0x7fd09819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.501+0000 7fd096800640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098109aa0 0x7fd09819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd095400640 1 -- 172.21.15.175:0/1751120924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd098103aa0 msgr2=0x7fd09819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd095400640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd098103aa0 0x7fd09819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd095400640 1 -- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098109aa0 msgr2=0x7fd09819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd095400640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098109aa0 0x7fd09819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd095400640 1 -- 172.21.15.175:0/1751120924 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09819f310 con 0x7fd098104440 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd095400640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 0x7fd09819a8a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd08400e9c0 tx=0x7fd08400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd08ea00640 1 -- 172.21.15.175:0/1751120924 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08400cde0 con 0x7fd098104440 2024-07-29T00:41:21.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.502+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09819f600 con 0x7fd098104440 2024-07-29T00:41:21.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.503+0000 7fd08ea00640 1 -- 172.21.15.175:0/1751120924 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0840035f0 con 0x7fd098104440 2024-07-29T00:41:21.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.503+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a6ee0 con 0x7fd098104440 2024-07-29T00:41:21.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.503+0000 7fd08ea00640 1 -- 172.21.15.175:0/1751120924 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084010650 con 0x7fd098104440 2024-07-29T00:41:21.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.506+0000 7fd08ea00640 1 -- 172.21.15.175:0/1751120924 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd0840037a0 con 0x7fd098104440 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.506+0000 7fd09f805740 1 Processor -- start 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 -- start start 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f136440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f143b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55df3f09cab0 con 0x55df3f136980 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55df3f034a80 con 0x55df3f141720 2024-07-29T00:41:21.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd09f805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55df3f098a70 con 0x55df3f137ef0 2024-07-29T00:41:21.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd096800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f136440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd095400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd096800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f136440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49594/0 (socket says 172.21.15.175:49594) 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd096800640 1 -- 172.21.15.175:0/1363446808 learned_addr learned my addr 172.21.15.175:0/1363446808 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.507+0000 7fd095e00640 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 -- 172.21.15.175:0/1363446808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 msgr2=0x55df3f143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 -- 172.21.15.175:0/1363446808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 msgr2=0x55df3f136dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f136dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 -- 172.21.15.175:0/1363446808 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3ee0e3c0 con 0x55df3f137ef0 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd095e00640 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f136dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd08ea00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd07c0a2d70 0x7fd07c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd08ea00640 1 -- 172.21.15.175:0/1751120924 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd0840cf770 con 0x7fd098104440 2024-07-29T00:41:21.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f136440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd090009be0 tx=0x7fd090030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d955f5ec4fb48b49 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:21.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.508+0000 7fd096800640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd07c0a2d70 0x7fd07c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.509+0000 7fd083400640 1 -- 172.21.15.175:0/1363446808 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090041040 con 0x55df3f137ef0 2024-07-29T00:41:21.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.509+0000 7fd083400640 1 -- 172.21.15.175:0/1363446808 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd090003e50 con 0x55df3f137ef0 2024-07-29T00:41:21.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.509+0000 7fd083400640 1 -- 172.21.15.175:0/1363446808 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090036040 con 0x55df3f137ef0 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.509+0000 7fd09f805740 1 -- 172.21.15.175:0/1363446808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 msgr2=0x55df3f136440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.509+0000 7fd096800640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd07c0a2d70 0x7fd07c0a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd090049000 tx=0x7fd09003d510 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.509+0000 7fd09f805740 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f136440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd090009be0 tx=0x7fd090030a30 comp rx=0 tx=0).stop 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 -- 172.21.15.175:0/1363446808 shutdown_connections 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f136dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 --2- 172.21.15.175:0/1363446808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f136440 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 -- 172.21.15.175:0/1363446808 >> 172.21.15.175:0/1363446808 conn(0x55df3f0ff950 msgr2=0x55df3f133010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 -- 172.21.15.175:0/1363446808 shutdown_connections 2024-07-29T00:41:21.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 -- 172.21.15.175:0/1363446808 wait complete. 2024-07-29T00:41:21.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.510+0000 7fd09f805740 1 -- start start 2024-07-29T00:41:21.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 Processor -- start 2024-07-29T00:41:21.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f136980 0x55df3f13e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f13f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55df3f1444c0 con 0x55df3f137ef0 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55df3f057c10 con 0x55df3f141720 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55df3f144640 con 0x55df3f136980 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd09f805740 10 client.0 _ll_register_callbacks cb 0x55df3f020a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd096800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.511+0000 7fd096800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f13f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57170/0 (socket says 172.21.15.175:57170) 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 -- 172.21.15.175:0/2980307113 learned_addr learned my addr 172.21.15.175:0/2980307113 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd095e00640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f136980 0x55df3f13e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd095400640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 -- 172.21.15.175:0/2980307113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f136980 msgr2=0x55df3f13e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f136980 0x55df3f13e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 -- 172.21.15.175:0/2980307113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 msgr2=0x55df3f13ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f13ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 -- 172.21.15.175:0/2980307113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3f05b1e0 con 0x55df3f141720 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd095e00640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f136980 0x55df3f13e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd096800640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f13f110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd090005f00 tx=0x7fd090040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e81a91aab051a58 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.512+0000 7fd077e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd077e00640 1 -- 172.21.15.175:0/2980307113 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09005c060 con 0x55df3f141720 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd077e00640 1 -- 172.21.15.175:0/2980307113 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd0900045e0 con 0x55df3f141720 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd090003e50 con 0x55df3f141720 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd077e00640 1 -- 172.21.15.175:0/2980307113 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd090036030 con 0x55df3f141720 2024-07-29T00:41:21.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd09f805740 10 client.26405 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55df3f150230 con 0x55df3f141720 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd077e00640 1 -- 172.21.15.175:0/2980307113 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd090003e50 con 0x55df3f141720 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.513+0000 7fd077e00640 1 -- 172.21.15.175:0/2980307113 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd09003db90 con 0x55df3f141720 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd09f805740 10 client.26405 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd09f805740 10 client.26405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd09f805740 20 client.26405 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:21.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd09f805740 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f151580 0x55df3f153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd09f805740 4 client.26405 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3f0aa020 con 0x55df3f151580 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd095e00640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f151580 0x55df3f153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd095e00640 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f151580 0x55df3f153970 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17e140e7ab53d536 server_cookie=41bfe55db3d86f91 in_seq=0 out_seq=0 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.514+0000 7fd077e00640 10 client.26405 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.515+0000 7fd077e00640 1 -- 172.21.15.175:0/2980307113 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df3f0aa020 con 0x55df3f151580 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.515+0000 7fd077e00640 10 client.26405 handle_command_reply: tid=0 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.515+0000 7fd09f805740 1 client.26405 shutdown 2024-07-29T00:41:21.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.515+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 msgr2=0x55df3f13f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.515+0000 7fd09f805740 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f13f110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd090005f00 tx=0x7fd090040000 comp rx=0 tx=0).stop 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 shutdown_connections 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f151580 0x55df3f153970 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f13f110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f13ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 --2- 172.21.15.175:0/2980307113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f136980 0x55df3f13e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd077e00640 0 client.26405 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 >> 172.21.15.175:0/2980307113 conn(0x55df3f0ff950 msgr2=0x55df3f138d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 shutdown_connections 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 1 -- 172.21.15.175:0/2980307113 wait complete. 2024-07-29T00:41:21.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.516+0000 7fd09f805740 20 client.26405 trim_cache size 0 max 16384 2024-07-29T00:41:21.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.532+0000 7fd09f805740 1 Processor -- start 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.532+0000 7fd09f805740 1 -- start start 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd09f805740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 0x55df3f13d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd09f805740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f158f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd09f805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f141720 0x55df3f159480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd09f805740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981af8c0 con 0x55df3f137ef0 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd09f805740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b1460 con 0x55df3f136980 2024-07-29T00:41:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd09f805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981aff90 con 0x55df3f141720 2024-07-29T00:41:21.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 0x55df3f13d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd096800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f141720 0x55df3f159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 0x55df3f13d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57178/0 (socket says 172.21.15.175:57178) 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 -- 172.21.15.175:0/2199956272 learned_addr learned my addr 172.21.15.175:0/2199956272 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095400640 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 -- 172.21.15.175:0/2199956272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f141720 msgr2=0x55df3f159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f141720 0x55df3f159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 -- 172.21.15.175:0/2199956272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 msgr2=0x55df3f158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd095e00640 1 -- 172.21.15.175:0/2199956272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3f137100 con 0x55df3f136980 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.533+0000 7fd096800640 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f141720 0x55df3f159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.534+0000 7fd095e00640 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 0x55df3f13d670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd088009fd0 tx=0x7fd088005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ee403159baa683 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.534+0000 7fd082a00640 1 -- 172.21.15.175:0/2199956272 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088005020 con 0x55df3f136980 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.534+0000 7fd082a00640 1 -- 172.21.15.175:0/2199956272 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0880047a0 con 0x55df3f136980 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.534+0000 7fd082a00640 1 -- 172.21.15.175:0/2199956272 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08800f8c0 con 0x55df3f136980 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 -- 172.21.15.175:0/2199956272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 msgr2=0x55df3f13d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 0x55df3f13d670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd088009fd0 tx=0x7fd088005f90 comp rx=0 tx=0).stop 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 -- 172.21.15.175:0/2199956272 shutdown_connections 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f141720 0x55df3f159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f137ef0 0x55df3f158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 --2- 172.21.15.175:0/2199956272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f136980 0x55df3f13d670 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 -- 172.21.15.175:0/2199956272 >> 172.21.15.175:0/2199956272 conn(0x55df3f0ff950 msgr2=0x55df3f13cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 -- 172.21.15.175:0/2199956272 shutdown_connections 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 -- 172.21.15.175:0/2199956272 wait complete. 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 -- start start 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.535+0000 7fd09f805740 1 Processor -- start 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f1448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f144e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f145350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b0d90 con 0x55df3f136980 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b0660 con 0x55df3f141720 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ad8a0 con 0x55df3f137ef0 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd09f805740 10 client.0 _ll_register_callbacks cb 0x7fd0840d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd095e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f1448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd096800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd095e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f1448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44874/0 (socket says 172.21.15.175:44874) 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd095e00640 1 -- 172.21.15.175:0/2546727114 learned_addr learned my addr 172.21.15.175:0/2546727114 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.536+0000 7fd095400640 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd096800640 1 -- 172.21.15.175:0/2546727114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 msgr2=0x55df3f144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd096800640 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd096800640 1 -- 172.21.15.175:0/2546727114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 msgr2=0x55df3f1448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd096800640 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f1448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd096800640 1 -- 172.21.15.175:0/2546727114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3f1ba1a0 con 0x55df3f141720 2024-07-29T00:41:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd096800640 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f145350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd09003df30 tx=0x7fd09002f300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb540408046eb7a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd077e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd077e00640 1 -- 172.21.15.175:0/2546727114 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090041070 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd077e00640 1 -- 172.21.15.175:0/2546727114 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd090004350 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.537+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0880047a0 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd077e00640 1 -- 172.21.15.175:0/2546727114 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd09003dd20 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 10 client.26411 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55df3f1bb460 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd077e00640 1 -- 172.21.15.175:0/2546727114 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0880047a0 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd077e00640 1 -- 172.21.15.175:0/2546727114 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd090003ee0 con 0x55df3f141720 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 10 client.26411 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 10 client.26411 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 20 client.26411 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f1bba10 0x55df3f1bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 4 client.26411 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:41:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df3f0a4530 con 0x55df3f1bba10 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.538+0000 7fd095e00640 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f1bba10 0x55df3f1bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd077e00640 10 client.26411 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd095e00640 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f1bba10 0x55df3f1bde50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27065e218a030583 server_cookie=a2b85d19e0f1a9f8 in_seq=0 out_seq=0 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd077e00640 1 -- 172.21.15.175:0/2546727114 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df3f0a4530 con 0x55df3f1bba10 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd077e00640 10 client.26411 handle_command_reply: tid=0 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd09f805740 1 client.26411 shutdown 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 msgr2=0x55df3f145350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.539+0000 7fd09f805740 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f145350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd09003df30 tx=0x7fd09002f300 comp rx=0 tx=0).stop 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 shutdown_connections 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55df3f1bba10 0x55df3f1bde50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df3f141720 0x55df3f145350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd077e00640 0 client.26411 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:41:21.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df3f137ef0 0x55df3f144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 --2- 172.21.15.175:0/2546727114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df3f136980 0x55df3f1448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 >> 172.21.15.175:0/2546727114 conn(0x55df3f0ff950 msgr2=0x55df3f13d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:21.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 shutdown_connections 2024-07-29T00:41:21.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 1 -- 172.21.15.175:0/2546727114 wait complete. 2024-07-29T00:41:21.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.540+0000 7fd09f805740 20 client.26411 trim_cache size 0 max 16384 2024-07-29T00:41:21.543 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd07c0a2d70 msgr2=0x7fd07c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd07c0a2d70 0x7fd07c0a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd090049000 tx=0x7fd09003d510 comp rx=0 tx=0).stop 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 msgr2=0x7fd09819a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 0x7fd09819a8a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd08400e9c0 tx=0x7fd08400ee80 comp rx=0 tx=0).stop 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 shutdown_connections 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd07c0a2d70 0x7fd07c0a5230 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd098109aa0 0x7fd09819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd098104440 0x7fd09819a8a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 --2- 172.21.15.175:0/1751120924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd098103aa0 0x7fd09819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 >> 172.21.15.175:0/1751120924 conn(0x7fd0980ff740 msgr2=0x7fd0980fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 shutdown_connections 2024-07-29T00:41:21.544 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:21.541+0000 7fd09ce00640 1 -- 172.21.15.175:0/1751120924 wait complete. 2024-07-29T00:41:21.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:21 smithi181 ceph-mon[36706]: pgmap v148: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 545 KiB/s wr, 96 op/s 2024-07-29T00:41:21.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:21 smithi204 ceph-mon[37368]: pgmap v148: 97 pgs: 97 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 545 KiB/s wr, 96 op/s 2024-07-29T00:41:21.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 130218 1.96 MB/sec execute 72 sec latency 15.472 ms 2024-07-29T00:41:22.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:41:22.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 131518 2.09 MB/sec execute 73 sec latency 18.437 ms 2024-07-29T00:41:23.057 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:23.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:23 smithi175 ceph-mon[32138]: pgmap v149: 97 pgs: 97 active+clean; 85 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 905 KiB/s wr, 154 op/s 2024-07-29T00:41:23.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 -- 172.21.15.175:0/3647964561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 msgr2=0x7f101c10b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 --2- 172.21.15.175:0/3647964561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c10b840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1008009900 tx=0x7f100802a480 comp rx=0 tx=0).stop 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 -- 172.21.15.175:0/3647964561 shutdown_connections 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 --2- 172.21.15.175:0/3647964561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c10be60 0x7f101c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 --2- 172.21.15.175:0/3647964561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c10b840 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 --2- 172.21.15.175:0/3647964561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c102a30 0x7f101c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 -- 172.21.15.175:0/3647964561 >> 172.21.15.175:0/3647964561 conn(0x7f101c0fc490 msgr2=0x7f101c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 -- 172.21.15.175:0/3647964561 shutdown_connections 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.702+0000 7f1023200640 1 -- 172.21.15.175:0/3647964561 wait complete. 2024-07-29T00:41:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 Processor -- start 2024-07-29T00:41:23.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 -- start start 2024-07-29T00:41:23.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c102a30 0x7f101c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c10be60 0x7f101c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f101c111cb0 con 0x7f101c10be60 2024-07-29T00:41:23.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f101c111b30 con 0x7f101c103350 2024-07-29T00:41:23.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f1023200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f101c111e30 con 0x7f101c102a30 2024-07-29T00:41:23.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.704+0000 7f101b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c102a30 0x7f101c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57206/0 (socket says 172.21.15.175:57206) 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 -- 172.21.15.175:0/2488524249 learned_addr learned my addr 172.21.15.175:0/2488524249 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f1021200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c10be60 0x7f101c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 -- 172.21.15.175:0/2488524249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c102a30 msgr2=0x7f101c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c102a30 0x7f101c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 -- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c10be60 msgr2=0x7f101c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c10be60 0x7f101c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101b400640 1 -- 172.21.15.175:0/2488524249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c19f310 con 0x7f101c103350 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f1021200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c10be60 0x7f101c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.705+0000 7f101be00640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c102a30 0x7f101c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:23.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.706+0000 7f101b400640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c19a8a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1008004770 tx=0x7f1008003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:23.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.706+0000 7f1018c00640 1 -- 172.21.15.175:0/2488524249 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100803f070 con 0x7f101c103350 2024-07-29T00:41:23.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.706+0000 7f1018c00640 1 -- 172.21.15.175:0/2488524249 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1008031c90 con 0x7f101c103350 2024-07-29T00:41:23.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.706+0000 7f1018c00640 1 -- 172.21.15.175:0/2488524249 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008035030 con 0x7f101c103350 2024-07-29T00:41:23.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.706+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c19f5a0 con 0x7f101c103350 2024-07-29T00:41:23.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.706+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c19fa30 con 0x7f101c103350 2024-07-29T00:41:23.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.710+0000 7f1025e3a740 1 Processor -- start 2024-07-29T00:41:23.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.710+0000 7f1018c00640 1 -- 172.21.15.175:0/2488524249 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f1008042070 con 0x7f101c103350 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 -- start start 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 0x557f32e20440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e20dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e33f30 0x557f32e36320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f32d86ab0 con 0x557f32e33f30 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557f32d1ea80 con 0x557f32e20980 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1025e3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557f32d82a70 con 0x557f32e21ef0 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1021200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 0x557f32e20440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1021200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 0x557f32e20440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49666/0 (socket says 172.21.15.175:49666) 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f1021200640 1 -- 172.21.15.175:0/4175866631 learned_addr learned my addr 172.21.15.175:0/4175866631 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.711+0000 7f101be00640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e20dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f101b400640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e33f30 0x557f32e36320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f1018c00640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f10000a2d70 0x7f10000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f1021200640 1 -- 172.21.15.175:0/4175866631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 msgr2=0x557f32e20dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f1021200640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e20dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f1021200640 1 -- 172.21.15.175:0/4175866631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e33f30 msgr2=0x557f32e36320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f1021200640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e33f30 0x557f32e36320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f1021200640 1 -- 172.21.15.175:0/4175866631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f32af83c0 con 0x557f32e21ef0 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f101be00640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e20dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.712+0000 7f101b400640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e33f30 0x557f32e36320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.713+0000 7f1018c00640 1 -- 172.21.15.175:0/2488524249 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f10080ef9e0 con 0x7f101c103350 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.713+0000 7f1021200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f10000a2d70 0x7f10000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.713+0000 7f1021200640 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 0x557f32e20440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1014009fd0 tx=0x7f1014010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dba9c8f9610585e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.713+0000 7f1011600640 1 -- 172.21.15.175:0/4175866631 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014011860 con 0x557f32e21ef0 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.714+0000 7f1011600640 1 -- 172.21.15.175:0/4175866631 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f101401b3f0 con 0x557f32e21ef0 2024-07-29T00:41:23.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.714+0000 7f1011600640 1 -- 172.21.15.175:0/4175866631 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101401a600 con 0x557f32e21ef0 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 -- 172.21.15.175:0/4175866631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 msgr2=0x557f32e20440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 0x557f32e20440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1014009fd0 tx=0x7f1014010860 comp rx=0 tx=0).stop 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1021200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f10000a2d70 0x7f10000a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1014011fd0 tx=0x7f1014011f60 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 -- 172.21.15.175:0/4175866631 shutdown_connections 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e33f30 0x557f32e36320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e20dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 --2- 172.21.15.175:0/4175866631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e21ef0 0x557f32e20440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 -- 172.21.15.175:0/4175866631 >> 172.21.15.175:0/4175866631 conn(0x557f32de9950 msgr2=0x557f32e1d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:23.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 -- 172.21.15.175:0/4175866631 shutdown_connections 2024-07-29T00:41:23.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.715+0000 7f1025e3a740 1 -- 172.21.15.175:0/4175866631 wait complete. 2024-07-29T00:41:23.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.716+0000 7f1025e3a740 1 -- start start 2024-07-29T00:41:23.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.716+0000 7f1025e3a740 1 Processor -- start 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e2dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e2e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f32e23b30 con 0x557f32e21ef0 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557f32d41c10 con 0x557f32e20980 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557f32e23cb0 con 0x557f32e33f30 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1025e3a740 10 client.0 _ll_register_callbacks cb 0x557f32d0aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:23.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1021200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e2e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1021200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e2e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49674/0 (socket says 172.21.15.175:49674) 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f1021200640 1 -- 172.21.15.175:0/1774171740 learned_addr learned my addr 172.21.15.175:0/1774171740 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f101b400640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.717+0000 7f101be00640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e2dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1021200640 1 -- 172.21.15.175:0/1774171740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 msgr2=0x557f32e2dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1021200640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e2dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1021200640 1 -- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 msgr2=0x557f32e338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1021200640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1021200640 1 -- 172.21.15.175:0/1774171740 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f32d451e0 con 0x557f32e33f30 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f101be00640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e2dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f101b400640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1021200640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e2e170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f101401bf40 tx=0x7f1014024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db107d3cc2dddbd4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:23.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1006000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1006000640 1 -- 172.21.15.175:0/1774171740 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014035060 con 0x557f32e33f30 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f101401b3f0 con 0x557f32e33f30 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.718+0000 7f1006000640 1 -- 172.21.15.175:0/1774171740 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f101401b7a0 con 0x557f32e33f30 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1006000640 1 -- 172.21.15.175:0/1774171740 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1014009bf0 con 0x557f32e33f30 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 10 client.26383 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557f32e4b220 con 0x557f32e33f30 2024-07-29T00:41:23.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1006000640 1 -- 172.21.15.175:0/1774171740 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f101401b3f0 con 0x557f32e33f30 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1006000640 1 -- 172.21.15.175:0/1774171740 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f101400eb70 con 0x557f32e33f30 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 10 client.26383 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 10 client.26383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 20 client.26383 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32e4c570 0x557f32e4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 4 client.26383 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.719+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f32d94020 con 0x557f32e4c570 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.720+0000 7f101be00640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32e4c570 0x557f32e4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.720+0000 7f101be00640 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32e4c570 0x557f32e4e960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a86ed5d9e77b18fa server_cookie=12bc5e694ec39bde in_seq=0 out_seq=0 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.720+0000 7f1006000640 10 client.26383 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.721+0000 7f1006000640 1 -- 172.21.15.175:0/1774171740 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f32d94020 con 0x557f32e4c570 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.721+0000 7f1006000640 10 client.26383 handle_command_reply: tid=0 2024-07-29T00:41:23.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.721+0000 7f1025e3a740 1 client.26383 shutdown 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 msgr2=0x557f32e2e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e2e170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f101401bf40 tx=0x7f1014024000 comp rx=0 tx=0).stop 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 shutdown_connections 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32e4c570 0x557f32e4e960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e2e170 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1774171740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e2dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 >> 172.21.15.175:0/1774171740 conn(0x557f32de9950 msgr2=0x557f32e22d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1006000640 0 client.26383 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 shutdown_connections 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.722+0000 7f1025e3a740 1 -- 172.21.15.175:0/1774171740 wait complete. 2024-07-29T00:41:23.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.723+0000 7f1025e3a740 20 client.26383 trim_cache size 0 max 16384 2024-07-29T00:41:23.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.738+0000 7f1025e3a740 1 Processor -- start 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.738+0000 7f1025e3a740 1 -- start start 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.738+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 0x557f32e338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.738+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e2e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e33f30 0x557f32e53f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f1025e3a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1ab580 con 0x557f32e21ef0 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f1025e3a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1ad120 con 0x557f32e33f30 2024-07-29T00:41:23.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f1025e3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1abc50 con 0x557f32e20980 2024-07-29T00:41:23.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 0x557f32e338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 0x557f32e338c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49684/0 (socket says 172.21.15.175:49684) 2024-07-29T00:41:23.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 -- 172.21.15.175:0/974949221 learned_addr learned my addr 172.21.15.175:0/974949221 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:23.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f1021200640 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e33f30 0x557f32e53f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101b400640 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e2e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 -- 172.21.15.175:0/974949221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e33f30 msgr2=0x557f32e53f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e33f30 0x557f32e53f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 -- 172.21.15.175:0/974949221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 msgr2=0x557f32e2e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e2e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101be00640 1 -- 172.21.15.175:0/974949221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f32e23440 con 0x557f32e20980 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.739+0000 7f101b400640 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e2e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.740+0000 7f101be00640 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 0x557f32e338c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x557f32e4eed0 tx=0x7f100c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28cb654d5fa29e0c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.740+0000 7f1010c00640 1 -- 172.21.15.175:0/974949221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0122d0 con 0x557f32e20980 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.740+0000 7f1010c00640 1 -- 172.21.15.175:0/974949221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100c012470 con 0x557f32e20980 2024-07-29T00:41:23.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.740+0000 7f1010c00640 1 -- 172.21.15.175:0/974949221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0108d0 con 0x557f32e20980 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 -- 172.21.15.175:0/974949221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 msgr2=0x557f32e338c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 0x557f32e338c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x557f32e4eed0 tx=0x7f100c0046c0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 -- 172.21.15.175:0/974949221 shutdown_connections 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e33f30 0x557f32e53f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e2e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 --2- 172.21.15.175:0/974949221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e20980 0x557f32e338c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 -- 172.21.15.175:0/974949221 >> 172.21.15.175:0/974949221 conn(0x557f32de9950 msgr2=0x557f32e2d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 -- 172.21.15.175:0/974949221 shutdown_connections 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.741+0000 7f1025e3a740 1 -- 172.21.15.175:0/974949221 wait complete. 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.742+0000 7f1025e3a740 1 -- start start 2024-07-29T00:41:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.742+0000 7f1025e3a740 1 Processor -- start 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e5fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e60350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e5b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1aca50 con 0x557f32e21ef0 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1ac320 con 0x557f32e20980 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1a9500 con 0x557f32e33f30 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1025e3a740 10 client.0 _ll_register_callbacks cb 0x7f100c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:23.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e5fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e5fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57238/0 (socket says 172.21.15.175:57238) 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 -- 172.21.15.175:0/1077814176 learned_addr learned my addr 172.21.15.175:0/1077814176 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1021200640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e5b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101b400640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e60350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 -- 172.21.15.175:0/1077814176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 msgr2=0x557f32e5b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e5b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 -- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 msgr2=0x557f32e60350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e60350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f101be00640 1 -- 172.21.15.175:0/1077814176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f32e5ba50 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.743+0000 7f1021200640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e5b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f101be00640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e5fe10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f100c010540 tx=0x7f100c0057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f1abb95abef7f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f1006000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f1006000640 1 -- 172.21.15.175:0/1077814176 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c034060 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f1006000640 1 -- 172.21.15.175:0/1077814176 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f100c00f870 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100c012470 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f1006000640 1 -- 172.21.15.175:0/1077814176 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f100c00fa70 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.744+0000 7f1025e3a740 10 client.26426 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557f32eb68a0 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1006000640 1 -- 172.21.15.175:0/1077814176 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100c012470 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1006000640 1 -- 172.21.15.175:0/1077814176 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f100c026450 con 0x557f32e20980 2024-07-29T00:41:23.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 10 client.26426 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 10 client.26426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 20 client.26426 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32eb6f10 0x557f32eb9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 4 client.26426 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f32d8e530 con 0x557f32eb6f10 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.745+0000 7f101b400640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32eb6f10 0x557f32eb9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f101b400640 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32eb6f10 0x557f32eb9330 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=944cb5de9f3fd17a server_cookie=bd09693ff596fb98 in_seq=0 out_seq=0 2024-07-29T00:41:23.748 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f1006000640 10 client.26426 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f1006000640 1 -- 172.21.15.175:0/1077814176 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f32d8e530 con 0x557f32eb6f10 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f1006000640 10 client.26426 handle_command_reply: tid=0 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f1025e3a740 1 client.26426 shutdown 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 msgr2=0x557f32e5fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.746+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e5fe10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f100c010540 tx=0x7f100c0057d0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 shutdown_connections 2024-07-29T00:41:23.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x557f32eb6f10 0x557f32eb9330 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f32e33f30 0x557f32e5b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f32e21ef0 0x557f32e60350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 --2- 172.21.15.175:0/1077814176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f32e20980 0x557f32e5fe10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 >> 172.21.15.175:0/1077814176 conn(0x557f32de9950 msgr2=0x557f32e22430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1006000640 0 client.26426 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 shutdown_connections 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 1 -- 172.21.15.175:0/1077814176 wait complete. 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.747+0000 7f1025e3a740 20 client.26426 trim_cache size 0 max 16384 2024-07-29T00:41:23.750 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:41:23.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f10000a2d70 msgr2=0x7f10000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f10000a2d70 0x7f10000a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1014011fd0 tx=0x7f1014011f60 comp rx=0 tx=0).stop 2024-07-29T00:41:23.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 msgr2=0x7f101c19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:23.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c19a8a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1008004770 tx=0x7f1008003b60 comp rx=0 tx=0).stop 2024-07-29T00:41:23.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 shutdown_connections 2024-07-29T00:41:23.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f10000a2d70 0x7f10000a5230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f101c10be60 0x7f101c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101c103350 0x7f101c19a8a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 --2- 172.21.15.175:0/2488524249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c102a30 0x7f101c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:23.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 >> 172.21.15.175:0/2488524249 conn(0x7f101c0fc490 msgr2=0x7f101c10d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:23.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 shutdown_connections 2024-07-29T00:41:23.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:23.748+0000 7f1023200640 1 -- 172.21.15.175:0/2488524249 wait complete. 2024-07-29T00:41:23.752 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:23 smithi181 ceph-mon[36706]: pgmap v149: 97 pgs: 97 active+clean; 85 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 905 KiB/s wr, 154 op/s 2024-07-29T00:41:23.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:23 smithi204 ceph-mon[37368]: pgmap v149: 97 pgs: 97 active+clean; 85 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 905 KiB/s wr, 154 op/s 2024-07-29T00:41:23.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 132256 2.08 MB/sec execute 74 sec latency 20.496 ms 2024-07-29T00:41:24.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 133708 2.13 MB/sec execute 75 sec latency 22.214 ms 2024-07-29T00:41:25.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:41:25.484 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:25.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:25 smithi181 ceph-mon[36706]: pgmap v150: 97 pgs: 97 active+clean; 85 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 641 KiB/s wr, 106 op/s 2024-07-29T00:41:25.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:25 smithi175 ceph-mon[32138]: pgmap v150: 97 pgs: 97 active+clean; 85 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 641 KiB/s wr, 106 op/s 2024-07-29T00:41:25.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:25 smithi204 ceph-mon[37368]: pgmap v150: 97 pgs: 97 active+clean; 85 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 641 KiB/s wr, 106 op/s 2024-07-29T00:41:25.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 135067 2.26 MB/sec execute 76 sec latency 20.788 ms 2024-07-29T00:41:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 -- 172.21.15.175:0/437631524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18081126d0 msgr2=0x7f1808114ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 --2- 172.21.15.175:0/437631524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18081126d0 0x7f1808114ac0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1800009860 tx=0x7f180002a3d0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 -- 172.21.15.175:0/437631524 shutdown_connections 2024-07-29T00:41:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 --2- 172.21.15.175:0/437631524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18081126d0 0x7f1808114ac0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 --2- 172.21.15.175:0/437631524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1808074ba0 0x7f1808112190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 --2- 172.21.15.175:0/437631524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 0x7f1808074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 -- 172.21.15.175:0/437631524 >> 172.21.15.175:0/437631524 conn(0x7f18080fd400 msgr2=0x7f18080ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:26.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.176+0000 7f180e800640 1 -- 172.21.15.175:0/437631524 shutdown_connections 2024-07-29T00:41:26.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.177+0000 7f180e800640 1 -- 172.21.15.175:0/437631524 wait complete. 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 Processor -- start 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 -- start start 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 0x7f180819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1808074ba0 0x7f180819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081126d0 0x7f18081a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18081180f0 con 0x7f1808074280 2024-07-29T00:41:26.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1808117f70 con 0x7f1808074ba0 2024-07-29T00:41:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.178+0000 7f180e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1808118270 con 0x7f18081126d0 2024-07-29T00:41:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 0x7f180819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1808074ba0 0x7f180819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 0x7f180819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44982/0 (socket says 172.21.15.175:44982) 2024-07-29T00:41:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180d400640 1 -- 172.21.15.175:0/2016740753 learned_addr learned my addr 172.21.15.175:0/2016740753 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:26.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180de00640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081126d0 0x7f18081a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180ca00640 1 -- 172.21.15.175:0/2016740753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081126d0 msgr2=0x7f18081a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180ca00640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081126d0 0x7f18081a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180ca00640 1 -- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 msgr2=0x7f180819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180ca00640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 0x7f180819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.179+0000 7f180ca00640 1 -- 172.21.15.175:0/2016740753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081a3520 con 0x7f1808074ba0 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.180+0000 7f180ca00640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1808074ba0 0x7f180819eab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.180+0000 7f1806000640 1 -- 172.21.15.175:0/2016740753 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000ccc0 con 0x7f1808074ba0 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.180+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081a3810 con 0x7f1808074ba0 2024-07-29T00:41:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.180+0000 7f1806000640 1 -- 172.21.15.175:0/2016740753 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0005020 con 0x7f1808074ba0 2024-07-29T00:41:26.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.180+0000 7f1806000640 1 -- 172.21.15.175:0/2016740753 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00049e0 con 0x7f1808074ba0 2024-07-29T00:41:26.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.180+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081ab0f0 con 0x7f1808074ba0 2024-07-29T00:41:26.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 Processor -- start 2024-07-29T00:41:26.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 -- start start 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1806000640 1 -- 172.21.15.175:0/2016740753 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f17f0005560 con 0x7f1808074ba0 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed5850e0 0x5595ed583630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed583b70 0x5595ed583fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed58e910 0x5595ed590d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595ed4e9ab0 con 0x5595ed58e910 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5595ed481a80 con 0x5595ed5850e0 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.184+0000 7f1812818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595ed4e5a70 con 0x5595ed583b70 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed583b70 0x5595ed583fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed5850e0 0x5595ed583630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed5850e0 0x5595ed583630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57258/0 (socket says 172.21.15.175:57258) 2024-07-29T00:41:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180de00640 1 -- 172.21.15.175:0/1845021296 learned_addr learned my addr 172.21.15.175:0/1845021296 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:26.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180ca00640 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed58e910 0x5595ed590d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180d400640 1 -- 172.21.15.175:0/1845021296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed5850e0 msgr2=0x5595ed583630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180d400640 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed5850e0 0x5595ed583630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180d400640 1 -- 172.21.15.175:0/1845021296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed58e910 msgr2=0x5595ed590d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180d400640 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed58e910 0x5595ed590d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.185+0000 7f180d400640 1 -- 172.21.15.175:0/1845021296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ed25b3c0 con 0x5595ed583b70 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.186+0000 7f180d400640 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed583b70 0x5595ed583fb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e4d0 tx=0x7f17f80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83317d6c3fa6139 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:26.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.186+0000 7f17fe600640 1 -- 172.21.15.175:0/1845021296 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f801d070 con 0x5595ed583b70 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.186+0000 7f17fe600640 1 -- 172.21.15.175:0/1845021296 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f8003e40 con 0x5595ed583b70 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.186+0000 7f17fe600640 1 -- 172.21.15.175:0/1845021296 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800dc40 con 0x5595ed583b70 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.186+0000 7f1806000640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17ec0a2d70 0x7f17ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1806000640 1 -- 172.21.15.175:0/2016740753 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f17f00cf350 con 0x7f1808074ba0 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f180de00640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17ec0a2d70 0x7f17ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 -- 172.21.15.175:0/1845021296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed583b70 msgr2=0x5595ed583fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed583b70 0x5595ed583fb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e4d0 tx=0x7f17f80095c0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 -- 172.21.15.175:0/1845021296 shutdown_connections 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed58e910 0x5595ed590d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed583b70 0x5595ed583fb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 --2- 172.21.15.175:0/1845021296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed5850e0 0x5595ed583630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 -- 172.21.15.175:0/1845021296 >> 172.21.15.175:0/1845021296 conn(0x5595ed54c950 msgr2=0x5595ed580200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.187+0000 7f1812818740 1 -- 172.21.15.175:0/1845021296 shutdown_connections 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.188+0000 7f1812818740 1 -- 172.21.15.175:0/1845021296 wait complete. 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.188+0000 7f180de00640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17ec0a2d70 0x7f17ec0a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18000095f0 tx=0x7f1800005c90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.188+0000 7f1812818740 1 -- start start 2024-07-29T00:41:26.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 Processor -- start 2024-07-29T00:41:26.190 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed58b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed58bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed58c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595ed5916b0 con 0x5595ed583b70 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5595ed4a4c10 con 0x5595ed58e910 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595ed591830 con 0x5595ed5850e0 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.189+0000 7f1812818740 10 client.0 _ll_register_callbacks cb 0x5595ed46da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed58c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed58b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed58bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed58bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49724/0 (socket says 172.21.15.175:49724) 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 -- 172.21.15.175:0/2667825944 learned_addr learned my addr 172.21.15.175:0/2667825944 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed58b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45010/0 (socket says 172.21.15.175:45010) 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed58c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57274/0 (socket says 172.21.15.175:57274) 2024-07-29T00:41:26.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 -- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 msgr2=0x5595ed58c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed58c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 -- 172.21.15.175:0/2667825944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 msgr2=0x5595ed58b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed58b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180ca00640 1 -- 172.21.15.175:0/2667825944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ed4a81e0 con 0x5595ed5850e0 2024-07-29T00:41:26.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.190+0000 7f180d400640 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed58b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f180ca00640 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed58bda0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9c0 tx=0x7f17f0005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea37fe9ddb7e360f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f17f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f17f6a00640 1 -- 172.21.15.175:0/2667825944 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00ce020 con 0x5595ed5850e0 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f17f6a00640 1 -- 172.21.15.175:0/2667825944 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f17f00ccca0 con 0x5595ed5850e0 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595ed4f7020 con 0x5595ed5850e0 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f17f6a00640 1 -- 172.21.15.175:0/2667825944 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f17f001c070 con 0x5595ed5850e0 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f1812818740 10 client.26401 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:26.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.191+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5595ed59d490 con 0x5595ed5850e0 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f17f6a00640 1 -- 172.21.15.175:0/2667825944 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5595ed4f7020 con 0x5595ed5850e0 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f17f6a00640 1 -- 172.21.15.175:0/2667825944 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f17f00a3fd0 con 0x5595ed5850e0 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f1812818740 10 client.26401 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f1812818740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f1812818740 20 client.26401 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f1812818740 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed59e950 0x5595ed5a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f1812818740 4 client.26401 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595ed4f1530 con 0x5595ed59e950 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.192+0000 7f180d400640 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed59e950 0x5595ed5a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.193+0000 7f180d400640 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed59e950 0x5595ed5a0dc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842de2cab0538129 server_cookie=7842e52a7e9cc163 in_seq=0 out_seq=0 2024-07-29T00:41:26.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.193+0000 7f17f6a00640 10 client.26401 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:26.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.193+0000 7f17f6a00640 1 -- 172.21.15.175:0/2667825944 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595ed4f1530 con 0x5595ed59e950 2024-07-29T00:41:26.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.193+0000 7f17f6a00640 10 client.26401 handle_command_reply: tid=0 2024-07-29T00:41:26.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 client.26401 shutdown 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 msgr2=0x5595ed58bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed58bda0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9c0 tx=0x7f17f0005440 comp rx=0 tx=0).stop 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 shutdown_connections 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed59e950 0x5595ed5a0dc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f17f6a00640 0 client.26401 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed58c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed58bda0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 --2- 172.21.15.175:0/2667825944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed58b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 >> 172.21.15.175:0/2667825944 conn(0x5595ed54c950 msgr2=0x5595ed585f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 shutdown_connections 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.194+0000 7f1812818740 1 -- 172.21.15.175:0/2667825944 wait complete. 2024-07-29T00:41:26.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.195+0000 7f1812818740 20 client.26401 trim_cache size 0 max 16384 2024-07-29T00:41:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 Processor -- start 2024-07-29T00:41:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 -- start start 2024-07-29T00:41:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 0x5595ed58fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed5850e0 0x5595ed590240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed58e910 0x5595ed58a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18081b3ad0 con 0x5595ed5850e0 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18081b5670 con 0x5595ed583b70 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f1812818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18081b41a0 con 0x5595ed58e910 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f180d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 0x5595ed58fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f180d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 0x5595ed58fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:57286/0 (socket says 172.21.15.175:57286) 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f180d400640 1 -- 172.21.15.175:0/2458951200 learned_addr learned my addr 172.21.15.175:0/2458951200 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.212+0000 7f180de00640 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed58e910 0x5595ed58a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180ca00640 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed5850e0 0x5595ed590240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180d400640 1 -- 172.21.15.175:0/2458951200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed58e910 msgr2=0x5595ed58a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180d400640 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed58e910 0x5595ed58a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180d400640 1 -- 172.21.15.175:0/2458951200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed5850e0 msgr2=0x5595ed590240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180d400640 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed5850e0 0x5595ed590240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180d400640 1 -- 172.21.15.175:0/2458951200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ed581370 con 0x5595ed583b70 2024-07-29T00:41:26.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f180d400640 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 0x5595ed58fd00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5595ed581060 tx=0x7f17f8005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cc1ba9dd2802101 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f17fdc00640 1 -- 172.21.15.175:0/2458951200 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f801d040 con 0x5595ed583b70 2024-07-29T00:41:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f17fdc00640 1 -- 172.21.15.175:0/2458951200 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f801e040 con 0x5595ed583b70 2024-07-29T00:41:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.213+0000 7f17fdc00640 1 -- 172.21.15.175:0/2458951200 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80058d0 con 0x5595ed583b70 2024-07-29T00:41:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 -- 172.21.15.175:0/2458951200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 msgr2=0x5595ed58fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 0x5595ed58fd00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5595ed581060 tx=0x7f17f8005eb0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 -- 172.21.15.175:0/2458951200 shutdown_connections 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed58e910 0x5595ed58a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed5850e0 0x5595ed590240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 --2- 172.21.15.175:0/2458951200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed583b70 0x5595ed58fd00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.214+0000 7f1812818740 1 -- 172.21.15.175:0/2458951200 >> 172.21.15.175:0/2458951200 conn(0x5595ed54c950 msgr2=0x5595ed57b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.215+0000 7f1812818740 1 -- 172.21.15.175:0/2458951200 shutdown_connections 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.215+0000 7f1812818740 1 -- 172.21.15.175:0/2458951200 wait complete. 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.215+0000 7f1812818740 1 -- start start 2024-07-29T00:41:26.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.215+0000 7f1812818740 1 Processor -- start 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed596af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed597030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed588ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18081b4fa0 con 0x5595ed583b70 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18081b4870 con 0x5595ed58e910 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18081b1ab0 con 0x5595ed5850e0 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f1812818740 10 client.0 _ll_register_callbacks cb 0x7f17f00d2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:26.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f180ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed597030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f180d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed596af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f180ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed597030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49738/0 (socket says 172.21.15.175:49738) 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f180ca00640 1 -- 172.21.15.175:0/390434803 learned_addr learned my addr 172.21.15.175:0/390434803 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.216+0000 7f180de00640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed588ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180ca00640 1 -- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 msgr2=0x5595ed588ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180ca00640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed588ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180ca00640 1 -- 172.21.15.175:0/390434803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 msgr2=0x5595ed596af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180ca00640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed596af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180ca00640 1 -- 172.21.15.175:0/390434803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ed589020 con 0x5595ed5850e0 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180de00640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed588ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f180ca00640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed597030 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f17f0098170 tx=0x7f17f0004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7b3f55fb097d760 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:26.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f17f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:26.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f17f6a00640 1 -- 172.21.15.175:0/390434803 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00ce040 con 0x5595ed5850e0 2024-07-29T00:41:26.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f17f6a00640 1 -- 172.21.15.175:0/390434803 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f17f009ae40 con 0x5595ed5850e0 2024-07-29T00:41:26.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.217+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f801e040 con 0x5595ed5850e0 2024-07-29T00:41:26.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f17f6a00640 1 -- 172.21.15.175:0/390434803 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f17f009b5c0 con 0x5595ed5850e0 2024-07-29T00:41:26.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 10 client.26407 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:26.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5595ed5a5da0 con 0x5595ed5850e0 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f17f6a00640 1 -- 172.21.15.175:0/390434803 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f801e040 con 0x5595ed5850e0 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f17f6a00640 1 -- 172.21.15.175:0/390434803 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f17f0097540 con 0x5595ed5850e0 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 10 client.26407 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 10 client.26407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 20 client.26407 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed609660 0x5595ed60ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 4 client.26407 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.218+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595ed46e010 con 0x5595ed609660 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f180d400640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed609660 0x5595ed60ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f180d400640 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed609660 0x5595ed60ba50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d48a16c01b1992 server_cookie=70b83d0d5c814a45 in_seq=0 out_seq=0 2024-07-29T00:41:26.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f17f6a00640 10 client.26407 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f17f6a00640 1 -- 172.21.15.175:0/390434803 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595ed46e010 con 0x5595ed609660 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f17f6a00640 10 client.26407 handle_command_reply: tid=0 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 client.26407 shutdown 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 msgr2=0x5595ed597030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed597030 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f17f0098170 tx=0x7f17f0004ed0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 shutdown_connections 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5595ed609660 0x5595ed60ba50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595ed58e910 0x5595ed588ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ed5850e0 0x5595ed597030 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 --2- 172.21.15.175:0/390434803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595ed583b70 0x5595ed596af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f17f6a00640 0 client.26407 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:26.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.219+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 >> 172.21.15.175:0/390434803 conn(0x5595ed54c950 msgr2=0x5595ed58a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:26.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.220+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 shutdown_connections 2024-07-29T00:41:26.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.220+0000 7f1812818740 1 -- 172.21.15.175:0/390434803 wait complete. 2024-07-29T00:41:26.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.220+0000 7f1812818740 20 client.26407 trim_cache size 0 max 16384 2024-07-29T00:41:26.223 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:41:26.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17ec0a2d70 msgr2=0x7f17ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17ec0a2d70 0x7f17ec0a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18000095f0 tx=0x7f1800005c90 comp rx=0 tx=0).stop 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1808074ba0 msgr2=0x7f180819eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1808074ba0 0x7f180819eab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 shutdown_connections 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081126d0 0x7f18081a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1808074ba0 0x7f180819eab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f17ec0a2d70 0x7f17ec0a5230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 --2- 172.21.15.175:0/2016740753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1808074280 0x7f180819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 >> 172.21.15.175:0/2016740753 conn(0x7f18080fd400 msgr2=0x7f18080fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 shutdown_connections 2024-07-29T00:41:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:26.221+0000 7f180e800640 1 -- 172.21.15.175:0/2016740753 wait complete. 2024-07-29T00:41:26.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 135610 2.24 MB/sec execute 77 sec latency 22.457 ms 2024-07-29T00:41:27.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:27 smithi181 ceph-mon[36706]: pgmap v151: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-07-29T00:41:27.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:27 smithi175 ceph-mon[32138]: pgmap v151: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-07-29T00:41:27.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:27 smithi204 ceph-mon[37368]: pgmap v151: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-07-29T00:41:27.889 INFO:tasks.workunit.client.0.smithi175.stdout: 1 136479 2.27 MB/sec execute 78 sec latency 16.726 ms 2024-07-29T00:41:28.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:41:28.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:41:28.699 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:28.890 INFO:tasks.workunit.client.0.smithi175.stdout: 1 138179 2.36 MB/sec execute 79 sec latency 17.293 ms 2024-07-29T00:41:29.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.389+0000 7f5bf7400640 1 -- 172.21.15.175:0/1903959559 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0319b0 con 0x7f5bf8108ee0 2024-07-29T00:41:29.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 -- 172.21.15.175:0/1903959559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 msgr2=0x7f5bf81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 --2- 172.21.15.175:0/1903959559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf81092c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009900 tx=0x7f5bec02a490 comp rx=0 tx=0).stop 2024-07-29T00:41:29.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 -- 172.21.15.175:0/1903959559 shutdown_connections 2024-07-29T00:41:29.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 --2- 172.21.15.175:0/1903959559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf8116b90 0x7f5bf8118f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 --2- 172.21.15.175:0/1903959559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8109800 0x7f5bf8116650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 --2- 172.21.15.175:0/1903959559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf81092c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 -- 172.21.15.175:0/1903959559 >> 172.21.15.175:0/1903959559 conn(0x7f5bf8073350 msgr2=0x7f5bf8073760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:29.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 -- 172.21.15.175:0/1903959559 shutdown_connections 2024-07-29T00:41:29.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.391+0000 7f5bfde00640 1 -- 172.21.15.175:0/1903959559 wait complete. 2024-07-29T00:41:29.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 Processor -- start 2024-07-29T00:41:29.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 -- start start 2024-07-29T00:41:29.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf8114b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf8109800 0x7f5bf81150d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8116b90 0x7f5bf81b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf810cfb0 con 0x7f5bf8109800 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf810ce30 con 0x7f5bf8108ee0 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.393+0000 7f5bfde00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf810d130 con 0x7f5bf8116b90 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf8114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf8114b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:36888/0 (socket says 172.21.15.175:36888) 2024-07-29T00:41:29.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 -- 172.21.15.175:0/3726466361 learned_addr learned my addr 172.21.15.175:0/3726466361 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:29.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf8109800 0x7f5bf81150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8116b90 0x7f5bf81b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 -- 172.21.15.175:0/3726466361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8116b90 msgr2=0x7f5bf81b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8116b90 0x7f5bf81b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 -- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf8109800 msgr2=0x7f5bf81150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf8109800 0x7f5bf81150d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.394+0000 7f5bfca00640 1 -- 172.21.15.175:0/3726466361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf81b31b0 con 0x7f5bf8108ee0 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.395+0000 7f5bfca00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf8114b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5bec006010 tx=0x7f5bec0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.395+0000 7f5bf5600640 1 -- 172.21.15.175:0/3726466361 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03f030 con 0x7f5bf8108ee0 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.395+0000 7f5bf5600640 1 -- 172.21.15.175:0/3726466361 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bec003ce0 con 0x7f5bf8108ee0 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.395+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf81b3380 con 0x7f5bf8108ee0 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.395+0000 7f5bf5600640 1 -- 172.21.15.175:0/3726466361 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec035070 con 0x7f5bf8108ee0 2024-07-29T00:41:29.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.395+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf81b3860 con 0x7f5bf8108ee0 2024-07-29T00:41:29.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.398+0000 7f5bf5600640 1 -- 172.21.15.175:0/3726466361 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5bec042030 con 0x7f5bf8108ee0 2024-07-29T00:41:29.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.399+0000 7f5c02052740 1 Processor -- start 2024-07-29T00:41:29.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.399+0000 7f5c02052740 1 -- start start 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.399+0000 7f5c02052740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5c02052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5c02052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384ebf00 0x564b384ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5c02052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b384468a0 con 0x564b384e1640 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5c02052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564b383de710 con 0x564b384ebf00 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5c02052740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564b38442860 con 0x564b384e0d20 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44996/0 (socket says 172.21.15.175:44996) 2024-07-29T00:41:29.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 -- 172.21.15.175:0/3123212251 learned_addr learned my addr 172.21.15.175:0/3123212251 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:29.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384ebf00 0x564b384ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfca00640 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 -- 172.21.15.175:0/3123212251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384ebf00 msgr2=0x564b384ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384ebf00 0x564b384ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 -- 172.21.15.175:0/3123212251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 msgr2=0x564b384eb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384eb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.400+0000 7f5bfd400640 1 -- 172.21.15.175:0/3123212251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b381b86a0 con 0x564b384e0d20 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384ebf00 0x564b384ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e1100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009f10 tx=0x7f5bf000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=143b7a2f48b7f6a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5be9c00640 1 -- 172.21.15.175:0/3123212251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0015070 con 0x564b384e0d20 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5bf5600640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5be00a2d70 0x7f5be00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5be9c00640 1 -- 172.21.15.175:0/3123212251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf001ab40 con 0x564b384e0d20 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5be9c00640 1 -- 172.21.15.175:0/3123212251 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0011be0 con 0x564b384e0d20 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5bf5600640 1 -- 172.21.15.175:0/3726466361 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f5bec0ef5c0 con 0x7f5bf8108ee0 2024-07-29T00:41:29.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.401+0000 7f5bfca00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5be00a2d70 0x7f5be00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.402+0000 7f5c02052740 1 -- 172.21.15.175:0/3123212251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 msgr2=0x564b384e1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.402+0000 7f5c02052740 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e1100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009f10 tx=0x7f5bf000f730 comp rx=0 tx=0).stop 2024-07-29T00:41:29.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.402+0000 7f5bfca00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5be00a2d70 0x7f5be00a5230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564b384de110 tx=0x7f5bec02a490 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:29.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 -- 172.21.15.175:0/3123212251 shutdown_connections 2024-07-29T00:41:29.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384ebf00 0x564b384ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384eb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 --2- 172.21.15.175:0/3123212251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e1100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 -- 172.21.15.175:0/3123212251 >> 172.21.15.175:0/3123212251 conn(0x564b384dd2b0 msgr2=0x564b384dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 -- 172.21.15.175:0/3123212251 shutdown_connections 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.403+0000 7f5c02052740 1 -- 172.21.15.175:0/3123212251 wait complete. 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.404+0000 7f5c02052740 1 -- start start 2024-07-29T00:41:29.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.404+0000 7f5c02052740 1 Processor -- start 2024-07-29T00:41:29.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.404+0000 7f5c02052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 0x564b384e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5c02052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5c02052740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384ebf00 0x564b384e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5c02052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b384eead0 con 0x564b384e1640 2024-07-29T00:41:29.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5c02052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564b384018c0 con 0x564b384e0d20 2024-07-29T00:41:29.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5c02052740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564b384eec50 con 0x564b384ebf00 2024-07-29T00:41:29.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5c02052740 10 client.0 _ll_register_callbacks cb 0x564b383eda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:29.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 0x564b384e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 0x564b384e8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:36914/0 (socket says 172.21.15.175:36914) 2024-07-29T00:41:29.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5bfd400640 1 -- 172.21.15.175:0/2481390318 learned_addr learned my addr 172.21.15.175:0/2481390318 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:29.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.405+0000 7f5bfd400640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384ebf00 0x564b384e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bf7e00640 1 -- 172.21.15.175:0/2481390318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384ebf00 msgr2=0x564b384e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384ebf00 0x564b384e97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bf7e00640 1 -- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 msgr2=0x564b384e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 0x564b384e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bf7e00640 1 -- 172.21.15.175:0/2481390318 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b383e4bc0 con 0x564b384e1640 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bfd400640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 0x564b384e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384e92a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005fd0 tx=0x7f5be400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf88abe18da669f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bde000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bde000640 1 -- 172.21.15.175:0/2481390318 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4011440 con 0x564b384e1640 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5bde000640 1 -- 172.21.15.175:0/2481390318 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f5be40115e0 con 0x564b384e1640 2024-07-29T00:41:29.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.406+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf001ab40 con 0x564b384e1640 2024-07-29T00:41:29.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5bde000640 1 -- 172.21.15.175:0/2481390318 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5be40117e0 con 0x564b384e1640 2024-07-29T00:41:29.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 10 client.16326 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:29.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b384fabb0 con 0x564b384e1640 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5bde000640 1 -- 172.21.15.175:0/2481390318 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf001ab40 con 0x564b384e1640 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5bde000640 1 -- 172.21.15.175:0/2481390318 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f5be4019030 con 0x564b384e1640 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 10 client.16326 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 20 client.16326 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b384fc110 0x564b384fe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 4 client.16326 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.407+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b38406c90 con 0x564b384fc110 2024-07-29T00:41:29.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.408+0000 7f5bfd400640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b384fc110 0x564b384fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.408+0000 7f5bfd400640 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b384fc110 0x564b384fe560 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbf6397554cb1c7 server_cookie=9d531aad80aeeb92 in_seq=0 out_seq=0 2024-07-29T00:41:29.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.408+0000 7f5bde000640 10 client.16326 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.409+0000 7f5bde000640 1 -- 172.21.15.175:0/2481390318 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b38406c90 con 0x564b384fc110 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.409+0000 7f5bde000640 10 client.16326 handle_command_reply: tid=0 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.409+0000 7f5c02052740 1 client.16326 shutdown 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.409+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 msgr2=0x564b384e92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.409+0000 7f5c02052740 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384e92a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005fd0 tx=0x7f5be400de50 comp rx=0 tx=0).stop 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 shutdown_connections 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b384fc110 0x564b384fe560 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384ebf00 0x564b384e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384e1640 0x564b384e92a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 --2- 172.21.15.175:0/2481390318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e0d20 0x564b384e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 >> 172.21.15.175:0/2481390318 conn(0x564b384dd2b0 msgr2=0x564b384e2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5bde000640 0 client.16326 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 shutdown_connections 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 1 -- 172.21.15.175:0/2481390318 wait complete. 2024-07-29T00:41:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.410+0000 7f5c02052740 20 client.16326 trim_cache size 0 max 16384 2024-07-29T00:41:29.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 Processor -- start 2024-07-29T00:41:29.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 -- start start 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b38503e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b385043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd800e850 con 0x564b384ebf00 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80103f0 con 0x564b384e1640 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.433+0000 7f5c02052740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd800ef20 con 0x564b384e0d20 2024-07-29T00:41:29.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b385043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b38503e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bfd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384eacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45018/0 (socket says 172.21.15.175:45018) 2024-07-29T00:41:29.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bfd400640 1 -- 172.21.15.175:0/3129334637 learned_addr learned my addr 172.21.15.175:0/3129334637 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:29.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bf7e00640 1 -- 172.21.15.175:0/3129334637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 msgr2=0x564b384eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bf7e00640 1 -- 172.21.15.175:0/3129334637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 msgr2=0x564b385043b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b385043b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.434+0000 7f5bf7e00640 1 -- 172.21.15.175:0/3129334637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b384e1470 con 0x564b384e1640 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.435+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b38503e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5be401ac30 tx=0x7f5be40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d93a966dc6c01 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.435+0000 7f5be9200640 1 -- 172.21.15.175:0/3129334637 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4026040 con 0x564b384e1640 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.435+0000 7f5be9200640 1 -- 172.21.15.175:0/3129334637 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be4011440 con 0x564b384e1640 2024-07-29T00:41:29.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.435+0000 7f5be9200640 1 -- 172.21.15.175:0/3129334637 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40054d0 con 0x564b384e1640 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 -- 172.21.15.175:0/3129334637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 msgr2=0x564b38503e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b38503e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5be401ac30 tx=0x7f5be40057f0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 -- 172.21.15.175:0/3129334637 shutdown_connections 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b385043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b38503e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 --2- 172.21.15.175:0/3129334637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 -- 172.21.15.175:0/3129334637 >> 172.21.15.175:0/3129334637 conn(0x564b384dd2b0 msgr2=0x564b384ea5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:29.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 -- 172.21.15.175:0/3129334637 shutdown_connections 2024-07-29T00:41:29.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.436+0000 7f5c02052740 1 -- 172.21.15.175:0/3129334637 wait complete. 2024-07-29T00:41:29.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.437+0000 7f5c02052740 1 -- start start 2024-07-29T00:41:29.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.437+0000 7f5c02052740 1 Processor -- start 2024-07-29T00:41:29.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.438 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b384e6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b384e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd800fd20 con 0x564b384ebf00 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd800f5f0 con 0x564b384e1640 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd800c830 con 0x564b384e0d20 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5c02052740 10 client.0 _ll_register_callbacks cb 0x7f5be40043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b384e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b384e6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:36922/0 (socket says 172.21.15.175:36922) 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bf7e00640 1 -- 172.21.15.175:0/3505259769 learned_addr learned my addr 172.21.15.175:0/3505259769 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b384e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 -- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 msgr2=0x564b384e6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b384e6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 -- 172.21.15.175:0/3505259769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 msgr2=0x564b384e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:29.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b384e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.438+0000 7f5bfd400640 1 -- 172.21.15.175:0/3505259769 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b38565e40 con 0x564b384e0d20 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.439+0000 7f5bfd400640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e5dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009730 tx=0x7f5bf001c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c48731befd8c784a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.439+0000 7f5bde000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.439+0000 7f5bde000640 1 -- 172.21.15.175:0/3505259769 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0023070 con 0x564b384e0d20 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.439+0000 7f5bde000640 1 -- 172.21.15.175:0/3505259769 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f5bf0011af0 con 0x564b384e0d20 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.439+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be4011440 con 0x564b384e0d20 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5bde000640 1 -- 172.21.15.175:0/3505259769 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5bf0015040 con 0x564b384e0d20 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 10 client.26425 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b38567120 con 0x564b384e0d20 2024-07-29T00:41:29.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5bde000640 1 -- 172.21.15.175:0/3505259769 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be4011440 con 0x564b384e0d20 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5bde000640 1 -- 172.21.15.175:0/3505259769 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f5bf001a8b0 con 0x564b384e0d20 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 10 client.26425 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 10 client.26425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 20 client.26425 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b385675f0 0x564b385699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 4 client.26425 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.440+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564b38453e10 con 0x564b385675f0 2024-07-29T00:41:29.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.442+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b385675f0 0x564b385699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.442+0000 7f5bf7e00640 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b385675f0 0x564b385699e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ee327012ab3f03 server_cookie=9cc7a14c794131f9 in_seq=0 out_seq=0 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.442+0000 7f5bde000640 10 client.26425 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.443+0000 7f5bde000640 1 -- 172.21.15.175:0/3505259769 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b38453e10 con 0x564b385675f0 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.443+0000 7f5bde000640 10 client.26425 handle_command_reply: tid=0 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.443+0000 7f5c02052740 1 client.26425 shutdown 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.443+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 msgr2=0x564b384e5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.443+0000 7f5c02052740 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e5dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009730 tx=0x7f5bf001c040 comp rx=0 tx=0).stop 2024-07-29T00:41:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 shutdown_connections 2024-07-29T00:41:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564b385675f0 0x564b385699e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b384ebf00 0x564b384e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b384e1640 0x564b384e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 --2- 172.21.15.175:0/3505259769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b384e0d20 0x564b384e5dc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 >> 172.21.15.175:0/3505259769 conn(0x564b384dd2b0 msgr2=0x564b384dbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5bde000640 0 client.26425 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 shutdown_connections 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 1 -- 172.21.15.175:0/3505259769 wait complete. 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.444+0000 7f5c02052740 20 client.26425 trim_cache size 0 max 16384 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:41:29.445 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "9d8fb3a9-7c5b-4205-b178-1acaf0398caf", 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5be00a2d70 msgr2=0x7f5be00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5be00a2d70 0x7f5be00a5230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564b384de110 tx=0x7f5bec02a490 comp rx=0 tx=0).stop 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 msgr2=0x7f5bf8114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf8114b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5bec006010 tx=0x7f5bec0042f0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 shutdown_connections 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8116b90 0x7f5bf81b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5be00a2d70 0x7f5be00a5230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf8109800 0x7f5bf81150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 --2- 172.21.15.175:0/3726466361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8108ee0 0x7f5bf8114b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 >> 172.21.15.175:0/3726466361 conn(0x7f5bf8073350 msgr2=0x7f5bf81170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 shutdown_connections 2024-07-29T00:41:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:29.445+0000 7f5bfde00640 1 -- 172.21.15.175:0/3726466361 wait complete. 2024-07-29T00:41:29.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:29 smithi175 ceph-mon[32138]: pgmap v152: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 760 KiB/s wr, 125 op/s 2024-07-29T00:41:29.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:29 smithi181 ceph-mon[36706]: pgmap v152: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 760 KiB/s wr, 125 op/s 2024-07-29T00:41:29.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:29 smithi204 ceph-mon[37368]: pgmap v152: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 760 KiB/s wr, 125 op/s 2024-07-29T00:41:29.890 INFO:tasks.workunit.client.0.smithi175.stdout: 1 138814 2.39 MB/sec execute 80 sec latency 29.096 ms 2024-07-29T00:41:30.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:41:30.701 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.702 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:30 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:30.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:30 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:41:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:30.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:30 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:30.890 INFO:tasks.workunit.client.0.smithi175.stdout: 1 139792 2.42 MB/sec execute 81 sec latency 20.310 ms 2024-07-29T00:41:31.102 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:41:31.525 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:31 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:31.525 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:31 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:31.525 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:31 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:41:31.525 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:31 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:31.525 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:31 smithi175 ceph-mon[32138]: pgmap v153: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 760 KiB/s wr, 125 op/s 2024-07-29T00:41:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:31 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:31 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:31 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:41:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:31 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:31.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:31 smithi181 ceph-mon[36706]: pgmap v153: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 760 KiB/s wr, 125 op/s 2024-07-29T00:41:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:31 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:31 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:31 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:41:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:31 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:41:31.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:31 smithi204 ceph-mon[37368]: pgmap v153: 97 pgs: 97 active+clean; 89 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 760 KiB/s wr, 125 op/s 2024-07-29T00:41:31.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 -- 172.21.15.175:0/401737691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 msgr2=0x7f639c1078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 --2- 172.21.15.175:0/401737691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c1078b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6388009930 tx=0x7f638802a420 comp rx=0 tx=0).stop 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 -- 172.21.15.175:0/401737691 shutdown_connections 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 --2- 172.21.15.175:0/401737691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c107ff0 0x7f639c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 --2- 172.21.15.175:0/401737691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c1078b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 --2- 172.21.15.175:0/401737691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 -- 172.21.15.175:0/401737691 >> 172.21.15.175:0/401737691 conn(0x7f639c0fc4c0 msgr2=0x7f639c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 -- 172.21.15.175:0/401737691 shutdown_connections 2024-07-29T00:41:31.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.881+0000 7f63a2400640 1 -- 172.21.15.175:0/401737691 wait complete. 2024-07-29T00:41:31.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.882+0000 7f63a2400640 1 Processor -- start 2024-07-29T00:41:31.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.882+0000 7f63a2400640 1 -- start start 2024-07-29T00:41:31.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f63a2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f63a2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f63a2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c107ff0 0x7f639c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f63a2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111db0 con 0x7f639c107ff0 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f63a2400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111c30 con 0x7f639c103470 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f63a2400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111f30 con 0x7f639c102ac0 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45042/0 (socket says 172.21.15.175:45042) 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 -- 172.21.15.175:0/1482490392 learned_addr learned my addr 172.21.15.175:0/1482490392 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:31.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639be00640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c107ff0 0x7f639c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 -- 172.21.15.175:0/1482490392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 msgr2=0x7f639c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 -- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c107ff0 msgr2=0x7f639c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c107ff0 0x7f639c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.883+0000 7f639b400640 1 -- 172.21.15.175:0/1482490392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c19f480 con 0x7f639c102ac0 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f639aa00640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f639b400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c19a4d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9f0 tx=0x7f638c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:31.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f6393e00640 1 -- 172.21.15.175:0/1482490392 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c00cd80 con 0x7f639c102ac0 2024-07-29T00:41:31.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f6393e00640 1 -- 172.21.15.175:0/1482490392 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638c005500 con 0x7f639c102ac0 2024-07-29T00:41:31.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c19f770 con 0x7f639c102ac0 2024-07-29T00:41:31.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f6393e00640 1 -- 172.21.15.175:0/1482490392 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c010690 con 0x7f639c102ac0 2024-07-29T00:41:31.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.884+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a7050 con 0x7f639c102ac0 2024-07-29T00:41:31.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.887+0000 7f6393e00640 1 -- 172.21.15.175:0/1482490392 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f638c0046c0 con 0x7f639c102ac0 2024-07-29T00:41:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.888+0000 7f63a4fba740 1 Processor -- start 2024-07-29T00:41:31.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.888+0000 7f63a4fba740 1 -- start start 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.888+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f01d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.888+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f0c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.888+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f0ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f63a4fba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560433e66ad0 con 0x560433f01930 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f63a4fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560433dfeaf0 con 0x560433f0cb10 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f63a4fba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560433e62a90 con 0x560433f02250 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f0ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f01d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f0ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:36962/0 (socket says 172.21.15.175:36962) 2024-07-29T00:41:31.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f01d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46864/0 (socket says 172.21.15.175:46864) 2024-07-29T00:41:31.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 -- 172.21.15.175:0/963589487 learned_addr learned my addr 172.21.15.175:0/963589487 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:31.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639b400640 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f0c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f6393e00640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63800a2d70 0x7f63800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.890 INFO:tasks.workunit.client.0.smithi175.stdout: 1 141341 2.48 MB/sec execute 82 sec latency 15.898 ms 2024-07-29T00:41:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f6393e00640 1 -- 172.21.15.175:0/1482490392 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f638c0cf850 con 0x7f639c102ac0 2024-07-29T00:41:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 -- 172.21.15.175:0/963589487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 msgr2=0x560433f0c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f0c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 -- 172.21.15.175:0/963589487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 msgr2=0x560433f01d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f01d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.889+0000 7f639aa00640 1 -- 172.21.15.175:0/963589487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560433bd83c0 con 0x560433f0cb10 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.890+0000 7f639be00640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63800a2d70 0x7f63800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.890+0000 7f639aa00640 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f0ef00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6388005ab0 tx=0x7f63880314c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a48f8b97068abc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.890+0000 7f6390c00640 1 -- 172.21.15.175:0/963589487 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388040070 con 0x560433f0cb10 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.890+0000 7f6390c00640 1 -- 172.21.15.175:0/963589487 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63880044b0 con 0x560433f0cb10 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.890+0000 7f6390c00640 1 -- 172.21.15.175:0/963589487 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388035040 con 0x560433f0cb10 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.891+0000 7f63a4fba740 1 -- 172.21.15.175:0/963589487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 msgr2=0x560433f0ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.891+0000 7f63a4fba740 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f0ef00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6388005ab0 tx=0x7f63880314c0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.891+0000 7f639be00640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63800a2d70 0x7f63800a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6394005fd0 tx=0x7f6394014040 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:31.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 -- 172.21.15.175:0/963589487 shutdown_connections 2024-07-29T00:41:31.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f0ef00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f0c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 --2- 172.21.15.175:0/963589487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f01d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 -- 172.21.15.175:0/963589487 >> 172.21.15.175:0/963589487 conn(0x560433efdec0 msgr2=0x560433efe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:31.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 -- 172.21.15.175:0/963589487 shutdown_connections 2024-07-29T00:41:31.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 -- 172.21.15.175:0/963589487 wait complete. 2024-07-29T00:41:31.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.892+0000 7f63a4fba740 1 -- start start 2024-07-29T00:41:31.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 Processor -- start 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 0x560433f09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f09f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f0a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560433f0f6e0 con 0x560433f0cb10 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560433e21bc0 con 0x560433f01930 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560433f0f860 con 0x560433f02250 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f63a4fba740 10 client.0 _ll_register_callbacks cb 0x560433e0dd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f639b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 0x560433f09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f639b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 0x560433f09a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:36978/0 (socket says 172.21.15.175:36978) 2024-07-29T00:41:31.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.893+0000 7f639b400640 1 -- 172.21.15.175:0/2703743514 learned_addr learned my addr 172.21.15.175:0/2703743514 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639aa00640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f09f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639be00640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f0a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639b400640 1 -- 172.21.15.175:0/2703743514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 msgr2=0x560433f09f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639b400640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f09f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639b400640 1 -- 172.21.15.175:0/2703743514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 msgr2=0x560433f0a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639b400640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f0a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639b400640 1 -- 172.21.15.175:0/2703743514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560433f18c00 con 0x560433f01930 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f639b400640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 0x560433f09a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9c0 tx=0x7f638c09b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b93202f82260f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f6385600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.894+0000 7f6385600640 1 -- 172.21.15.175:0/2703743514 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0cd3b0 con 0x560433f01930 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f6385600640 1 -- 172.21.15.175:0/2703743514 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f638c0a46d0 con 0x560433f01930 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63880044b0 con 0x560433f01930 2024-07-29T00:41:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f6385600640 1 -- 172.21.15.175:0/2703743514 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f638c0a4960 con 0x560433f01930 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 10 client.26474 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560433f1b5d0 con 0x560433f01930 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f6385600640 1 -- 172.21.15.175:0/2703743514 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63880044b0 con 0x560433f01930 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f6385600640 1 -- 172.21.15.175:0/2703743514 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f638c09abb0 con 0x560433f01930 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 10 client.26474 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 10 client.26474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 20 client.26474 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f1ca90 0x560433f1eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 4 client.26474 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.895+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560433e74040 con 0x560433f1ca90 2024-07-29T00:41:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.896+0000 7f639aa00640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f1ca90 0x560433f1eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.896+0000 7f639aa00640 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f1ca90 0x560433f1eea0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e56ce34109e9494 server_cookie=5870d8334f43830 in_seq=0 out_seq=0 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.896+0000 7f6385600640 10 client.26474 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.897+0000 7f6385600640 1 -- 172.21.15.175:0/2703743514 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560433e74040 con 0x560433f1ca90 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.897+0000 7f6385600640 10 client.26474 handle_command_reply: tid=0 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.897+0000 7f63a4fba740 1 client.26474 shutdown 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.897+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 msgr2=0x560433f09a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.897+0000 7f63a4fba740 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 0x560433f09a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9c0 tx=0x7f638c09b650 comp rx=0 tx=0).stop 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 shutdown_connections 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f1ca90 0x560433f1eea0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f0a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f09f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 --2- 172.21.15.175:0/2703743514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f01930 0x560433f09a10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 >> 172.21.15.175:0/2703743514 conn(0x560433efdec0 msgr2=0x560433f03ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f6385600640 0 client.26474 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:31.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 shutdown_connections 2024-07-29T00:41:31.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 1 -- 172.21.15.175:0/2703743514 wait complete. 2024-07-29T00:41:31.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.898+0000 7f63a4fba740 20 client.26474 trim_cache size 0 max 16384 2024-07-29T00:41:31.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.918+0000 7f63a4fba740 1 Processor -- start 2024-07-29T00:41:31.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 -- start start 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f01930 0x560433f04390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 0x560433f0a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1af780 con 0x560433f0cb10 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b1320 con 0x560433f02250 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f63a4fba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1afe50 con 0x560433f01930 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f639aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 0x560433f0a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f639aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 0x560433f0a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:36994/0 (socket says 172.21.15.175:36994) 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f639aa00640 1 -- 172.21.15.175:0/3258786437 learned_addr learned my addr 172.21.15.175:0/3258786437 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:31.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f639b400640 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f01930 0x560433f04390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.919+0000 7f639be00640 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f639aa00640 1 -- 172.21.15.175:0/3258786437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f01930 msgr2=0x560433f04390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f639aa00640 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f01930 0x560433f04390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f639aa00640 1 -- 172.21.15.175:0/3258786437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 msgr2=0x560433f24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f639aa00640 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f24740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f639aa00640 1 -- 172.21.15.175:0/3258786437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560433f0c560 con 0x560433f02250 2024-07-29T00:41:31.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f639aa00640 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 0x560433f0a2b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6388004e30 tx=0x7f6388004e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65e3fad11d41242c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:31.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f6387e00640 1 -- 172.21.15.175:0/3258786437 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388040040 con 0x560433f02250 2024-07-29T00:41:31.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f6387e00640 1 -- 172.21.15.175:0/3258786437 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63880049a0 con 0x560433f02250 2024-07-29T00:41:31.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.920+0000 7f6387e00640 1 -- 172.21.15.175:0/3258786437 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388035040 con 0x560433f02250 2024-07-29T00:41:31.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 -- 172.21.15.175:0/3258786437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 msgr2=0x560433f0a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 0x560433f0a2b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6388004e30 tx=0x7f6388004e60 comp rx=0 tx=0).stop 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 -- 172.21.15.175:0/3258786437 shutdown_connections 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f0cb10 0x560433f24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f02250 0x560433f0a2b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3258786437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f01930 0x560433f04390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 -- 172.21.15.175:0/3258786437 >> 172.21.15.175:0/3258786437 conn(0x560433efdec0 msgr2=0x560433f082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.921+0000 7f63a4fba740 1 -- 172.21.15.175:0/3258786437 shutdown_connections 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.922+0000 7f63a4fba740 1 -- 172.21.15.175:0/3258786437 wait complete. 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.922+0000 7f63a4fba740 1 -- start start 2024-07-29T00:41:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.922+0000 7f63a4fba740 1 Processor -- start 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f10530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b0c50 con 0x560433f01930 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b0520 con 0x560433f0cb10 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1ad760 con 0x560433f02250 2024-07-29T00:41:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f63a4fba740 10 client.0 _ll_register_callbacks cb 0x7f638c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f639aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f10530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f639be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f639aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f10530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45082/0 (socket says 172.21.15.175:45082) 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f639be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f10a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37004/0 (socket says 172.21.15.175:37004) 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f639be00640 1 -- 172.21.15.175:0/3065261014 learned_addr learned my addr 172.21.15.175:0/3065261014 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.923+0000 7f639b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f639aa00640 1 -- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 msgr2=0x560433f10a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f639aa00640 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f10a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f639aa00640 1 -- 172.21.15.175:0/3065261014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 msgr2=0x560433f15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f639aa00640 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f639aa00640 1 -- 172.21.15.175:0/3065261014 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560433f11250 con 0x560433f02250 2024-07-29T00:41:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f639aa00640 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f10530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x560433efeef0 tx=0x7f6388009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f863ecbef18fe9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.924+0000 7f6385600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f6385600640 1 -- 172.21.15.175:0/3065261014 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388035070 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f6385600640 1 -- 172.21.15.175:0/3065261014 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6388031c00 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63880049a0 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f6385600640 1 -- 172.21.15.175:0/3065261014 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6388040070 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f63a4fba740 10 client.26443 fetch_fsmap learned FSMap version 36 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560433f87730 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.925+0000 7f6385600640 1 -- 172.21.15.175:0/3065261014 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63880049a0 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f6385600640 1 -- 172.21.15.175:0/3065261014 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f63880037d0 con 0x560433f02250 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f63a4fba740 10 client.26443 fetch_fsmap finished waiting for FSMap version 36 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f63a4fba740 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f63a4fba740 20 client.26443 populate_metadata read hostname 'smithi175' 2024-07-29T00:41:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f87960 0x560433f89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:41:31.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f63a4fba740 4 client.26443 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:41:31.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560433e6e550 con 0x560433f87960 2024-07-29T00:41:31.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.926+0000 7f639b400640 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f87960 0x560433f89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f639b400640 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f87960 0x560433f89d50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92bdee979c3bdb8 server_cookie=69bc872309a7add2 in_seq=0 out_seq=0 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f6385600640 10 client.26443 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f6385600640 1 -- 172.21.15.175:0/3065261014 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560433e6e550 con 0x560433f87960 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f6385600640 10 client.26443 handle_command_reply: tid=0 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 client.26443 shutdown 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 msgr2=0x560433f10530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f10530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x560433efeef0 tx=0x7f6388009880 comp rx=0 tx=0).stop 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 shutdown_connections 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x560433f87960 0x560433f89d50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560433f0cb10 0x560433f10a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560433f02250 0x560433f10530 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f6385600640 0 client.26443 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 --2- 172.21.15.175:0/3065261014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560433f01930 0x560433f15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.927+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 >> 172.21.15.175:0/3065261014 conn(0x560433efdec0 msgr2=0x560433efc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.928+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 shutdown_connections 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.928+0000 7f63a4fba740 1 -- 172.21.15.175:0/3065261014 wait complete. 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.928+0000 7f63a4fba740 20 client.26443 trim_cache size 0 max 16384 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:41:31.930 INFO:teuthology.orchestra.run.smithi175.stdout: "9d8fb3a9-7c5b-4205-b178-1acaf0398caf": { 2024-07-29T00:41:31.931 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:41:31.931 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "9d8fb3a9-7c5b-4205-b178-1acaf0398caf", 2024-07-29T00:41:31.931 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:41:31.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:41:31.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:41:31.931 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63800a2d70 msgr2=0x7f63800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63800a2d70 0x7f63800a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6394005fd0 tx=0x7f6394014040 comp rx=0 tx=0).stop 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 msgr2=0x7f639c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c19a4d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9f0 tx=0x7f638c00edf0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 shutdown_connections 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63800a2d70 0x7f63800a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c107ff0 0x7f639c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f639c103470 0x7f639c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 --2- 172.21.15.175:0/1482490392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c102ac0 0x7f639c19a4d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 >> 172.21.15.175:0/1482490392 conn(0x7f639c0fc4c0 msgr2=0x7f639c100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:41:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 shutdown_connections 2024-07-29T00:41:31.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:41:31.929+0000 7f63a2400640 1 -- 172.21.15.175:0/1482490392 wait complete. 2024-07-29T00:41:32.890 INFO:tasks.workunit.client.0.smithi175.stdout: 1 142269 2.53 MB/sec execute 83 sec latency 21.155 ms 2024-07-29T00:41:33.076 INFO:tasks.cephfs.filesystem:scrub status for tag:9d8fb3a9-7c5b-4205-b178-1acaf0398caf - {'path': '/', 'tag': '9d8fb3a9-7c5b-4205-b178-1acaf0398caf', 'options': 'recursive,force'} 2024-07-29T00:41:33.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:33 smithi181 ceph-mon[36706]: pgmap v154: 97 pgs: 97 active+clean; 93 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 189 op/s 2024-07-29T00:41:33.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:33 smithi175 ceph-mon[32138]: pgmap v154: 97 pgs: 97 active+clean; 93 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 189 op/s 2024-07-29T00:41:33.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:33 smithi204 ceph-mon[37368]: pgmap v154: 97 pgs: 97 active+clean; 93 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 189 op/s 2024-07-29T00:41:33.890 INFO:tasks.workunit.client.0.smithi175.stdout: 1 142702 2.51 MB/sec execute 84 sec latency 39.356 ms 2024-07-29T00:41:34.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:34 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:34.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:34 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:34.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:34 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:34.891 INFO:tasks.workunit.client.0.smithi175.stdout: 1 143722 2.53 MB/sec execute 85 sec latency 23.721 ms 2024-07-29T00:41:35.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:35 smithi181 ceph-mon[36706]: pgmap v155: 97 pgs: 97 active+clean; 93 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 784 KiB/s wr, 131 op/s 2024-07-29T00:41:35.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:35 smithi175 ceph-mon[32138]: pgmap v155: 97 pgs: 97 active+clean; 93 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 784 KiB/s wr, 131 op/s 2024-07-29T00:41:35.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:35 smithi204 ceph-mon[37368]: pgmap v155: 97 pgs: 97 active+clean; 93 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 784 KiB/s wr, 131 op/s 2024-07-29T00:41:35.891 INFO:tasks.workunit.client.0.smithi175.stdout: 1 145075 2.60 MB/sec execute 86 sec latency 21.838 ms 2024-07-29T00:41:36.891 INFO:tasks.workunit.client.0.smithi175.stdout: 1 145726 2.63 MB/sec execute 87 sec latency 114.958 ms 2024-07-29T00:41:37.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:37 smithi181 ceph-mon[36706]: pgmap v156: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-07-29T00:41:37.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:37 smithi175 ceph-mon[32138]: pgmap v156: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-07-29T00:41:37.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:37 smithi204 ceph-mon[37368]: pgmap v156: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-07-29T00:41:37.891 INFO:tasks.workunit.client.0.smithi175.stdout: 1 145982 2.60 MB/sec execute 88 sec latency 374.534 ms 2024-07-29T00:41:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:38 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:41:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:38 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:41:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:38 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:41:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:38 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:41:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:38 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:41:38.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:38 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:38 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:41:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:38 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:41:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:38 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:41:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:38 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:41:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:38 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:41:38.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:38 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:38 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:41:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:38 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:41:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:38 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:41:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:38 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:41:38.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:38 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:41:38.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:38 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:41:38.891 INFO:tasks.workunit.client.0.smithi175.stdout: 1 146965 2.63 MB/sec execute 89 sec latency 18.526 ms 2024-07-29T00:41:39.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:39 smithi181 ceph-mon[36706]: pgmap v157: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 KiB/s wr, 126 op/s 2024-07-29T00:41:39.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:39 smithi175 ceph-mon[32138]: pgmap v157: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 KiB/s wr, 126 op/s 2024-07-29T00:41:39.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:39 smithi204 ceph-mon[37368]: pgmap v157: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 KiB/s wr, 126 op/s 2024-07-29T00:41:39.891 INFO:tasks.workunit.client.0.smithi175.stdout: 1 148182 2.67 MB/sec execute 90 sec latency 23.245 ms 2024-07-29T00:41:40.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 149246 2.73 MB/sec execute 91 sec latency 23.314 ms 2024-07-29T00:41:41.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:41 smithi181 ceph-mon[36706]: pgmap v158: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 KiB/s wr, 126 op/s 2024-07-29T00:41:41.764 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:41 smithi175 ceph-mon[32138]: pgmap v158: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 KiB/s wr, 126 op/s 2024-07-29T00:41:41.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:41 smithi204 ceph-mon[37368]: pgmap v158: 97 pgs: 97 active+clean; 97 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 KiB/s wr, 126 op/s 2024-07-29T00:41:41.892 INFO:tasks.workunit.client.0.smithi175.stdout: 1 149528 2.70 MB/sec execute 92 sec latency 106.524 ms 2024-07-29T00:41:42.892 INFO:tasks.workunit.client.0.smithi175.stdout: 1 149793 2.67 MB/sec execute 93 sec latency 758.810 ms 2024-07-29T00:41:43.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:43 smithi175 ceph-mon[32138]: pgmap v159: 97 pgs: 97 active+clean; 99 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-07-29T00:41:43.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:43 smithi204 ceph-mon[37368]: pgmap v159: 97 pgs: 97 active+clean; 99 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-07-29T00:41:43.892 INFO:tasks.workunit.client.0.smithi175.stdout: 1 150861 2.70 MB/sec execute 94 sec latency 23.658 ms 2024-07-29T00:41:43.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:43 smithi181 ceph-mon[36706]: pgmap v159: 97 pgs: 97 active+clean; 99 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-07-29T00:41:44.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 152208 2.75 MB/sec execute 95 sec latency 20.113 ms 2024-07-29T00:41:45.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:45 smithi175 ceph-mon[32138]: pgmap v160: 97 pgs: 97 active+clean; 99 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 132 op/s 2024-07-29T00:41:45.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:45 smithi204 ceph-mon[37368]: pgmap v160: 97 pgs: 97 active+clean; 99 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 132 op/s 2024-07-29T00:41:45.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 152914 2.79 MB/sec execute 96 sec latency 21.970 ms 2024-07-29T00:41:45.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:45 smithi181 ceph-mon[36706]: pgmap v160: 97 pgs: 97 active+clean; 99 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 132 op/s 2024-07-29T00:41:46.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 153370 2.76 MB/sec execute 97 sec latency 27.959 ms 2024-07-29T00:41:47.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:47 smithi175 ceph-mon[32138]: pgmap v161: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-07-29T00:41:47.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:47 smithi204 ceph-mon[37368]: pgmap v161: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-07-29T00:41:47.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 154046 2.78 MB/sec execute 98 sec latency 329.329 ms 2024-07-29T00:41:47.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:47 smithi181 ceph-mon[36706]: pgmap v161: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-07-29T00:41:48.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:48 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:41:48.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 155372 2.81 MB/sec execute 99 sec latency 30.231 ms 2024-07-29T00:41:48.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:48 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:41:49.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:48 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:41:49.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:49 smithi204 ceph-mon[37368]: pgmap v162: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-07-29T00:41:49.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:49 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:41:49.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:49 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:41:49.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:49 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:41:49.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 156386 2.86 MB/sec execute 100 sec latency 25.464 ms 2024-07-29T00:41:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:49 smithi181 ceph-mon[36706]: pgmap v162: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-07-29T00:41:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:49 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:41:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:49 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:41:49.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:49 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:41:50.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:49 smithi175 ceph-mon[32138]: pgmap v162: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-07-29T00:41:50.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:49 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:41:50.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:49 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:41:50.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:49 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:41:50.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 156754 2.84 MB/sec execute 101 sec latency 24.719 ms 2024-07-29T00:41:51.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:51 smithi204 ceph-mon[37368]: pgmap v163: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-07-29T00:41:51.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 157742 2.86 MB/sec execute 102 sec latency 19.655 ms 2024-07-29T00:41:51.984 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:51 smithi175 ceph-mon[32138]: pgmap v163: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-07-29T00:41:51.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:51 smithi181 ceph-mon[36706]: pgmap v163: 97 pgs: 97 active+clean; 101 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-07-29T00:41:52.893 INFO:tasks.workunit.client.0.smithi175.stdout: 1 157797 2.83 MB/sec execute 103 sec latency 867.920 ms 2024-07-29T00:41:53.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:53 smithi204 ceph-mon[37368]: pgmap v164: 97 pgs: 97 active+clean; 104 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-07-29T00:41:53.894 INFO:tasks.workunit.client.0.smithi175.stdout: 1 159102 2.88 MB/sec execute 104 sec latency 973.392 ms 2024-07-29T00:41:53.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:53 smithi181 ceph-mon[36706]: pgmap v164: 97 pgs: 97 active+clean; 104 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-07-29T00:41:54.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:53 smithi175 ceph-mon[32138]: pgmap v164: 97 pgs: 97 active+clean; 104 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-07-29T00:41:54.894 INFO:tasks.workunit.client.0.smithi175.stdout: 1 160004 2.91 MB/sec execute 105 sec latency 20.079 ms 2024-07-29T00:41:55.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:55 smithi204 ceph-mon[37368]: pgmap v165: 97 pgs: 97 active+clean; 104 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-07-29T00:41:55.894 INFO:tasks.workunit.client.0.smithi175.stdout: 1 160611 2.89 MB/sec execute 106 sec latency 21.419 ms 2024-07-29T00:41:55.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:55 smithi181 ceph-mon[36706]: pgmap v165: 97 pgs: 97 active+clean; 104 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-07-29T00:41:56.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:55 smithi175 ceph-mon[32138]: pgmap v165: 97 pgs: 97 active+clean; 104 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-07-29T00:41:56.894 INFO:tasks.workunit.client.0.smithi175.stdout: 1 161975 2.92 MB/sec execute 107 sec latency 22.856 ms 2024-07-29T00:41:57.890 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:57 smithi175 ceph-mon[32138]: pgmap v166: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-07-29T00:41:57.894 INFO:tasks.workunit.client.0.smithi175.stdout: 1 162094 2.90 MB/sec execute 108 sec latency 558.657 ms 2024-07-29T00:41:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:57 smithi181 ceph-mon[36706]: pgmap v166: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-07-29T00:41:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:57 smithi204 ceph-mon[37368]: pgmap v166: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-07-29T00:41:58.895 INFO:tasks.workunit.client.0.smithi175.stdout: 1 163403 2.98 MB/sec execute 109 sec latency 577.348 ms 2024-07-29T00:41:59.895 INFO:tasks.workunit.client.0.smithi175.stdout: 1 163710 2.95 MB/sec execute 110 sec latency 20.371 ms 2024-07-29T00:41:59.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:41:59 smithi181 ceph-mon[36706]: pgmap v167: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:00.037 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:41:59 smithi175 ceph-mon[32138]: pgmap v167: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:41:59 smithi204 ceph-mon[37368]: pgmap v167: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:00.895 INFO:tasks.workunit.client.0.smithi175.stdout: 1 164218 2.94 MB/sec execute 111 sec latency 54.205 ms 2024-07-29T00:42:01.895 INFO:tasks.workunit.client.0.smithi175.stdout: 1 165549 2.96 MB/sec execute 112 sec latency 25.802 ms 2024-07-29T00:42:01.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:01 smithi181 ceph-mon[36706]: pgmap v168: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:01 smithi175 ceph-mon[32138]: pgmap v168: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:01 smithi204 ceph-mon[37368]: pgmap v168: 97 pgs: 97 active+clean; 106 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:02.895 INFO:tasks.workunit.client.0.smithi175.stdout: 1 165791 2.95 MB/sec execute 113 sec latency 457.762 ms 2024-07-29T00:42:03.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:42:03.485 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:03.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:03 smithi175 ceph-mon[32138]: pgmap v169: 97 pgs: 97 active+clean; 108 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-07-29T00:42:03.896 INFO:tasks.workunit.client.0.smithi175.stdout: 1 166557 2.98 MB/sec execute 114 sec latency 779.852 ms 2024-07-29T00:42:03.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:03 smithi181 ceph-mon[36706]: pgmap v169: 97 pgs: 97 active+clean; 108 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-07-29T00:42:04.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:03 smithi204 ceph-mon[37368]: pgmap v169: 97 pgs: 97 active+clean; 108 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-07-29T00:42:04.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 -- 172.21.15.175:0/2185186579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20104440 msgr2=0x7f3c20110970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 --2- 172.21.15.175:0/2185186579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20104440 0x7f3c20110970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009cd0 tx=0x7f3c1802a8f0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 -- 172.21.15.175:0/2185186579 shutdown_connections 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 --2- 172.21.15.175:0/2185186579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20104440 0x7f3c20110970 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 --2- 172.21.15.175:0/2185186579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c20103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 --2- 172.21.15.175:0/2185186579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c20109aa0 0x7f3c20109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 -- 172.21.15.175:0/2185186579 >> 172.21.15.175:0/2185186579 conn(0x7f3c200ff740 msgr2=0x7f3c20101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 -- 172.21.15.175:0/2185186579 shutdown_connections 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.171+0000 7f3c26200640 1 -- 172.21.15.175:0/2185186579 wait complete. 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.172+0000 7f3c26200640 1 Processor -- start 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.172+0000 7f3c26200640 1 -- start start 2024-07-29T00:42:04.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c26200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c2019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c26200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c20104440 0x7f3c2019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c26200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20109aa0 0x7f3c2019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c26200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20113ef0 con 0x7f3c20109aa0 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c26200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20113d70 con 0x7f3c20103aa0 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c26200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20114070 con 0x7f3c20104440 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c2019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c2019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59802/0 (socket says 172.21.15.175:59802) 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 -- 172.21.15.175:0/1749643973 learned_addr learned my addr 172.21.15.175:0/1749643973 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1fe00640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20109aa0 0x7f3c2019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c20104440 0x7f3c2019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 -- 172.21.15.175:0/1749643973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c20104440 msgr2=0x7f3c2019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c20104440 0x7f3c2019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 -- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20109aa0 msgr2=0x7f3c2019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20109aa0 0x7f3c2019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.173+0000 7f3c1f400640 1 -- 172.21.15.175:0/1749643973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2019f2c0 con 0x7f3c20103aa0 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.174+0000 7f3c1f400640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c2019a310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e9f0 tx=0x7f3c1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.174+0000 7f3c17e00640 1 -- 172.21.15.175:0/1749643973 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000cd80 con 0x7f3c20103aa0 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.174+0000 7f3c17e00640 1 -- 172.21.15.175:0/1749643973 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c10010910 con 0x7f3c20103aa0 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.174+0000 7f3c17e00640 1 -- 172.21.15.175:0/1749643973 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c100188f0 con 0x7f3c20103aa0 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.175+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2019f5b0 con 0x7f3c20103aa0 2024-07-29T00:42:04.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.175+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c201a6e90 con 0x7f3c20103aa0 2024-07-29T00:42:04.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 Processor -- start 2024-07-29T00:42:04.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c17e00640 1 -- 172.21.15.175:0/1749643973 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3c10010ac0 con 0x7f3c20103aa0 2024-07-29T00:42:04.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 -- start start 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x560699416d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699417250 0x5606994215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x560699423f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56069937bad0 con 0x560699417250 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560699313af0 con 0x560699416930 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.179+0000 7f3c28bed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560699377a90 con 0x560699421b10 2024-07-29T00:42:04.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699417250 0x5606994215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699417250 0x5606994215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41696/0 (socket says 172.21.15.175:41696) 2024-07-29T00:42:04.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1f400640 1 -- 172.21.15.175:0/2848729767 learned_addr learned my addr 172.21.15.175:0/2848729767 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:04.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x560699423f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1fe00640 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x560699416d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1ea00640 1 -- 172.21.15.175:0/2848729767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 msgr2=0x560699416d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x560699416d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1ea00640 1 -- 172.21.15.175:0/2848729767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699417250 msgr2=0x5606994215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699417250 0x5606994215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.180+0000 7f3c1ea00640 1 -- 172.21.15.175:0/2848729767 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606990ed3c0 con 0x560699421b10 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x560699423f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009bf0 tx=0x7f3c0c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46157374bc494ba9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c0be00640 1 -- 172.21.15.175:0/2848729767 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c004050 con 0x560699421b10 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c0be00640 1 -- 172.21.15.175:0/2848729767 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c005500 con 0x560699421b10 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c0be00640 1 -- 172.21.15.175:0/2848729767 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c004e30 con 0x560699421b10 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c17e00640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c040a2cf0 0x7f3c040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c17e00640 1 -- 172.21.15.175:0/1749643973 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3c100cf7f0 con 0x7f3c20103aa0 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.181+0000 7f3c1fe00640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c040a2cf0 0x7f3c040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.182+0000 7f3c28bed740 1 -- 172.21.15.175:0/2848729767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 msgr2=0x560699423f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.182+0000 7f3c28bed740 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x560699423f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009bf0 tx=0x7f3c0c011810 comp rx=0 tx=0).stop 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.182+0000 7f3c1fe00640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c040a2cf0 0x7f3c040a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c18004af0 tx=0x7f3c18005d80 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 -- 172.21.15.175:0/2848729767 shutdown_connections 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x560699423f00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699417250 0x5606994215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 --2- 172.21.15.175:0/2848729767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x560699416d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 -- 172.21.15.175:0/2848729767 >> 172.21.15.175:0/2848729767 conn(0x560699412ec0 msgr2=0x5606994132a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 -- 172.21.15.175:0/2848729767 shutdown_connections 2024-07-29T00:42:04.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 -- 172.21.15.175:0/2848729767 wait complete. 2024-07-29T00:42:04.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.183+0000 7f3c28bed740 1 -- start start 2024-07-29T00:42:04.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 Processor -- start 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699416930 0x56069941e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x56069941ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x56069941f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5606994246e0 con 0x560699416930 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560699336bc0 con 0x560699417250 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560699424860 con 0x560699421b10 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c28bed740 10 client.0 _ll_register_callbacks cb 0x560699322d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699416930 0x56069941e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699416930 0x56069941e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41702/0 (socket says 172.21.15.175:41702) 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.184+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x56069941ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1f400640 1 -- 172.21.15.175:0/276936047 learned_addr learned my addr 172.21.15.175:0/276936047 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1fe00640 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x56069941f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1ea00640 1 -- 172.21.15.175:0/276936047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 msgr2=0x56069941f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x56069941f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1ea00640 1 -- 172.21.15.175:0/276936047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699416930 msgr2=0x56069941e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699416930 0x56069941e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1ea00640 1 -- 172.21.15.175:0/276936047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069942db50 con 0x560699417250 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x56069941ee70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c004620 tx=0x7f3c0c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf348f9baab1286a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.185+0000 7f3c08c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c08c00640 1 -- 172.21.15.175:0/276936047 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c004db0 con 0x560699417250 2024-07-29T00:42:04.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0c005500 con 0x560699417250 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c08c00640 1 -- 172.21.15.175:0/276936047 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3c0c004f50 con 0x560699417250 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c08c00640 1 -- 172.21.15.175:0/276936047 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c033050 con 0x560699417250 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c28bed740 10 client.26492 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560699430490 con 0x560699417250 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c08c00640 1 -- 172.21.15.175:0/276936047 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c005500 con 0x560699417250 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.186+0000 7f3c08c00640 1 -- 172.21.15.175:0/276936047 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f3c0c019430 con 0x560699417250 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c28bed740 10 client.26492 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c28bed740 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c28bed740 20 client.26492 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c28bed740 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5606994317e0 0x560699433bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c28bed740 4 client.26492 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560699389040 con 0x5606994317e0 2024-07-29T00:42:04.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c1f400640 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5606994317e0 0x560699433bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c1f400640 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5606994317e0 0x560699433bd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6ead994ed71f1c server_cookie=874f6073bb4f5239 in_seq=0 out_seq=0 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.187+0000 7f3c08c00640 10 client.26492 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.188+0000 7f3c08c00640 1 -- 172.21.15.175:0/276936047 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560699389040 con 0x5606994317e0 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.188+0000 7f3c08c00640 10 client.26492 handle_command_reply: tid=0 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.188+0000 7f3c28bed740 1 client.26492 shutdown 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.188+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 msgr2=0x56069941ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.188+0000 7f3c28bed740 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x56069941ee70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c004620 tx=0x7f3c0c009fd0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 shutdown_connections 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5606994317e0 0x560699433bd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699421b10 0x56069941f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x56069941ee70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 --2- 172.21.15.175:0/276936047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699416930 0x56069941e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c08c00640 0 client.26492 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 >> 172.21.15.175:0/276936047 conn(0x560699412ec0 msgr2=0x560699418bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 shutdown_connections 2024-07-29T00:42:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 1 -- 172.21.15.175:0/276936047 wait complete. 2024-07-29T00:42:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.189+0000 7f3c28bed740 20 client.26492 trim_cache size 0 max 16384 2024-07-29T00:42:04.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 Processor -- start 2024-07-29T00:42:04.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 -- start start 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699416930 0x56069941f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x5606994393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x560699439910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201af9b0 con 0x560699421b10 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201b1610 con 0x560699417250 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c28bed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201b0080 con 0x560699416930 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x5606994393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699416930 0x56069941f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x5606994393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59834/0 (socket says 172.21.15.175:59834) 2024-07-29T00:42:04.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c1ea00640 1 -- 172.21.15.175:0/3097486091 learned_addr learned my addr 172.21.15.175:0/3097486091 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:04.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.211+0000 7f3c1fe00640 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x560699439910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c1ea00640 1 -- 172.21.15.175:0/3097486091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699416930 msgr2=0x56069941f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699416930 0x56069941f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c1ea00640 1 -- 172.21.15.175:0/3097486091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 msgr2=0x560699439910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x560699439910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c1ea00640 1 -- 172.21.15.175:0/3097486091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560699417080 con 0x560699417250 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x5606994393d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c019800 tx=0x7f3c0c009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6afc31aefa549f92 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c0b400640 1 -- 172.21.15.175:0/3097486091 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c019430 con 0x560699417250 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c0b400640 1 -- 172.21.15.175:0/3097486091 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c0195d0 con 0x560699417250 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.212+0000 7f3c0b400640 1 -- 172.21.15.175:0/3097486091 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c023430 con 0x560699417250 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.213+0000 7f3c28bed740 1 -- 172.21.15.175:0/3097486091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 msgr2=0x5606994393d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.213+0000 7f3c28bed740 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x5606994393d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c019800 tx=0x7f3c0c009a70 comp rx=0 tx=0).stop 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 -- 172.21.15.175:0/3097486091 shutdown_connections 2024-07-29T00:42:04.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x560699439910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699417250 0x5606994393d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 --2- 172.21.15.175:0/3097486091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699416930 0x56069941f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 -- 172.21.15.175:0/3097486091 >> 172.21.15.175:0/3097486091 conn(0x560699412ec0 msgr2=0x56069941da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:04.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 -- 172.21.15.175:0/3097486091 shutdown_connections 2024-07-29T00:42:04.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 -- 172.21.15.175:0/3097486091 wait complete. 2024-07-29T00:42:04.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.214+0000 7f3c28bed740 1 -- start start 2024-07-29T00:42:04.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 Processor -- start 2024-07-29T00:42:04.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x56069942a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 0x56069941e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x56069941eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201b0e80 con 0x560699421b10 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201b0750 con 0x560699416930 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201ad990 con 0x560699417250 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c28bed740 10 client.0 _ll_register_callbacks cb 0x7f3c0c0043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 0x56069941e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 0x56069941e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33364/0 (socket says 172.21.15.175:33364) 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x56069942a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c1fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x56069941eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.215+0000 7f3c1ea00640 1 -- 172.21.15.175:0/967675854 learned_addr learned my addr 172.21.15.175:0/967675854 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c1ea00640 1 -- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 msgr2=0x56069942a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x56069942a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c1ea00640 1 -- 172.21.15.175:0/967675854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 msgr2=0x56069941eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:04.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x56069941eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c1ea00640 1 -- 172.21.15.175:0/967675854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069941f2d0 con 0x560699417250 2024-07-29T00:42:04.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c1ea00640 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 0x56069941e560 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c011850 tx=0x7f3c0c0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6160ec88bf0fe516 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:04.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.216+0000 7f3c08c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c08c00640 1 -- 172.21.15.175:0/967675854 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c019430 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c08c00640 1 -- 172.21.15.175:0/967675854 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f3c0c0037f0 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0c0195d0 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c08c00640 1 -- 172.21.15.175:0/967675854 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c003e60 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c28bed740 10 client.26461 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56069949c400 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c08c00640 1 -- 172.21.15.175:0/967675854 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c0195d0 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.217+0000 7f3c08c00640 1 -- 172.21.15.175:0/967675854 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f3c0c036070 con 0x560699417250 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c28bed740 10 client.26461 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:04.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c28bed740 10 client.26461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c28bed740 20 client.26461 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c28bed740 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56069949cad0 0x56069949eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c28bed740 4 client.26461 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560699383550 con 0x56069949cad0 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.218+0000 7f3c1f400640 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56069949cad0 0x56069949eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c08c00640 10 client.26461 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c1f400640 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56069949cad0 0x56069949eef0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bda59c5a9105e96 server_cookie=2676b6c9dddd6fb in_seq=0 out_seq=0 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c08c00640 1 -- 172.21.15.175:0/967675854 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560699383550 con 0x56069949cad0 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c08c00640 10 client.26461 handle_command_reply: tid=0 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 client.26461 shutdown 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 msgr2=0x56069941e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 0x56069941e560 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c011850 tx=0x7f3c0c0093b0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 shutdown_connections 2024-07-29T00:42:04.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56069949cad0 0x56069949eef0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560699421b10 0x56069941eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560699417250 0x56069941e560 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 --2- 172.21.15.175:0/967675854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560699416930 0x56069942a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 >> 172.21.15.175:0/967675854 conn(0x560699412ec0 msgr2=0x560699420190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.219+0000 7f3c08c00640 0 client.26461 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.220+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 shutdown_connections 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.220+0000 7f3c28bed740 1 -- 172.21.15.175:0/967675854 wait complete. 2024-07-29T00:42:04.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.220+0000 7f3c28bed740 20 client.26461 trim_cache size 0 max 16384 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c040a2cf0 msgr2=0x7f3c040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c040a2cf0 0x7f3c040a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c18004af0 tx=0x7f3c18005d80 comp rx=0 tx=0).stop 2024-07-29T00:42:04.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 msgr2=0x7f3c2019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c2019a310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e9f0 tx=0x7f3c1000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 shutdown_connections 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3c040a2cf0 0x7f3c040a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c20109aa0 0x7f3c2019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c20104440 0x7f3c2019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 --2- 172.21.15.175:0/1749643973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c20103aa0 0x7f3c2019a310 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 >> 172.21.15.175:0/1749643973 conn(0x7f3c200ff740 msgr2=0x7f3c20100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 shutdown_connections 2024-07-29T00:42:04.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:04.221+0000 7f3c26200640 1 -- 172.21.15.175:0/1749643973 wait complete. 2024-07-29T00:42:04.896 INFO:tasks.workunit.client.0.smithi175.stdout: 1 167185 2.99 MB/sec execute 115 sec latency 20.691 ms 2024-07-29T00:42:05.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:42:05.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:42:05.700 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:05.896 INFO:tasks.workunit.client.0.smithi175.stdout: 1 168309 3.01 MB/sec execute 116 sec latency 17.832 ms 2024-07-29T00:42:05.933 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:05 smithi175 ceph-mon[32138]: pgmap v170: 97 pgs: 97 active+clean; 108 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-07-29T00:42:05.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:05 smithi181 ceph-mon[36706]: pgmap v170: 97 pgs: 97 active+clean; 108 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-07-29T00:42:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:05 smithi204 ceph-mon[37368]: pgmap v170: 97 pgs: 97 active+clean; 108 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-07-29T00:42:06.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 -- 172.21.15.175:0/1432221822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0074c30 msgr2=0x7f00c010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:06.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 --2- 172.21.15.175:0/1432221822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0074c30 0x7f00c010d9b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02a480 comp rx=0 tx=0).stop 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 -- 172.21.15.175:0/1432221822 shutdown_connections 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 --2- 172.21.15.175:0/1432221822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c010def0 0x7f00c0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 --2- 172.21.15.175:0/1432221822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0074c30 0x7f00c010d9b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 --2- 172.21.15.175:0/1432221822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 0x7f00c00746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 -- 172.21.15.175:0/1432221822 >> 172.21.15.175:0/1432221822 conn(0x7f00c00fd3d0 msgr2=0x7f00c00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 -- 172.21.15.175:0/1432221822 shutdown_connections 2024-07-29T00:42:06.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.414+0000 7f00bfe00640 1 -- 172.21.15.175:0/1432221822 wait complete. 2024-07-29T00:42:06.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.415+0000 7f00bfe00640 1 Processor -- start 2024-07-29T00:42:06.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.415+0000 7f00bfe00640 1 -- start start 2024-07-29T00:42:06.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 0x7f00c019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:06.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 0x7f00c019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:06.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c010def0 0x7f00c019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:06.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0113cf0 con 0x7f00c0074310 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bfe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0113b70 con 0x7f00c010def0 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0113e70 con 0x7f00c0074c30 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 0x7f00c019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 0x7f00c019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33392/0 (socket says 172.21.15.175:33392) 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 -- 172.21.15.175:0/3101392319 learned_addr learned my addr 172.21.15.175:0/3101392319 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bf400640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c010def0 0x7f00c019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:06.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00bea00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 0x7f00c019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 -- 172.21.15.175:0/3101392319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c010def0 msgr2=0x7f00c019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c010def0 0x7f00c019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 -- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 msgr2=0x7f00c019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 0x7f00c019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.416+0000 7f00be000640 1 -- 172.21.15.175:0/3101392319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c019f1a0 con 0x7f00c0074c30 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00bea00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 0x7f00c019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00be000640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 0x7f00c019a730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00ac004770 tx=0x7f00ac003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac03f070 con 0x7f00c0074c30 2024-07-29T00:42:06.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c019f430 con 0x7f00c0074c30 2024-07-29T00:42:06.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac031ca0 con 0x7f00c0074c30 2024-07-29T00:42:06.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac035030 con 0x7f00c0074c30 2024-07-29T00:42:06.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.417+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c01a6d70 con 0x7f00c0074c30 2024-07-29T00:42:06.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.419+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c00746f0 con 0x7f00c0074c30 2024-07-29T00:42:06.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.421+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f00ac042070 con 0x7f00c0074c30 2024-07-29T00:42:06.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.422+0000 7f00b7400640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00a40a2cf0 0x7f00a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:06.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.422+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f00ac0ef7e0 con 0x7f00c0074c30 2024-07-29T00:42:06.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.422+0000 7f00bea00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00a40a2cf0 0x7f00a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:06.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.423+0000 7f00bea00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00a40a2cf0 0x7f00a40a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009c20 tx=0x7f00b0005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:06.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.427+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f00ac0bc440 con 0x7f00c0074c30 2024-07-29T00:42:06.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.688+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00c0109070 con 0x7f00c0074c30 2024-07-29T00:42:06.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.689+0000 7f00b7400640 1 -- 172.21.15.175:0/3101392319 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+12004 (secure 0 0 0) 0x7f00ac0bbd40 con 0x7f00c0074c30 2024-07-29T00:42:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:42:06.690 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 38 2024-07-29T00:42:06.690 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":38,"btime":"2024-07-29T00:41:37:467718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:41:37.440667+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:42:06.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.691+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00a40a2cf0 msgr2=0x7f00a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:06.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.691+0000 7f00bfe00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00a40a2cf0 0x7f00a40a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009c20 tx=0x7f00b0005f90 comp rx=0 tx=0).stop 2024-07-29T00:42:06.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.691+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 msgr2=0x7f00c019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:06.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.691+0000 7f00bfe00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 0x7f00c019a730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00ac004770 tx=0x7f00ac003c10 comp rx=0 tx=0).stop 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 shutdown_connections 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c010def0 0x7f00c019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c0074c30 0x7f00c019a730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00a40a2cf0 0x7f00a40a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 --2- 172.21.15.175:0/3101392319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c0074310 0x7f00c019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 >> 172.21.15.175:0/3101392319 conn(0x7f00c00fd3d0 msgr2=0x7f00c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 shutdown_connections 2024-07-29T00:42:06.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:06.692+0000 7f00bfe00640 1 -- 172.21.15.175:0/3101392319 wait complete. 2024-07-29T00:42:06.897 INFO:tasks.workunit.client.0.smithi175.stdout: 1 169704 3.05 MB/sec execute 117 sec latency 24.600 ms 2024-07-29T00:42:07.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:42:07.896 INFO:tasks.workunit.client.0.smithi175.stdout: 1 170347 3.06 MB/sec execute 118 sec latency 567.167 ms 2024-07-29T00:42:07.908 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:07 smithi175 ceph-mon[32138]: pgmap v171: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-07-29T00:42:07.908 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:07 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3101392319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:07 smithi181 ceph-mon[36706]: pgmap v171: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-07-29T00:42:07.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:07 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3101392319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:07 smithi204 ceph-mon[37368]: pgmap v171: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-07-29T00:42:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:07 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3101392319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:08.085 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.729+0000 7fdd08e00640 1 -- 172.21.15.175:0/1982874576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04106770 msgr2=0x7fdd04106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.729+0000 7fdd08e00640 1 --2- 172.21.15.175:0/1982874576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04106770 0x7fdd04106b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009cd0 tx=0x7fdcfc02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 -- 172.21.15.175:0/1982874576 shutdown_connections 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 --2- 172.21.15.175:0/1982874576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04101110 0x7fdd0410d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 --2- 172.21.15.175:0/1982874576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04100770 0x7fdd04100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 --2- 172.21.15.175:0/1982874576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04106770 0x7fdd04106b50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 -- 172.21.15.175:0/1982874576 >> 172.21.15.175:0/1982874576 conn(0x7fdd040fc490 msgr2=0x7fdd040fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 -- 172.21.15.175:0/1982874576 shutdown_connections 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.730+0000 7fdd08e00640 1 -- 172.21.15.175:0/1982874576 wait complete. 2024-07-29T00:42:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.731+0000 7fdd08e00640 1 Processor -- start 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 -- start start 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04100770 0x7fdd04102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04101110 0x7fdd04102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04106770 0x7fdd04102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04110bc0 con 0x7fdd04100770 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04110a40 con 0x7fdd04106770 2024-07-29T00:42:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd08e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04110d40 con 0x7fdd04101110 2024-07-29T00:42:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04100770 0x7fdd04102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd01400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04101110 0x7fdd04102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04100770 0x7fdd04102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41760/0 (socket says 172.21.15.175:41760) 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd01e00640 1 -- 172.21.15.175:0/2919738648 learned_addr learned my addr 172.21.15.175:0/2919738648 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.732+0000 7fdd02800640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04106770 0x7fdd04102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.733+0000 7fdd01400640 1 -- 172.21.15.175:0/2919738648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04106770 msgr2=0x7fdd04102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.733+0000 7fdd01400640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04106770 0x7fdd04102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.733+0000 7fdd01400640 1 -- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04100770 msgr2=0x7fdd04102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.733+0000 7fdd01400640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04100770 0x7fdd04102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.733+0000 7fdd01400640 1 -- 172.21.15.175:0/2919738648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd041a6620 con 0x7fdd04101110 2024-07-29T00:42:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.733+0000 7fdd01400640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04101110 0x7fdd04102880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e9f0 tx=0x7fdcec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.734+0000 7fdcfaa00640 1 -- 172.21.15.175:0/2919738648 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00ccc0 con 0x7fdd04101110 2024-07-29T00:42:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.734+0000 7fdcfaa00640 1 -- 172.21.15.175:0/2919738648 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcec010910 con 0x7fdd04101110 2024-07-29T00:42:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.734+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd041a6910 con 0x7fdd04101110 2024-07-29T00:42:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.734+0000 7fdcfaa00640 1 -- 172.21.15.175:0/2919738648 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec018960 con 0x7fdd04101110 2024-07-29T00:42:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.734+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd041a6ea0 con 0x7fdd04101110 2024-07-29T00:42:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.737+0000 7fdcfaa00640 1 -- 172.21.15.175:0/2919738648 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fdcec010ac0 con 0x7fdd04101110 2024-07-29T00:42:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 Processor -- start 2024-07-29T00:42:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 -- start start 2024-07-29T00:42:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e85dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e92b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1debab0 con 0x55c5e1e90720 2024-07-29T00:42:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1d83a80 con 0x55c5e1e85980 2024-07-29T00:42:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1de7a70 con 0x55c5e1e86ef0 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e85dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e85dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:59886/0 (socket says 172.21.15.175:59886) 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.738+0000 7fdd01e00640 1 -- 172.21.15.175:0/2480316132 learned_addr learned my addr 172.21.15.175:0/2480316132 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd02800640 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd01400640 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e92b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd02800640 1 -- 172.21.15.175:0/2480316132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 msgr2=0x55c5e1e85dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd02800640 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e85dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd02800640 1 -- 172.21.15.175:0/2480316132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 msgr2=0x55c5e1e92b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd02800640 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e92b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdd02800640 1 -- 172.21.15.175:0/2480316132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e1b5d3c0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdcfaa00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdce80a2d70 0x7fdce80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.739+0000 7fdcfaa00640 1 -- 172.21.15.175:0/2919738648 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdcec0cf670 con 0x7fdd04101110 2024-07-29T00:42:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.740+0000 7fdd01e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdce80a2d70 0x7fdce80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.740+0000 7fdd02800640 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e85440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4004530 tx=0x7fdcf400ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd5beba1663a52d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.740+0000 7fdcf3400640 1 -- 172.21.15.175:0/2480316132 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4003ad0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.740+0000 7fdcf3400640 1 -- 172.21.15.175:0/2480316132 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf4015070 con 0x55c5e1e86ef0 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.740+0000 7fdcf3400640 1 -- 172.21.15.175:0/2480316132 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40116a0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 -- 172.21.15.175:0/2480316132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 msgr2=0x55c5e1e85440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e85440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4004530 tx=0x7fdcf400ba50 comp rx=0 tx=0).stop 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd01e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdce80a2d70 0x7fdce80a5230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55c5e1e82e70 tx=0x7fdcfc03d040 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 -- 172.21.15.175:0/2480316132 shutdown_connections 2024-07-29T00:42:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e92b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e85dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/2480316132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e85440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.741+0000 7fdd0ba65740 1 -- 172.21.15.175:0/2480316132 >> 172.21.15.175:0/2480316132 conn(0x55c5e1e4e950 msgr2=0x55c5e1e82010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:08.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.742+0000 7fdd0ba65740 1 -- 172.21.15.175:0/2480316132 shutdown_connections 2024-07-29T00:42:08.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.742+0000 7fdd0ba65740 1 -- 172.21.15.175:0/2480316132 wait complete. 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.742+0000 7fdd0ba65740 1 -- start start 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.742+0000 7fdd0ba65740 1 Processor -- start 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e85980 0x55c5e1e8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e90720 0x55c5e1e8e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1e934c0 con 0x55c5e1e85980 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1da6c10 con 0x55c5e1e90720 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1e93640 con 0x55c5e1e86ef0 2024-07-29T00:42:08.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd0ba65740 10 client.0 _ll_register_callbacks cb 0x55c5e1d6fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:08.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd02800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e90720 0x55c5e1e8e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd01400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e85980 0x55c5e1e8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.743+0000 7fdd01400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33426/0 (socket says 172.21.15.175:33426) 2024-07-29T00:42:08.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 -- 172.21.15.175:0/1618644372 learned_addr learned my addr 172.21.15.175:0/1618644372 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:08.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 -- 172.21.15.175:0/1618644372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e90720 msgr2=0x55c5e1e8e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e90720 0x55c5e1e8e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 -- 172.21.15.175:0/1618644372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e85980 msgr2=0x55c5e1e8d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e85980 0x55c5e1e8d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 -- 172.21.15.175:0/1618644372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e1daa1e0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd02800640 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e90720 0x55c5e1e8e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd01400640 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8dc20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdcec018490 tx=0x7fdcec009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad2cbdd5f430cd server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdce7e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdce7e00640 1 -- 172.21.15.175:0/1618644372 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0cd1d0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdce7e00640 1 -- 172.21.15.175:0/1618644372 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdcec0d1de0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.744+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf4015070 con 0x55c5e1e86ef0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.745+0000 7fdce7e00640 1 -- 172.21.15.175:0/1618644372 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdcec010db0 con 0x55c5e1e86ef0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.745+0000 7fdd0ba65740 10 client.26479 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.745+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c5e1e9f300 con 0x55c5e1e86ef0 2024-07-29T00:42:08.747 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.745+0000 7fdce7e00640 1 -- 172.21.15.175:0/1618644372 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcec0b4020 con 0x55c5e1e86ef0 2024-07-29T00:42:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.746+0000 7fdce7e00640 1 -- 172.21.15.175:0/1618644372 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fdcec010430 con 0x55c5e1e86ef0 2024-07-29T00:42:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.746+0000 7fdd0ba65740 10 client.26479 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.746+0000 7fdd0ba65740 10 client.26479 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:42:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.746+0000 7fdd0ba65740 20 client.26479 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:08.749 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.747+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1ea07c0 0x55c5e1ea2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.747+0000 7fdd0ba65740 4 client.26479 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.747+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5e1df9020 con 0x55c5e1ea07c0 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.747+0000 7fdd01e00640 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1ea07c0 0x55c5e1ea2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.747+0000 7fdd01e00640 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1ea07c0 0x55c5e1ea2bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e5c53028d2a86307 server_cookie=3eb8b9e3a8862b8b in_seq=0 out_seq=0 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.748+0000 7fdce7e00640 10 client.26479 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.748+0000 7fdce7e00640 1 -- 172.21.15.175:0/1618644372 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c5e1df9020 con 0x55c5e1ea07c0 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.748+0000 7fdce7e00640 10 client.26479 handle_command_reply: tid=0 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.748+0000 7fdd0ba65740 1 client.26479 shutdown 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 msgr2=0x55c5e1e8dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8dc20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdcec018490 tx=0x7fdcec009700 comp rx=0 tx=0).stop 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 shutdown_connections 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1ea07c0 0x55c5e1ea2bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e90720 0x55c5e1e8e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8dc20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1618644372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e85980 0x55c5e1e8d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdce7e00640 0 client.26479 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 >> 172.21.15.175:0/1618644372 conn(0x55c5e1e4e950 msgr2=0x55c5e1e87ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 shutdown_connections 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1618644372 wait complete. 2024-07-29T00:42:08.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.749+0000 7fdd0ba65740 20 client.26479 trim_cache size 0 max 16384 2024-07-29T00:42:08.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.765+0000 7fdd0ba65740 1 Processor -- start 2024-07-29T00:42:08.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.765+0000 7fdd0ba65740 1 -- start start 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.765+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.765+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e86ef0 0x55c5e1ea8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.765+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e90720 0x55c5e1ea8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdce000e850 con 0x55c5e1e86ef0 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdce00104b0 con 0x55c5e1e85980 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdce000ef20 con 0x55c5e1e90720 2024-07-29T00:42:08.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42832/0 (socket says 172.21.15.175:42832) 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 -- 172.21.15.175:0/1483063569 learned_addr learned my addr 172.21.15.175:0/1483063569 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd02800640 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e90720 0x55c5e1ea8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01400640 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e86ef0 0x55c5e1ea8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 -- 172.21.15.175:0/1483063569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e90720 msgr2=0x55c5e1ea8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e90720 0x55c5e1ea8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 -- 172.21.15.175:0/1483063569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e86ef0 msgr2=0x55c5e1ea8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e86ef0 0x55c5e1ea8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 -- 172.21.15.175:0/1483063569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e1ea31e0 con 0x55c5e1e85980 2024-07-29T00:42:08.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.766+0000 7fdd01e00640 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e821c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc033040 tx=0x7fdcfc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48784070fbbdc499 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:08.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.767+0000 7fdcf2a00640 1 -- 172.21.15.175:0/1483063569 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc004510 con 0x55c5e1e85980 2024-07-29T00:42:08.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.767+0000 7fdcf2a00640 1 -- 172.21.15.175:0/1483063569 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcfc036040 con 0x55c5e1e85980 2024-07-29T00:42:08.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.767+0000 7fdcf2a00640 1 -- 172.21.15.175:0/1483063569 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc030800 con 0x55c5e1e85980 2024-07-29T00:42:08.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1483063569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 msgr2=0x55c5e1e821c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e821c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc033040 tx=0x7fdcfc0095c0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1483063569 shutdown_connections 2024-07-29T00:42:08.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e90720 0x55c5e1ea8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e86ef0 0x55c5e1ea8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/1483063569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e821c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1483063569 >> 172.21.15.175:0/1483063569 conn(0x55c5e1e4e950 msgr2=0x55c5e1e81ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:08.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1483063569 shutdown_connections 2024-07-29T00:42:08.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.768+0000 7fdd0ba65740 1 -- 172.21.15.175:0/1483063569 wait complete. 2024-07-29T00:42:08.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.769+0000 7fdd0ba65740 1 -- start start 2024-07-29T00:42:08.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.769+0000 7fdd0ba65740 1 Processor -- start 2024-07-29T00:42:08.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.769+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e98d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.769+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd0ba65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e8a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdce000fd20 con 0x55c5e1e90720 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdce000f5f0 con 0x55c5e1e85980 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd0ba65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdce000c830 con 0x55c5e1e86ef0 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd0ba65740 10 client.0 _ll_register_callbacks cb 0x7fdcec0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:08.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e98d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e98d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42846/0 (socket says 172.21.15.175:42846) 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 -- 172.21.15.175:0/717061271 learned_addr learned my addr 172.21.15.175:0/717061271 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd02800640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e8a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 -- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 msgr2=0x55c5e1e8a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 -- 172.21.15.175:0/717061271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 msgr2=0x55c5e1e8a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e8a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd01e00640 1 -- 172.21.15.175:0/717061271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e1e8ab20 con 0x55c5e1e85980 2024-07-29T00:42:08.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.770+0000 7fdd02800640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e8a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdd01e00640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e98d60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc053b40 tx=0x7fdcfc0317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b647770f42a0979 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdce7e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdce7e00640 1 -- 172.21.15.175:0/717061271 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc004510 con 0x55c5e1e85980 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdce7e00640 1 -- 172.21.15.175:0/717061271 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fdcfc0046b0 con 0x55c5e1e85980 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcfc036040 con 0x55c5e1e85980 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdce7e00640 1 -- 172.21.15.175:0/717061271 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdcfc0533c0 con 0x55c5e1e85980 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdd0ba65740 10 client.26522 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:08.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.771+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c5e1f0b0b0 con 0x55c5e1e85980 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdce7e00640 1 -- 172.21.15.175:0/717061271 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcfc036040 con 0x55c5e1e85980 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdce7e00640 1 -- 172.21.15.175:0/717061271 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fdcfc030070 con 0x55c5e1e85980 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd0ba65740 10 client.26522 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd0ba65740 10 client.26522 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd0ba65740 20 client.26522 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1f0b810 0x55c5e1f0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:08.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd0ba65740 4 client.26522 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:42:08.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5e1df3530 con 0x55c5e1f0b810 2024-07-29T00:42:08.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.772+0000 7fdd01400640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1f0b810 0x55c5e1f0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:08.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.773+0000 7fdce7e00640 10 client.26522 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:42:08.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.773+0000 7fdd01400640 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1f0b810 0x55c5e1f0dc30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d618589a58800bf server_cookie=b992fb4fdb1960ab in_seq=0 out_seq=0 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.773+0000 7fdce7e00640 1 -- 172.21.15.175:0/717061271 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5e1df3530 con 0x55c5e1f0b810 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.773+0000 7fdce7e00640 10 client.26522 handle_command_reply: tid=0 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.773+0000 7fdd0ba65740 1 client.26522 shutdown 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 msgr2=0x55c5e1e98d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e98d60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc053b40 tx=0x7fdcfc0317b0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 shutdown_connections 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5e1e90720 0x55c5e1e8a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e1e86ef0 0x55c5e1e8a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c5e1f0b810 0x55c5e1f0dc30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 --2- 172.21.15.175:0/717061271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5e1e85980 0x55c5e1e98d60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdce7e00640 0 client.26522 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 >> 172.21.15.175:0/717061271 conn(0x55c5e1e4e950 msgr2=0x55c5e1e874c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 shutdown_connections 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 1 -- 172.21.15.175:0/717061271 wait complete. 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.774+0000 7fdd0ba65740 20 client.26522 trim_cache size 0 max 16384 2024-07-29T00:42:08.782 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdce80a2d70 msgr2=0x7fdce80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdce80a2d70 0x7fdce80a5230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55c5e1e82e70 tx=0x7fdcfc03d040 comp rx=0 tx=0).stop 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04101110 msgr2=0x7fdd04102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04101110 0x7fdd04102880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e9f0 tx=0x7fdcec00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 shutdown_connections 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04106770 0x7fdd04102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd04101110 0x7fdd04102880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdce80a2d70 0x7fdce80a5230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 --2- 172.21.15.175:0/2919738648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd04100770 0x7fdd04102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:08.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 >> 172.21.15.175:0/2919738648 conn(0x7fdd040fc490 msgr2=0x7fdd0406a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:08.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 shutdown_connections 2024-07-29T00:42:08.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:08.775+0000 7fdd08e00640 1 -- 172.21.15.175:0/2919738648 wait complete. 2024-07-29T00:42:08.896 INFO:tasks.workunit.client.0.smithi175.stdout: 1 170811 3.05 MB/sec execute 119 sec latency 637.489 ms 2024-07-29T00:42:09.077 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.3 2024-07-29T00:42:09.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:42:09.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:42:09.896 INFO:tasks.workunit.client.0.smithi175.stdout: 1 171922 3.07 MB/sec execute 120 sec latency 19.366 ms 2024-07-29T00:42:09.912 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:09.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:09 smithi181 ceph-mon[36706]: pgmap v172: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-07-29T00:42:10.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:09 smithi175 ceph-mon[32138]: pgmap v172: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-07-29T00:42:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:09 smithi204 ceph-mon[37368]: pgmap v172: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-07-29T00:42:10.897 INFO:tasks.workunit.client.0.smithi175.stdout: 1 173272 3.10 MB/sec execute 121 sec latency 147.399 ms 2024-07-29T00:42:11.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.534+0000 7fba49e00640 1 -- 172.21.15.175:0/736216947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba4410be90 msgr2=0x7fba4410e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:11.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.534+0000 7fba49e00640 1 --2- 172.21.15.175:0/736216947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba4410be90 0x7fba4410e300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009be0 tx=0x7fba3c02a750 comp rx=0 tx=0).stop 2024-07-29T00:42:11.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 -- 172.21.15.175:0/736216947 shutdown_connections 2024-07-29T00:42:11.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 --2- 172.21.15.175:0/736216947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba4410be90 0x7fba4410e300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 --2- 172.21.15.175:0/736216947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba44103410 0x7fba4410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 --2- 172.21.15.175:0/736216947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44102af0 0x7fba44102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 -- 172.21.15.175:0/736216947 >> 172.21.15.175:0/736216947 conn(0x7fba440fc490 msgr2=0x7fba440fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 -- 172.21.15.175:0/736216947 shutdown_connections 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.535+0000 7fba49e00640 1 -- 172.21.15.175:0/736216947 wait complete. 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.536+0000 7fba49e00640 1 Processor -- start 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.536+0000 7fba49e00640 1 -- start start 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba49e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 0x7fba441980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba49e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44103410 0x7fba44198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba49e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4410be90 0x7fba4419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba49e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba44111c60 con 0x7fba4410be90 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba49e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba44111ae0 con 0x7fba44102af0 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba49e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba44111de0 con 0x7fba44103410 2024-07-29T00:42:11.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 0x7fba441980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44103410 0x7fba44198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 0x7fba441980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42872/0 (socket says 172.21.15.175:42872) 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 -- 172.21.15.175:0/4004851264 learned_addr learned my addr 172.21.15.175:0/4004851264 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba43400640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4410be90 0x7fba4419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 -- 172.21.15.175:0/4004851264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44103410 msgr2=0x7fba44198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44103410 0x7fba44198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 -- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4410be90 msgr2=0x7fba4419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4410be90 0x7fba4419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42a00640 1 -- 172.21.15.175:0/4004851264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4419d070 con 0x7fba44102af0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.537+0000 7fba42000640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44103410 0x7fba44198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.538+0000 7fba42a00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 0x7fba441980c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e9c0 tx=0x7fba3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.538+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400cd60 con 0x7fba44102af0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.538+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba340035f0 con 0x7fba44102af0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.538+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34010670 con 0x7fba44102af0 2024-07-29T00:42:11.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.538+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4419d360 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.538+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba44100680 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.539+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba44102ed0 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.541+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fba340037a0 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.543+0000 7fba48a00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fba280a2d70 0x7fba280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.543+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fba340cf680 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.543+0000 7fba42000640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fba280a2d70 0x7fba280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.544+0000 7fba42000640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fba280a2d70 0x7fba280a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba4410d290 tx=0x7fba30005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.547+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fba3409c220 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.803+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba44199420 con 0x7fba44102af0 2024-07-29T00:42:11.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.805+0000 7fba48a00640 1 -- 172.21.15.175:0/4004851264 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+12004 (secure 0 0 0) 0x7fba3409bb20 con 0x7fba44102af0 2024-07-29T00:42:11.299 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:42:11.299 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":38,"btime":"2024-07-29T00:41:37:467718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:41:37.440667+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:42:11.299 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 38 2024-07-29T00:42:11.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.807+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fba280a2d70 msgr2=0x7fba280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:11.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.807+0000 7fba49e00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fba280a2d70 0x7fba280a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba4410d290 tx=0x7fba30005f90 comp rx=0 tx=0).stop 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.807+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 msgr2=0x7fba441980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.807+0000 7fba49e00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 0x7fba441980c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e9c0 tx=0x7fba3400ee80 comp rx=0 tx=0).stop 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 shutdown_connections 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fba280a2d70 0x7fba280a5230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4410be90 0x7fba4419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba44103410 0x7fba44198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 --2- 172.21.15.175:0/4004851264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba44102af0 0x7fba441980c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 >> 172.21.15.175:0/4004851264 conn(0x7fba440fc490 msgr2=0x7fba440fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 shutdown_connections 2024-07-29T00:42:11.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:10.808+0000 7fba49e00640 1 -- 172.21.15.175:0/4004851264 wait complete. 2024-07-29T00:42:11.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 174141 3.13 MB/sec execute 122 sec latency 191.902 ms 2024-07-29T00:42:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:11 smithi181 ceph-mon[36706]: pgmap v173: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-07-29T00:42:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:11 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/4004851264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:11 smithi175 ceph-mon[32138]: pgmap v173: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-07-29T00:42:12.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:11 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/4004851264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:11 smithi204 ceph-mon[37368]: pgmap v173: 97 pgs: 97 active+clean; 112 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-07-29T00:42:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:11 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/4004851264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:12.214 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:12.262 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0' 2024-07-29T00:42:12.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0 2024-07-29T00:42:12.897 INFO:tasks.workunit.client.0.smithi175.stdout: 1 174380 3.11 MB/sec execute 123 sec latency 450.387 ms 2024-07-29T00:42:13.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 -- 172.21.15.175:0/3600250386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 msgr2=0x7f0044102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 --2- 172.21.15.175:0/3600250386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044102e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0034009930 tx=0x7f003402a440 comp rx=0 tx=0).stop 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 -- 172.21.15.175:0/3600250386 shutdown_connections 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 --2- 172.21.15.175:0/3600250386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0044107f90 0x7f004410e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 --2- 172.21.15.175:0/3600250386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 0x7f0044107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 --2- 172.21.15.175:0/3600250386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044102e00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.445+0000 7f0048a00640 1 -- 172.21.15.175:0/3600250386 >> 172.21.15.175:0/3600250386 conn(0x7f00440fc4c0 msgr2=0x7f00440fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.446+0000 7f0048a00640 1 -- 172.21.15.175:0/3600250386 shutdown_connections 2024-07-29T00:42:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.446+0000 7f0048a00640 1 -- 172.21.15.175:0/3600250386 wait complete. 2024-07-29T00:42:13.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.447+0000 7f0048a00640 1 Processor -- start 2024-07-29T00:42:13.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.447+0000 7f0048a00640 1 -- start start 2024-07-29T00:42:13.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.447+0000 7f0048a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.447+0000 7f0048a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 0x7f0044100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0048a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0044107f90 0x7f0044101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0048a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0044111de0 con 0x7f00441033d0 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0048a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0044111c60 con 0x7f0044102a20 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0048a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0044111f60 con 0x7f0044107f90 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42888/0 (socket says 172.21.15.175:42888) 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 -- 172.21.15.175:0/3451189816 learned_addr learned my addr 172.21.15.175:0/3451189816 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0042800640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0044107f90 0x7f0044101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041400640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 0x7f0044100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 -- 172.21.15.175:0/3451189816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0044107f90 msgr2=0x7f0044101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0044107f90 0x7f0044101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 -- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 msgr2=0x7f0044100bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 0x7f0044100bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041e00640 1 -- 172.21.15.175:0/3451189816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441a6790 con 0x7f0044102a20 2024-07-29T00:42:13.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.448+0000 7f0041400640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 0x7f0044100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:13.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.449+0000 7f0041e00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044100680 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f003402a960 tx=0x7f0034003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:13.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.449+0000 7f003aa00640 1 -- 172.21.15.175:0/3451189816 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003403f070 con 0x7f0044102a20 2024-07-29T00:42:13.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.449+0000 7f003aa00640 1 -- 172.21.15.175:0/3451189816 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0034031c90 con 0x7f0044102a20 2024-07-29T00:42:13.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.449+0000 7f003aa00640 1 -- 172.21.15.175:0/3451189816 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034035030 con 0x7f0044102a20 2024-07-29T00:42:13.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.449+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441a6a20 con 0x7f0044102a20 2024-07-29T00:42:13.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.449+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a6eb0 con 0x7f0044102a20 2024-07-29T00:42:13.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.452+0000 7f004b5f5740 1 Processor -- start 2024-07-29T00:42:13.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f003aa00640 1 -- 172.21.15.175:0/3451189816 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0034042070 con 0x7f0044102a20 2024-07-29T00:42:13.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 -- start start 2024-07-29T00:42:13.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e362440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e362dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55945e2c8ab0 con 0x55945e363ef0 2024-07-29T00:42:13.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55945e260a80 con 0x55945e36d720 2024-07-29T00:42:13.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.453+0000 7f004b5f5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55945e2c4a70 con 0x55945e362980 2024-07-29T00:42:13.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e362dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e362dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45184/0 (socket says 172.21.15.175:45184) 2024-07-29T00:42:13.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041e00640 1 -- 172.21.15.175:0/2183310206 learned_addr learned my addr 172.21.15.175:0/2183310206 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:13.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0042800640 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e362440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041400640 1 -- 172.21.15.175:0/2183310206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 msgr2=0x55945e362dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041400640 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e362dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041400640 1 -- 172.21.15.175:0/2183310206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 msgr2=0x55945e362440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041400640 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e362440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041400640 1 -- 172.21.15.175:0/2183310206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945e03a3c0 con 0x55945e36d720 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.454+0000 7f0041e00640 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e362dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f0041400640 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36fb10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f004406a060 tx=0x7f00300117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ff0e6e142035e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f002f000640 1 -- 172.21.15.175:0/2183310206 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030012450 con 0x55945e36d720 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f002f000640 1 -- 172.21.15.175:0/2183310206 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0030012ad0 con 0x55945e36d720 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f003aa00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00280a2d70 0x7f00280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f002f000640 1 -- 172.21.15.175:0/2183310206 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003000fad0 con 0x55945e36d720 2024-07-29T00:42:13.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f003aa00640 1 -- 172.21.15.175:0/3451189816 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f00340ef9b0 con 0x7f0044102a20 2024-07-29T00:42:13.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.455+0000 7f0042800640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00280a2d70 0x7f00280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 -- 172.21.15.175:0/2183310206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 msgr2=0x55945e36fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36fb10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f004406a060 tx=0x7f00300117b0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 -- 172.21.15.175:0/2183310206 shutdown_connections 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36fb10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e362dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 --2- 172.21.15.175:0/2183310206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e362440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 -- 172.21.15.175:0/2183310206 >> 172.21.15.175:0/2183310206 conn(0x55945e32b950 msgr2=0x55945e35f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f0042800640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00280a2d70 0x7f00280a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f003c005fd0 tx=0x7f003c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 -- 172.21.15.175:0/2183310206 shutdown_connections 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.456+0000 7f004b5f5740 1 -- 172.21.15.175:0/2183310206 wait complete. 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.457+0000 7f004b5f5740 1 -- start start 2024-07-29T00:42:13.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.457+0000 7f004b5f5740 1 Processor -- start 2024-07-29T00:42:13.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e36a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e36ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55945e3704c0 con 0x55945e362980 2024-07-29T00:42:13.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55945e283c10 con 0x55945e36d720 2024-07-29T00:42:13.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55945e370640 con 0x55945e363ef0 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f004b5f5740 10 client.0 _ll_register_callbacks cb 0x55945e24ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0042800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0041e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e36a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0042800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42910/0 (socket says 172.21.15.175:42910) 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0042800640 1 -- 172.21.15.175:0/431536560 learned_addr learned my addr 172.21.15.175:0/431536560 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0041400640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e36ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0041400640 1 -- 172.21.15.175:0/431536560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 msgr2=0x55945e36b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0041400640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0041400640 1 -- 172.21.15.175:0/431536560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 msgr2=0x55945e36a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.458+0000 7f0041400640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e36a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0041400640 1 -- 172.21.15.175:0/431536560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945e2871e0 con 0x55945e363ef0 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0042800640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0041400640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e36ab20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f003001c040 tx=0x7f0030009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d12dce00e7c2077 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0023400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0023400640 1 -- 172.21.15.175:0/431536560 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030029020 con 0x55945e363ef0 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0030012ad0 con 0x55945e363ef0 2024-07-29T00:42:13.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0023400640 1 -- 172.21.15.175:0/431536560 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f0030012660 con 0x55945e363ef0 2024-07-29T00:42:13.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.459+0000 7f0023400640 1 -- 172.21.15.175:0/431536560 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0030034050 con 0x55945e363ef0 2024-07-29T00:42:13.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.460+0000 7f004b5f5740 10 client.26500 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:13.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.460+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55945e37c210 con 0x55945e363ef0 2024-07-29T00:42:13.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.460+0000 7f0023400640 1 -- 172.21.15.175:0/431536560 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0030012ad0 con 0x55945e363ef0 2024-07-29T00:42:13.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f0023400640 1 -- 172.21.15.175:0/431536560 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f003000fad0 con 0x55945e363ef0 2024-07-29T00:42:13.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f004b5f5740 10 client.26500 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:13.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f004b5f5740 10 client.26500 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:42:13.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f004b5f5740 20 client.26500 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:13.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f004b5f5740 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e37d5c0 0x55945e37f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f004b5f5740 4 client.26500 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:13.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55945e2d6020 con 0x55945e37d5c0 2024-07-29T00:42:13.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.461+0000 7f0041e00640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e37d5c0 0x55945e37f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.462+0000 7f0041e00640 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e37d5c0 0x55945e37f9b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a537c54f4c21e49 server_cookie=915a4aa1384c562d in_seq=0 out_seq=0 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.462+0000 7f0023400640 10 client.26500 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f0023400640 1 -- 172.21.15.175:0/431536560 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55945e2d6020 con 0x55945e37d5c0 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f0023400640 10 client.26500 handle_command_reply: tid=0 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 client.26500 shutdown 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 msgr2=0x55945e36ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e36ab20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f003001c040 tx=0x7f0030009ed0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 shutdown_connections 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e37d5c0 0x55945e37f9b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e36b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e36ab20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f0023400640 0 client.26500 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 --2- 172.21.15.175:0/431536560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e36a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.463+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 >> 172.21.15.175:0/431536560 conn(0x55945e32b950 msgr2=0x55945e364ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:13.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.464+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 shutdown_connections 2024-07-29T00:42:13.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.464+0000 7f004b5f5740 1 -- 172.21.15.175:0/431536560 wait complete. 2024-07-29T00:42:13.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.464+0000 7f004b5f5740 20 client.26500 trim_cache size 0 max 16384 2024-07-29T00:42:13.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.487+0000 7f004b5f5740 1 Processor -- start 2024-07-29T00:42:13.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.487+0000 7f004b5f5740 1 -- start start 2024-07-29T00:42:13.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e385040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e385580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e35d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f004b5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f002400e850 con 0x55945e363ef0 2024-07-29T00:42:13.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f004b5f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00240104b0 con 0x55945e36d720 2024-07-29T00:42:13.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f004b5f5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f002400ef20 con 0x55945e362980 2024-07-29T00:42:13.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f0042800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e35d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0042800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e35d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42916/0 (socket says 172.21.15.175:42916) 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0042800640 1 -- 172.21.15.175:0/4083833302 learned_addr learned my addr 172.21.15.175:0/4083833302 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0041400640 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e385580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.488+0000 7f0041e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e385040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0041400640 1 -- 172.21.15.175:0/4083833302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 msgr2=0x55945e385040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0041400640 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e385040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0041400640 1 -- 172.21.15.175:0/4083833302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 msgr2=0x55945e35d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0041400640 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e35d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.489+0000 7f0041400640 1 -- 172.21.15.175:0/4083833302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945e3654d0 con 0x55945e363ef0 2024-07-29T00:42:13.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.490+0000 7f0041e00640 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e385040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:13.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.490+0000 7f0041400640 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e385580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0030012fd0 tx=0x7f0030009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af46d6a16a51702a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:13.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.491+0000 7f002e600640 1 -- 172.21.15.175:0/4083833302 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030043020 con 0x55945e363ef0 2024-07-29T00:42:13.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.491+0000 7f002e600640 1 -- 172.21.15.175:0/4083833302 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0030012ad0 con 0x55945e363ef0 2024-07-29T00:42:13.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.491+0000 7f002e600640 1 -- 172.21.15.175:0/4083833302 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030029070 con 0x55945e363ef0 2024-07-29T00:42:13.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.492+0000 7f004b5f5740 1 -- 172.21.15.175:0/4083833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 msgr2=0x55945e385580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.492+0000 7f004b5f5740 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e385580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0030012fd0 tx=0x7f0030009a80 comp rx=0 tx=0).stop 2024-07-29T00:42:13.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.493+0000 7f004b5f5740 1 -- 172.21.15.175:0/4083833302 shutdown_connections 2024-07-29T00:42:13.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.493+0000 7f004b5f5740 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e35d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.493+0000 7f004b5f5740 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e363ef0 0x55945e385580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.493+0000 7f004b5f5740 1 --2- 172.21.15.175:0/4083833302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e362980 0x55945e385040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.493+0000 7f004b5f5740 1 -- 172.21.15.175:0/4083833302 >> 172.21.15.175:0/4083833302 conn(0x55945e32b950 msgr2=0x55945e3696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:13.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.494+0000 7f004b5f5740 1 -- 172.21.15.175:0/4083833302 shutdown_connections 2024-07-29T00:42:13.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.494+0000 7f004b5f5740 1 -- 172.21.15.175:0/4083833302 wait complete. 2024-07-29T00:42:13.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.495+0000 7f004b5f5740 1 -- start start 2024-07-29T00:42:13.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.495+0000 7f004b5f5740 1 Processor -- start 2024-07-29T00:42:13.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.497+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e3761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.497+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e371560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.497+0000 7f004b5f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e371b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.497+0000 7f0041e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e3761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.497+0000 7f0041e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e3761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37900/0 (socket says 172.21.15.175:37900) 2024-07-29T00:42:13.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.498+0000 7f0041400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e371560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.498+0000 7f0041400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e371560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45210/0 (socket says 172.21.15.175:45210) 2024-07-29T00:42:13.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.499+0000 7f0041e00640 1 -- 172.21.15.175:0/1844326136 learned_addr learned my addr 172.21.15.175:0/1844326136 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:13.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.499+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f002400fd20 con 0x55945e362980 2024-07-29T00:42:13.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.499+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f002400f5f0 con 0x55945e36d720 2024-07-29T00:42:13.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.499+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f002400c830 con 0x55945e363ef0 2024-07-29T00:42:13.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.499+0000 7f0042800640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e371b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.499+0000 7f004b5f5740 10 client.0 _ll_register_callbacks cb 0x7f003c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:13.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.500+0000 7f0041e00640 1 -- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 msgr2=0x55945e371560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.500+0000 7f0041e00640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e371560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.500+0000 7f0041e00640 1 -- 172.21.15.175:0/1844326136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 msgr2=0x55945e371b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:13.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.500+0000 7f0041e00640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e371b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.500+0000 7f0041e00640 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945e3723a0 con 0x55945e362980 2024-07-29T00:42:13.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.501+0000 7f0041e00640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e3761e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f00340bbb50 tx=0x7f00340f1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0fd6236b2bd717 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:13.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.501+0000 7f0023400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:42:13.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f0023400640 1 -- 172.21.15.175:0/1844326136 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034049040 con 0x55945e362980 2024-07-29T00:42:13.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f0041400640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e371560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:13.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f0042800640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e371b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:13.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0030012ad0 con 0x55945e362980 2024-07-29T00:42:13.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f0023400640 1 -- 172.21.15.175:0/1844326136 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f003403b460 con 0x55945e362980 2024-07-29T00:42:13.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f0023400640 1 -- 172.21.15.175:0/1844326136 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f00340bc550 con 0x55945e362980 2024-07-29T00:42:13.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f004b5f5740 10 client.16413 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:13.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.502+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55945e3e8060 con 0x55945e362980 2024-07-29T00:42:13.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f0023400640 1 -- 172.21.15.175:0/1844326136 <== mon.0 v2:172.21.15.175:3300/0 4 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f00340b84c0 con 0x55945e362980 2024-07-29T00:42:13.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f004b5f5740 10 client.16413 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:13.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f004b5f5740 10 client.16413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:42:13.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f004b5f5740 20 client.16413 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:13.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f004b5f5740 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e3e8620 0x55945e3eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:13.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f004b5f5740 4 client.16413 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:42:13.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55945e2d0530 con 0x55945e3e8620 2024-07-29T00:42:13.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.503+0000 7f0041400640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e3e8620 0x55945e3eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.504+0000 7f0023400640 10 client.16413 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.504+0000 7f0041400640 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e3e8620 0x55945e3eaa10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a6282266914a587 server_cookie=191b96ca60239171 in_seq=0 out_seq=0 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.504+0000 7f0023400640 1 -- 172.21.15.175:0/1844326136 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55945e2d0530 con 0x55945e3e8620 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.504+0000 7f0023400640 10 client.16413 handle_command_reply: tid=0 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 client.16413 shutdown 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 msgr2=0x55945e3761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e3761e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f00340bbb50 tx=0x7f00340f1e00 comp rx=0 tx=0).stop 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 shutdown_connections 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55945e3e8620 0x55945e3eaa10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55945e36d720 0x55945e371b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945e363ef0 0x55945e371560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 --2- 172.21.15.175:0/1844326136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945e362980 0x55945e3761e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f0023400640 0 client.16413 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:42:13.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.505+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 >> 172.21.15.175:0/1844326136 conn(0x55945e32b950 msgr2=0x55945e3644c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:13.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.506+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 shutdown_connections 2024-07-29T00:42:13.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.506+0000 7f004b5f5740 1 -- 172.21.15.175:0/1844326136 wait complete. 2024-07-29T00:42:13.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.506+0000 7f004b5f5740 20 client.16413 trim_cache size 0 max 16384 2024-07-29T00:42:13.507 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00280a2d70 msgr2=0x7f00280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00280a2d70 0x7f00280a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f003c005fd0 tx=0x7f003c009500 comp rx=0 tx=0).stop 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 msgr2=0x7f0044100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044100680 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f003402a960 tx=0x7f0034003b20 comp rx=0 tx=0).stop 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 shutdown_connections 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0044107f90 0x7f0044101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f00280a2d70 0x7f00280a5230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00441033d0 0x7f0044100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 --2- 172.21.15.175:0/3451189816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0044102a20 0x7f0044100680 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:13.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 >> 172.21.15.175:0/3451189816 conn(0x7f00440fc4c0 msgr2=0x7f0044105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:13.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 shutdown_connections 2024-07-29T00:42:13.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:13.507+0000 7f0048a00640 1 -- 172.21.15.175:0/3451189816 wait complete. 2024-07-29T00:42:13.509 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:13.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 175278 3.12 MB/sec execute 124 sec latency 619.711 ms 2024-07-29T00:42:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:13 smithi175 ceph-mon[32138]: pgmap v174: 97 pgs: 97 active+clean; 115 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-07-29T00:42:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:13 smithi181 ceph-mon[36706]: pgmap v174: 97 pgs: 97 active+clean; 115 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-07-29T00:42:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:13 smithi204 ceph-mon[37368]: pgmap v174: 97 pgs: 97 active+clean; 115 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-07-29T00:42:14.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:42:14.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176313 3.12 MB/sec execute 125 sec latency 26.338 ms 2024-07-29T00:42:15.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 -- 172.21.15.175:0/2830036008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c109aa0 msgr2=0x7f924c109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 --2- 172.21.15.175:0/2830036008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c109aa0 0x7f924c109e80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9240009930 tx=0x7f924002a420 comp rx=0 tx=0).stop 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 -- 172.21.15.175:0/2830036008 shutdown_connections 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 --2- 172.21.15.175:0/2830036008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c104440 0x7f924c110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 --2- 172.21.15.175:0/2830036008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 --2- 172.21.15.175:0/2830036008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c109aa0 0x7f924c109e80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 -- 172.21.15.175:0/2830036008 >> 172.21.15.175:0/2830036008 conn(0x7f924c0ff740 msgr2=0x7f924c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 -- 172.21.15.175:0/2830036008 shutdown_connections 2024-07-29T00:42:15.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.156+0000 7f9254000640 1 -- 172.21.15.175:0/2830036008 wait complete. 2024-07-29T00:42:15.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.157+0000 7f9254000640 1 Processor -- start 2024-07-29T00:42:15.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 -- start start 2024-07-29T00:42:15.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c104440 0x7f924c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c109aa0 0x7f924c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f924c113ef0 con 0x7f924c109aa0 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f924c113d70 con 0x7f924c103aa0 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9254000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f924c114070 con 0x7f924c104440 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9250c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c104440 0x7f924c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9251600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9251600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42936/0 (socket says 172.21.15.175:42936) 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.158+0000 7f9251600640 1 -- 172.21.15.175:0/2822450212 learned_addr learned my addr 172.21.15.175:0/2822450212 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:15.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9252000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c109aa0 0x7f924c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9251600640 1 -- 172.21.15.175:0/2822450212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c104440 msgr2=0x7f924c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9251600640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c104440 0x7f924c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9251600640 1 -- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c109aa0 msgr2=0x7f924c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9251600640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c109aa0 0x7f924c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9251600640 1 -- 172.21.15.175:0/2822450212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c19f340 con 0x7f924c103aa0 2024-07-29T00:42:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9250c00640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c104440 0x7f924c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.159+0000 7f9251600640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c19a390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f924002a940 tx=0x7f9240003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.160+0000 7f924a000640 1 -- 172.21.15.175:0/2822450212 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924003f070 con 0x7f924c103aa0 2024-07-29T00:42:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.160+0000 7f924a000640 1 -- 172.21.15.175:0/2822450212 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9240031c90 con 0x7f924c103aa0 2024-07-29T00:42:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.160+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c19f5d0 con 0x7f924c103aa0 2024-07-29T00:42:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.160+0000 7f924a000640 1 -- 172.21.15.175:0/2822450212 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240035030 con 0x7f924c103aa0 2024-07-29T00:42:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.160+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c1a6f10 con 0x7f924c103aa0 2024-07-29T00:42:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.163+0000 7f924a000640 1 -- 172.21.15.175:0/2822450212 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9240042070 con 0x7f924c103aa0 2024-07-29T00:42:15.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.164+0000 7f9256a20740 1 Processor -- start 2024-07-29T00:42:15.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.164+0000 7f9256a20740 1 -- start start 2024-07-29T00:42:15.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9256a20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 0x557b4b7ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9256a20740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7adb70 0x557b4b7b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9256a20740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b7ba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9256a20740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b4b712a80 con 0x557b4b7adb70 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9256a20740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557b4b6aa080 con 0x557b4b7b8430 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9256a20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557b4b70ea40 con 0x557b4b7ad250 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9252000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 0x557b4b7ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9252000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 0x557b4b7ad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45244/0 (socket says 172.21.15.175:45244) 2024-07-29T00:42:15.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9252000640 1 -- 172.21.15.175:0/2682425083 learned_addr learned my addr 172.21.15.175:0/2682425083 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f924a000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92340a2cf0 0x7f92340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.165+0000 7f9251600640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7adb70 0x557b4b7b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9250c00640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b7ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f924a000640 1 -- 172.21.15.175:0/2822450212 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f92400ef820 con 0x7f924c103aa0 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9252000640 1 -- 172.21.15.175:0/2682425083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 msgr2=0x557b4b7ba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9252000640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b7ba820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9252000640 1 -- 172.21.15.175:0/2682425083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7adb70 msgr2=0x557b4b7b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9252000640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7adb70 0x557b4b7b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9252000640 1 -- 172.21.15.175:0/2682425083 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4b484b90 con 0x557b4b7ad250 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9251600640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7adb70 0x557b4b7b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9250c00640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b7ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.166+0000 7f9252000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92340a2cf0 0x7f92340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.167+0000 7f9252000640 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 0x557b4b7ad630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f92440124c0 tx=0x7f9244012980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=178d60fba77f56a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.167+0000 7f923e600640 1 -- 172.21.15.175:0/2682425083 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924401aaa0 con 0x557b4b7ad250 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.167+0000 7f923e600640 1 -- 172.21.15.175:0/2682425083 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f924401ac40 con 0x557b4b7ad250 2024-07-29T00:42:15.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.167+0000 7f923e600640 1 -- 172.21.15.175:0/2682425083 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244025600 con 0x557b4b7ad250 2024-07-29T00:42:15.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 -- 172.21.15.175:0/2682425083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 msgr2=0x557b4b7ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9252000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92340a2cf0 0x7f92340a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9244021000 tx=0x7f924401e000 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 0x557b4b7ad630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f92440124c0 tx=0x7f9244012980 comp rx=0 tx=0).stop 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 -- 172.21.15.175:0/2682425083 shutdown_connections 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b7ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7adb70 0x557b4b7b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 --2- 172.21.15.175:0/2682425083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7ad250 0x557b4b7ad630 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 -- 172.21.15.175:0/2682425083 >> 172.21.15.175:0/2682425083 conn(0x557b4b7a9800 msgr2=0x557b4b7a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:15.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 -- 172.21.15.175:0/2682425083 shutdown_connections 2024-07-29T00:42:15.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.168+0000 7f9256a20740 1 -- 172.21.15.175:0/2682425083 wait complete. 2024-07-29T00:42:15.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.169+0000 7f9256a20740 1 -- start start 2024-07-29T00:42:15.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.169+0000 7f9256a20740 1 Processor -- start 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b4b7bafa0 con 0x557b4b7ad250 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557b4b6cd750 con 0x557b4b7adb70 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557b4b7bb120 con 0x557b4b7b8430 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9256a20740 10 client.0 _ll_register_callbacks cb 0x557b4b422090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9250c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9250c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42952/0 (socket says 172.21.15.175:42952) 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.170+0000 7f9250c00640 1 -- 172.21.15.175:0/3945867600 learned_addr learned my addr 172.21.15.175:0/3945867600 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9251600640 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9250c00640 1 -- 172.21.15.175:0/3945867600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 msgr2=0x557b4b7b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9250c00640 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9250c00640 1 -- 172.21.15.175:0/3945867600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 msgr2=0x557b4b7b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9250c00640 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9250c00640 1 -- 172.21.15.175:0/3945867600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4b7c4590 con 0x557b4b7adb70 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9250c00640 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7b5a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9238012d20 tx=0x7f923800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77fb93de6ba15ee server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9232a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9232a00640 1 -- 172.21.15.175:0/3945867600 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238009e30 con 0x557b4b7adb70 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.171+0000 7f9232a00640 1 -- 172.21.15.175:0/3945867600 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f9238004050 con 0x557b4b7adb70 2024-07-29T00:42:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.172+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f924401ac40 con 0x557b4b7adb70 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.172+0000 7f9232a00640 1 -- 172.21.15.175:0/3945867600 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9238004250 con 0x557b4b7adb70 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.172+0000 7f9256a20740 10 client.26558 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.172+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557b4b7c6eb0 con 0x557b4b7adb70 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9232a00640 1 -- 172.21.15.175:0/3945867600 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f924401ac40 con 0x557b4b7adb70 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9232a00640 1 -- 172.21.15.175:0/3945867600 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f9238010540 con 0x557b4b7adb70 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9256a20740 10 client.26558 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9256a20740 10 client.26558 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9256a20740 20 client.26558 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9256a20740 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b7c8440 0x557b4b7ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9256a20740 4 client.26558 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4b7201b0 con 0x557b4b7c8440 2024-07-29T00:42:15.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.173+0000 7f9251600640 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b7c8440 0x557b4b7ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.174+0000 7f9251600640 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b7c8440 0x557b4b7ca8e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec233ca14a39ade4 server_cookie=72934f0e000ac1d1 in_seq=0 out_seq=0 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.174+0000 7f9232a00640 10 client.26558 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.174+0000 7f9232a00640 1 -- 172.21.15.175:0/3945867600 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b4b7201b0 con 0x557b4b7c8440 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9232a00640 10 client.26558 handle_command_reply: tid=0 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 client.26558 shutdown 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 msgr2=0x557b4b7b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7b5a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9238012d20 tx=0x7f923800f790 comp rx=0 tx=0).stop 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 shutdown_connections 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b7c8440 0x557b4b7ca8e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9232a00640 0 client.26558 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7b5a50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 --2- 172.21.15.175:0/3945867600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 >> 172.21.15.175:0/3945867600 conn(0x557b4b7a9800 msgr2=0x557b4b7a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 shutdown_connections 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 1 -- 172.21.15.175:0/3945867600 wait complete. 2024-07-29T00:42:15.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.175+0000 7f9256a20740 20 client.26558 trim_cache size 0 max 16384 2024-07-29T00:42:15.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.198+0000 7f9256a20740 1 Processor -- start 2024-07-29T00:42:15.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 -- start start 2024-07-29T00:42:15.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1af8b0 con 0x557b4b7ad250 2024-07-29T00:42:15.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b1510 con 0x557b4b7adb70 2024-07-29T00:42:15.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9256a20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1aff80 con 0x557b4b7b8430 2024-07-29T00:42:15.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9250c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9250c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42956/0 (socket says 172.21.15.175:42956) 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9250c00640 1 -- 172.21.15.175:0/1938882806 learned_addr learned my addr 172.21.15.175:0/1938882806 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.199+0000 7f9251600640 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9252000640 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9250c00640 1 -- 172.21.15.175:0/1938882806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 msgr2=0x557b4b7d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9250c00640 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9250c00640 1 -- 172.21.15.175:0/1938882806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 msgr2=0x557b4b7a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9250c00640 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9250c00640 1 -- 172.21.15.175:0/1938882806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4b7b44c0 con 0x557b4b7adb70 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9251600640 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.200+0000 7f9250c00640 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7a8870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9238001eb0 tx=0x7f9238012910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f52694399b25691 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.201+0000 7f923dc00640 1 -- 172.21.15.175:0/1938882806 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923802b070 con 0x557b4b7adb70 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.201+0000 7f923dc00640 1 -- 172.21.15.175:0/1938882806 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9238009e30 con 0x557b4b7adb70 2024-07-29T00:42:15.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.201+0000 7f923dc00640 1 -- 172.21.15.175:0/1938882806 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238010690 con 0x557b4b7adb70 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 -- 172.21.15.175:0/1938882806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 msgr2=0x557b4b7a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7a8870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9238001eb0 tx=0x7f9238012910 comp rx=0 tx=0).stop 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 -- 172.21.15.175:0/1938882806 shutdown_connections 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7b8430 0x557b4b7d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7adb70 0x557b4b7a8870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 --2- 172.21.15.175:0/1938882806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 -- 172.21.15.175:0/1938882806 >> 172.21.15.175:0/1938882806 conn(0x557b4b7a9800 msgr2=0x557b4b7a8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 -- 172.21.15.175:0/1938882806 shutdown_connections 2024-07-29T00:42:15.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 -- 172.21.15.175:0/1938882806 wait complete. 2024-07-29T00:42:15.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.202+0000 7f9256a20740 1 -- start start 2024-07-29T00:42:15.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 Processor -- start 2024-07-29T00:42:15.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7adb70 0x557b4b7bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b82b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b0d80 con 0x557b4b7ad250 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b0650 con 0x557b4b7b8430 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ad890 con 0x557b4b7adb70 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.203+0000 7f9256a20740 10 client.0 _ll_register_callbacks cb 0x7f9238009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9251600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9251600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7bcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37956/0 (socket says 172.21.15.175:37956) 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9251600640 1 -- 172.21.15.175:0/3801017886 learned_addr learned my addr 172.21.15.175:0/3801017886 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9252000640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b82b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9250c00640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7adb70 0x557b4b7bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9252000640 1 -- 172.21.15.175:0/3801017886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7adb70 msgr2=0x557b4b7bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9252000640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7adb70 0x557b4b7bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9252000640 1 -- 172.21.15.175:0/3801017886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 msgr2=0x557b4b7bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9252000640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9252000640 1 -- 172.21.15.175:0/3801017886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4b82b9b0 con 0x557b4b7b8430 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9251600640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.204+0000 7f9250c00640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7adb70 0x557b4b7bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9252000640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b82b180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9244012e80 tx=0x7f92440122a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c2c96825612227 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9232a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9232a00640 1 -- 172.21.15.175:0/3801017886 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924401acb0 con 0x557b4b7b8430 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9232a00640 1 -- 172.21.15.175:0/3801017886 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f924401b440 con 0x557b4b7b8430 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9238009e30 con 0x557b4b7b8430 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9232a00640 1 -- 172.21.15.175:0/3801017886 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f924401ae50 con 0x557b4b7b8430 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9256a20740 10 client.26564 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.205+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557b4b833d00 con 0x557b4b7b8430 2024-07-29T00:42:15.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9232a00640 1 -- 172.21.15.175:0/3801017886 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9238009e30 con 0x557b4b7b8430 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9232a00640 1 -- 172.21.15.175:0/3801017886 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f9244026440 con 0x557b4b7b8430 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9256a20740 10 client.26564 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9256a20740 10 client.26564 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9256a20740 20 client.26564 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9256a20740 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b834330 0x557b4b836750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9256a20740 4 client.26564 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}] 2024-07-29T00:42:15.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}) -- 0x557b4b71a500 con 0x557b4b834330 2024-07-29T00:42:15.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.206+0000 7f9251600640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b834330 0x557b4b836750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:15.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.207+0000 7f9251600640 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b834330 0x557b4b836750 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cf3fe7459d5f79 server_cookie=e35b7f958ac01d10 in_seq=0 out_seq=0 2024-07-29T00:42:15.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.207+0000 7f9232a00640 10 client.26564 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:15.286 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:15.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.373+0000 7f9232a00640 1 -- 172.21.15.175:0/3801017886 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557b4b71a500 con 0x557b4b834330 2024-07-29T00:42:15.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.373+0000 7f9232a00640 10 client.26564 handle_command_reply: tid=0 2024-07-29T00:42:15.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.373+0000 7f9256a20740 1 client.26564 shutdown 2024-07-29T00:42:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 msgr2=0x557b4b82b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b82b180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9244012e80 tx=0x7f92440122a0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 shutdown_connections 2024-07-29T00:42:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557b4b834330 0x557b4b836750 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b4b7b8430 0x557b4b82b180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b4b7adb70 0x557b4b7bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9232a00640 0 client.26564 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:15.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 --2- 172.21.15.175:0/3801017886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b4b7ad250 0x557b4b7bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 >> 172.21.15.175:0/3801017886 conn(0x557b4b7a9800 msgr2=0x557b4b7a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:15.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 shutdown_connections 2024-07-29T00:42:15.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 1 -- 172.21.15.175:0/3801017886 wait complete. 2024-07-29T00:42:15.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.374+0000 7f9256a20740 20 client.26564 trim_cache size 0 max 16384 2024-07-29T00:42:15.378 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:42:15.378 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 38, 2024-07-29T00:42:15.378 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:42:15.378 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 18, 2024-07-29T00:42:15.378 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "2caef02d": { 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 18, 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 154.3, 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:42:15.379 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:42:15.380 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:42:15.380 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "QUIESCED", 2024-07-29T00:42:15.380 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:42:15.380 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:15.380 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:15.380 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:15.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:15.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:15.381 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:42:15.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92340a2cf0 msgr2=0x7f92340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92340a2cf0 0x7f92340a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9244021000 tx=0x7f924401e000 comp rx=0 tx=0).stop 2024-07-29T00:42:15.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 msgr2=0x7f924c19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:15.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c19a390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f924002a940 tx=0x7f9240003b90 comp rx=0 tx=0).stop 2024-07-29T00:42:15.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 shutdown_connections 2024-07-29T00:42:15.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f92340a2cf0 0x7f92340a51b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c109aa0 0x7f924c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c104440 0x7f924c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 --2- 172.21.15.175:0/2822450212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f924c103aa0 0x7f924c19a390 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:15.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 >> 172.21.15.175:0/2822450212 conn(0x7f924c0ff740 msgr2=0x7f924c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:15.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 shutdown_connections 2024-07-29T00:42:15.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:15.375+0000 7f9254000640 1 -- 172.21.15.175:0/2822450212 wait complete. 2024-07-29T00:42:15.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 3.13 MB/sec execute 126 sec latency 689.332 ms 2024-07-29T00:42:15.989 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:15 smithi175 ceph-mon[32138]: pgmap v175: 97 pgs: 97 active+clean; 115 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-07-29T00:42:15.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:15 smithi181 ceph-mon[36706]: pgmap v175: 97 pgs: 97 active+clean; 115 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-07-29T00:42:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:15 smithi204 ceph-mon[37368]: pgmap v175: 97 pgs: 97 active+clean; 115 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-07-29T00:42:16.473 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2caef02d', quiesce took 7.4 seconds. Will release after: 26.9 2024-07-29T00:42:16.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 -- 172.21.15.175:0/3888056289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858109a80 msgr2=0x7fd858109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 --2- 172.21.15.175:0/3888056289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858109a80 0x7fd858109e60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009860 tx=0x7fd84c02a340 comp rx=0 tx=0).stop 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 -- 172.21.15.175:0/3888056289 shutdown_connections 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 --2- 172.21.15.175:0/3888056289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd858104420 0x7fd858110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 --2- 172.21.15.175:0/3888056289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd858103a80 0x7fd858103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 --2- 172.21.15.175:0/3888056289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858109a80 0x7fd858109e60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 -- 172.21.15.175:0/3888056289 >> 172.21.15.175:0/3888056289 conn(0x7fd8580ff760 msgr2=0x7fd858101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:16.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 -- 172.21.15.175:0/3888056289 shutdown_connections 2024-07-29T00:42:16.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.675+0000 7fd860200640 1 -- 172.21.15.175:0/3888056289 wait complete. 2024-07-29T00:42:16.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.676+0000 7fd860200640 1 Processor -- start 2024-07-29T00:42:16.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.676+0000 7fd860200640 1 -- start start 2024-07-29T00:42:16.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd860200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 0x7fd85819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd860200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd858104420 0x7fd85819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd860200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd858109a80 0x7fd85819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd860200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd858113e80 con 0x7fd858109a80 2024-07-29T00:42:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd860200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd858113d00 con 0x7fd858103a80 2024-07-29T00:42:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd860200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd858114000 con 0x7fd858104420 2024-07-29T00:42:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 0x7fd85819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 0x7fd85819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:42986/0 (socket says 172.21.15.175:42986) 2024-07-29T00:42:16.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd858104420 0x7fd85819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd858109a80 0x7fd85819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 -- 172.21.15.175:0/3436690621 learned_addr learned my addr 172.21.15.175:0/3436690621 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:16.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 -- 172.21.15.175:0/3436690621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd858104420 msgr2=0x7fd85819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd858104420 0x7fd85819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 -- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd858109a80 msgr2=0x7fd85819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd858109a80 0x7fd85819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.677+0000 7fd85d800640 1 -- 172.21.15.175:0/3436690621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85819f2f0 con 0x7fd858103a80 2024-07-29T00:42:16.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.678+0000 7fd85d800640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 0x7fd85819a340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd84c035040 tx=0x7fd84c033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:16.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.678+0000 7fd856000640 1 -- 172.21.15.175:0/3436690621 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c037040 con 0x7fd858103a80 2024-07-29T00:42:16.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.678+0000 7fd856000640 1 -- 172.21.15.175:0/3436690621 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c040070 con 0x7fd858103a80 2024-07-29T00:42:16.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.678+0000 7fd856000640 1 -- 172.21.15.175:0/3436690621 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c003a10 con 0x7fd858103a80 2024-07-29T00:42:16.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.678+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85819f580 con 0x7fd858103a80 2024-07-29T00:42:16.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.678+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8581a6ec0 con 0x7fd858103a80 2024-07-29T00:42:16.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd856000640 1 -- 172.21.15.175:0/3436690621 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd84c004150 con 0x7fd858103a80 2024-07-29T00:42:16.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 Processor -- start 2024-07-29T00:42:16.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 -- start start 2024-07-29T00:42:16.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eecc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eeccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eed7720 0x55b37eed9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b37ee32ab0 con 0x55b37eecc980 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b37edcaa80 con 0x55b37eecdef0 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd862c7f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b37ee2ea70 con 0x55b37eed7720 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd85d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eeccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd85e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eecc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd85d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eeccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37980/0 (socket says 172.21.15.175:37980) 2024-07-29T00:42:16.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.682+0000 7fd85d800640 1 -- 172.21.15.175:0/1708603141 learned_addr learned my addr 172.21.15.175:0/1708603141 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:16.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eed7720 0x55b37eed9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 -- 172.21.15.175:0/1708603141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 msgr2=0x55b37eecc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eecc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 -- 172.21.15.175:0/1708603141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 msgr2=0x55b37eeccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eeccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 -- 172.21.15.175:0/1708603141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37eba43c0 con 0x55b37eed7720 2024-07-29T00:42:16.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.683+0000 7fd85ce00640 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eed7720 0x55b37eed9b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd848009530 tx=0x7fd848011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d326367096eab718 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:16.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd846a00640 1 -- 172.21.15.175:0/1708603141 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848004020 con 0x55b37eed7720 2024-07-29T00:42:16.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd846a00640 1 -- 172.21.15.175:0/1708603141 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8480041c0 con 0x55b37eed7720 2024-07-29T00:42:16.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd846a00640 1 -- 172.21.15.175:0/1708603141 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8480198b0 con 0x55b37eed7720 2024-07-29T00:42:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd856000640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8400a2cf0 0x7fd8400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd856000640 1 -- 172.21.15.175:0/3436690621 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd84c0ef350 con 0x7fd858103a80 2024-07-29T00:42:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd85e200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8400a2cf0 0x7fd8400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd862c7f740 1 -- 172.21.15.175:0/1708603141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eed7720 msgr2=0x55b37eed9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.684+0000 7fd862c7f740 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eed7720 0x55b37eed9b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd848009530 tx=0x7fd848011810 comp rx=0 tx=0).stop 2024-07-29T00:42:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 -- 172.21.15.175:0/1708603141 shutdown_connections 2024-07-29T00:42:16.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eed7720 0x55b37eed9b10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eeccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 --2- 172.21.15.175:0/1708603141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eecc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 -- 172.21.15.175:0/1708603141 >> 172.21.15.175:0/1708603141 conn(0x55b37ee95950 msgr2=0x55b37eec9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:16.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 -- 172.21.15.175:0/1708603141 shutdown_connections 2024-07-29T00:42:16.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd85e200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8400a2cf0 0x7fd8400a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd850005fd0 tx=0x7fd850009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:16.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 -- 172.21.15.175:0/1708603141 wait complete. 2024-07-29T00:42:16.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 -- start start 2024-07-29T00:42:16.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.685+0000 7fd862c7f740 1 Processor -- start 2024-07-29T00:42:16.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eed46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eed4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b37eeda4c0 con 0x55b37eed7720 2024-07-29T00:42:16.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b37ededc10 con 0x55b37eecdef0 2024-07-29T00:42:16.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b37eeda640 con 0x55b37eecc980 2024-07-29T00:42:16.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd862c7f740 10 client.0 _ll_register_callbacks cb 0x55b37edb6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:16.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd85d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eed46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd85d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eed46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45318/0 (socket says 172.21.15.175:45318) 2024-07-29T00:42:16.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd85d800640 1 -- 172.21.15.175:0/726603153 learned_addr learned my addr 172.21.15.175:0/726603153 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:16.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd85ce00640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eed4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.686+0000 7fd85e200640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd85d800640 1 -- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 msgr2=0x55b37eed4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd85d800640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eed4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd85d800640 1 -- 172.21.15.175:0/726603153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 msgr2=0x55b37eed5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd85d800640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd85d800640 1 -- 172.21.15.175:0/726603153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37edf11e0 con 0x55b37eecc980 2024-07-29T00:42:16.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd85d800640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eed46d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0c4040 tx=0x7fd84c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d091200b0d5c4a9b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:16.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd83b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:16.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd83b400640 1 -- 172.21.15.175:0/726603153 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0ee030 con 0x55b37eecc980 2024-07-29T00:42:16.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd83b400640 1 -- 172.21.15.175:0/726603153 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd84c0c40e0 con 0x55b37eecc980 2024-07-29T00:42:16.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.687+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8480041c0 con 0x55b37eecc980 2024-07-29T00:42:16.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd83b400640 1 -- 172.21.15.175:0/726603153 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd84c0bb440 con 0x55b37eecc980 2024-07-29T00:42:16.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 10 client.26533 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:16.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b37eeda520 con 0x55b37eecc980 2024-07-29T00:42:16.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd83b400640 1 -- 172.21.15.175:0/726603153 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8480041c0 con 0x55b37eecc980 2024-07-29T00:42:16.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd83b400640 1 -- 172.21.15.175:0/726603153 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fd84c03a070 con 0x55b37eecc980 2024-07-29T00:42:16.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 10 client.26533 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:16.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 10 client.26533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:42:16.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 20 client.26533 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:16.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37eee7640 0x55b37eee9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 4 client.26533 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:16.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.688+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b37ee40020 con 0x55b37eee7640 2024-07-29T00:42:16.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.689+0000 7fd85ce00640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37eee7640 0x55b37eee9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.689+0000 7fd85ce00640 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37eee7640 0x55b37eee9ac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f3745632e7e1610 server_cookie=ab8604caa933bf2a in_seq=0 out_seq=0 2024-07-29T00:42:16.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.689+0000 7fd83b400640 10 client.26533 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:42:16.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd83b400640 1 -- 172.21.15.175:0/726603153 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b37ee40020 con 0x55b37eee7640 2024-07-29T00:42:16.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd83b400640 10 client.26533 handle_command_reply: tid=0 2024-07-29T00:42:16.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 client.26533 shutdown 2024-07-29T00:42:16.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 msgr2=0x55b37eed46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eed46d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0c4040 tx=0x7fd84c005bd0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 shutdown_connections 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37eee7640 0x55b37eee9ac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eed4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 --2- 172.21.15.175:0/726603153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eed46d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd83b400640 0 client.26533 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 >> 172.21.15.175:0/726603153 conn(0x55b37ee95950 msgr2=0x55b37eecece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 shutdown_connections 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.690+0000 7fd862c7f740 1 -- 172.21.15.175:0/726603153 wait complete. 2024-07-29T00:42:16.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.691+0000 7fd862c7f740 20 client.26533 trim_cache size 0 max 16384 2024-07-29T00:42:16.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 Processor -- start 2024-07-29T00:42:16.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 -- start start 2024-07-29T00:42:16.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eed3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecdef0 0x55b37eed4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eed7720 0x55b37eeef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581af8c0 con 0x55b37eecc980 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b1520 con 0x55b37eed7720 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd862c7f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581aff90 con 0x55b37eecdef0 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd85d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eed3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd85d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eed3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37992/0 (socket says 172.21.15.175:37992) 2024-07-29T00:42:16.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.706+0000 7fd85d800640 1 -- 172.21.15.175:0/3715241469 learned_addr learned my addr 172.21.15.175:0/3715241469 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85e200640 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eed7720 0x55b37eeef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85ce00640 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecdef0 0x55b37eed4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85d800640 1 -- 172.21.15.175:0/3715241469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecdef0 msgr2=0x55b37eed4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85d800640 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecdef0 0x55b37eed4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85d800640 1 -- 172.21.15.175:0/3715241469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eed7720 msgr2=0x55b37eeef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85d800640 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eed7720 0x55b37eeef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85d800640 1 -- 172.21.15.175:0/3715241469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37eecc680 con 0x55b37eecc980 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85e200640 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eed7720 0x55b37eeef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.707+0000 7fd85d800640 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eed3630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0bb890 tx=0x7fd84c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260515561763d728 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd846000640 1 -- 172.21.15.175:0/3715241469 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0bb260 con 0x55b37eecc980 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd846000640 1 -- 172.21.15.175:0/3715241469 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c0a62f0 con 0x55b37eecc980 2024-07-29T00:42:16.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd846000640 1 -- 172.21.15.175:0/3715241469 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0bafb0 con 0x55b37eecc980 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 -- 172.21.15.175:0/3715241469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 msgr2=0x55b37eed3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eed3630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0bb890 tx=0x7fd84c005f90 comp rx=0 tx=0).stop 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 -- 172.21.15.175:0/3715241469 shutdown_connections 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eed7720 0x55b37eeef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecdef0 0x55b37eed4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 --2- 172.21.15.175:0/3715241469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eecc980 0x55b37eed3630 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.708+0000 7fd862c7f740 1 -- 172.21.15.175:0/3715241469 >> 172.21.15.175:0/3715241469 conn(0x55b37ee95950 msgr2=0x55b37eed4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.709+0000 7fd862c7f740 1 -- 172.21.15.175:0/3715241469 shutdown_connections 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.709+0000 7fd862c7f740 1 -- 172.21.15.175:0/3715241469 wait complete. 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.709+0000 7fd862c7f740 1 -- start start 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.709+0000 7fd862c7f740 1 Processor -- start 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eedfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eee0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b0d90 con 0x55b37eed7720 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581b0660 con 0x55b37eecdef0 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581ad8a0 con 0x55b37eecc980 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd862c7f740 10 client.0 _ll_register_callbacks cb 0x7fd84c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd85e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd85d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eedfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd85e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37998/0 (socket says 172.21.15.175:37998) 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd85e200640 1 -- 172.21.15.175:0/2563391670 learned_addr learned my addr 172.21.15.175:0/2563391670 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.710+0000 7fd85ce00640 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eee0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd85d800640 1 -- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 msgr2=0x55b37eee0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd85d800640 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eee0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd85d800640 1 -- 172.21.15.175:0/2563391670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 msgr2=0x55b37eed10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:16.724 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd85d800640 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd85d800640 1 -- 172.21.15.175:0/2563391670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37eed18e0 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd85d800640 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eedfbf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0ecb90 tx=0x7fd84c0c46f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916dadb42999b2bf server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd83b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd83b400640 1 -- 172.21.15.175:0/2563391670 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0ee070 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd83b400640 1 -- 172.21.15.175:0/2563391670 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fd84c0b7650 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd84c0a62f0 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd83b400640 1 -- 172.21.15.175:0/2563391670 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd84c0bb260 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.711+0000 7fd862c7f740 10 client.26539 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b37eed18e0 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd83b400640 1 -- 172.21.15.175:0/2563391670 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c0a62f0 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd83b400640 1 -- 172.21.15.175:0/2563391670 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fd84c03a070 con 0x55b37eecc980 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 10 client.26539 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:16.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 10 client.26539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 20 client.26539 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37ef521d0 0x55b37ef545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 4 client.26539 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b37ee3a530 con 0x55b37ef521d0 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.712+0000 7fd85ce00640 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37ef521d0 0x55b37ef545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd85ce00640 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37ef521d0 0x55b37ef545c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1c9b5bd5c51e8f5 server_cookie=510cf5dd8f90c443 in_seq=0 out_seq=0 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd83b400640 10 client.26539 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd83b400640 1 -- 172.21.15.175:0/2563391670 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b37ee3a530 con 0x55b37ef521d0 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd83b400640 10 client.26539 handle_command_reply: tid=0 2024-07-29T00:42:16.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd862c7f740 1 client.26539 shutdown 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 msgr2=0x55b37eedfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.713+0000 7fd862c7f740 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eedfbf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0ecb90 tx=0x7fd84c0c46f0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 shutdown_connections 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b37eed7720 0x55b37eed10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55b37ef521d0 0x55b37ef545c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b37eecdef0 0x55b37eee0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 --2- 172.21.15.175:0/2563391670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b37eecc980 0x55b37eedfbf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 >> 172.21.15.175:0/2563391670 conn(0x55b37ee95950 msgr2=0x55b37eed4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd83b400640 0 client.26539 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:42:16.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 shutdown_connections 2024-07-29T00:42:16.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 1 -- 172.21.15.175:0/2563391670 wait complete. 2024-07-29T00:42:16.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.714+0000 7fd862c7f740 20 client.26539 trim_cache size 0 max 16384 2024-07-29T00:42:16.728 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:42:16.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8400a2cf0 msgr2=0x7fd8400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8400a2cf0 0x7fd8400a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd850005fd0 tx=0x7fd850009500 comp rx=0 tx=0).stop 2024-07-29T00:42:16.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 msgr2=0x7fd85819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:16.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 0x7fd85819a340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd84c035040 tx=0x7fd84c033b60 comp rx=0 tx=0).stop 2024-07-29T00:42:16.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 shutdown_connections 2024-07-29T00:42:16.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8400a2cf0 0x7fd8400a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd858109a80 0x7fd85819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd858104420 0x7fd85819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 --2- 172.21.15.175:0/3436690621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd858103a80 0x7fd85819a340 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:16.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 >> 172.21.15.175:0/3436690621 conn(0x7fd8580ff760 msgr2=0x7fd8580fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:16.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 shutdown_connections 2024-07-29T00:42:16.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:16.715+0000 7fd860200640 1 -- 172.21.15.175:0/3436690621 wait complete. 2024-07-29T00:42:16.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 3.11 MB/sec execute 127 sec latency 1689.421 ms 2024-07-29T00:42:17.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:42:17.853 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:17 smithi175 ceph-mon[32138]: pgmap v176: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-07-29T00:42:17.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 3.08 MB/sec execute 128 sec latency 2689.489 ms 2024-07-29T00:42:17.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:17 smithi181 ceph-mon[36706]: pgmap v176: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-07-29T00:42:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:17 smithi204 ceph-mon[37368]: pgmap v176: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-07-29T00:42:18.128 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:18.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.796+0000 7f62c2a00640 1 -- 172.21.15.175:0/2753722757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc10bce0 msgr2=0x7f62bc10e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.796+0000 7f62c2a00640 1 --2- 172.21.15.175:0/2753722757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc10bce0 0x7f62bc10e150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009ca0 tx=0x7f62b402a950 comp rx=0 tx=0).stop 2024-07-29T00:42:18.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 -- 172.21.15.175:0/2753722757 shutdown_connections 2024-07-29T00:42:18.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 --2- 172.21.15.175:0/2753722757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc10bce0 0x7f62bc10e150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 --2- 172.21.15.175:0/2753722757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc103260 0x7f62bc10b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 --2- 172.21.15.175:0/2753722757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 0x7f62bc102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 -- 172.21.15.175:0/2753722757 >> 172.21.15.175:0/2753722757 conn(0x7f62bc0fc300 msgr2=0x7f62bc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:18.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 -- 172.21.15.175:0/2753722757 shutdown_connections 2024-07-29T00:42:18.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.797+0000 7f62c2a00640 1 -- 172.21.15.175:0/2753722757 wait complete. 2024-07-29T00:42:18.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 Processor -- start 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 -- start start 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 0x7f62bc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc103260 0x7f62bc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc10bce0 0x7f62bc19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc111b00 con 0x7f62bc103260 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc111980 con 0x7f62bc10bce0 2024-07-29T00:42:18.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.798+0000 7f62c2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc111c80 con 0x7f62bc102940 2024-07-29T00:42:18.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc103260 0x7f62bc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc103260 0x7f62bc1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40252/0 (socket says 172.21.15.175:40252) 2024-07-29T00:42:18.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c0c00640 1 -- 172.21.15.175:0/1132946979 learned_addr learned my addr 172.21.15.175:0/1132946979 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:18.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 0x7f62bc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c2000640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc10bce0 0x7f62bc19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c1600640 1 -- 172.21.15.175:0/1132946979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc10bce0 msgr2=0x7f62bc19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c1600640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc10bce0 0x7f62bc19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c1600640 1 -- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc103260 msgr2=0x7f62bc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c1600640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc103260 0x7f62bc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.799+0000 7f62c1600640 1 -- 172.21.15.175:0/1132946979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc19cf10 con 0x7f62bc102940 2024-07-29T00:42:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.800+0000 7f62c1600640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 0x7f62bc197f60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62b000e9f0 tx=0x7f62b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.800+0000 7f62ba000640 1 -- 172.21.15.175:0/1132946979 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000ccc0 con 0x7f62bc102940 2024-07-29T00:42:18.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.800+0000 7f62ba000640 1 -- 172.21.15.175:0/1132946979 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b0010910 con 0x7f62bc102940 2024-07-29T00:42:18.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.800+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc19d200 con 0x7f62bc102940 2024-07-29T00:42:18.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.800+0000 7f62ba000640 1 -- 172.21.15.175:0/1132946979 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0018960 con 0x7f62bc102940 2024-07-29T00:42:18.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.800+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc1a4ae0 con 0x7f62bc102940 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.803+0000 7f62ba000640 1 -- 172.21.15.175:0/1132946979 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f62b0010ac0 con 0x7f62bc102940 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.803+0000 7f62c6a86740 1 Processor -- start 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 -- start start 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c41630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c41fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561054ba7ab0 con 0x561054c41b70 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561054b3fa80 con 0x561054c4c910 2024-07-29T00:42:18.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c6a86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561054ba3a70 con 0x561054c430e0 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c41fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c41630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c41fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40254/0 (socket says 172.21.15.175:40254) 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c1600640 1 -- 172.21.15.175:0/1569076169 learned_addr learned my addr 172.21.15.175:0/1569076169 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.804+0000 7f62c2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c41630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46834/0 (socket says 172.21.15.175:46834) 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c0c00640 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c2000640 1 -- 172.21.15.175:0/1569076169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 msgr2=0x561054c4ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c2000640 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c2000640 1 -- 172.21.15.175:0/1569076169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 msgr2=0x561054c41fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c2000640 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c41fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c2000640 1 -- 172.21.15.175:0/1569076169 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610549193c0 con 0x561054c430e0 2024-07-29T00:42:18.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c1600640 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c41fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62ba000640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f62a40a2d70 0x7f62a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62ba000640 1 -- 172.21.15.175:0/1132946979 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f62b00cf7c0 con 0x7f62bc102940 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c2000640 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c41630 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005e30 tx=0x7f62b4005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac2e79c08dbf70f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.805+0000 7f62c1600640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f62a40a2d70 0x7f62a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.806+0000 7f62ae600640 1 -- 172.21.15.175:0/1569076169 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4048070 con 0x561054c430e0 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.806+0000 7f62ae600640 1 -- 172.21.15.175:0/1569076169 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b4003e50 con 0x561054c430e0 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.806+0000 7f62ae600640 1 -- 172.21.15.175:0/1569076169 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4034040 con 0x561054c430e0 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.806+0000 7f62c6a86740 1 -- 172.21.15.175:0/1569076169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 msgr2=0x561054c41630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.806+0000 7f62c1600640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f62a40a2d70 0x7f62a40a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f62b000c6a0 tx=0x7f62b00d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.806+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c41630 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005e30 tx=0x7f62b4005f80 comp rx=0 tx=0).stop 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 -- 172.21.15.175:0/1569076169 shutdown_connections 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c41fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1569076169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c41630 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 -- 172.21.15.175:0/1569076169 >> 172.21.15.175:0/1569076169 conn(0x561054c0a950 msgr2=0x561054c3e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 -- 172.21.15.175:0/1569076169 shutdown_connections 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 -- 172.21.15.175:0/1569076169 wait complete. 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 -- start start 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.807+0000 7f62c6a86740 1 Processor -- start 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c49860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c49da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561054c4f6a0 con 0x561054c41b70 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561054b62c10 con 0x561054c4c910 2024-07-29T00:42:18.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561054c4f820 con 0x561054c430e0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c6a86740 10 client.0 _ll_register_callbacks cb 0x561054b2ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c49860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c49da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c49da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46836/0 (socket says 172.21.15.175:46836) 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.808+0000 7f62c0c00640 1 -- 172.21.15.175:0/1914940332 learned_addr learned my addr 172.21.15.175:0/1914940332 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c2000640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c0c00640 1 -- 172.21.15.175:0/1914940332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 msgr2=0x561054c4a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c0c00640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c0c00640 1 -- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 msgr2=0x561054c49860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c0c00640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c49860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c0c00640 1 -- 172.21.15.175:0/1914940332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561054b661e0 con 0x561054c430e0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c2000640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c0c00640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c49da0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62a80127d0 tx=0x7f62a8012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eba39472d258cef server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:18.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62a2a00640 1 -- 172.21.15.175:0/1914940332 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8004020 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62a2a00640 1 -- 172.21.15.175:0/1914940332 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f62a80041c0 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.809+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b4003e50 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.810+0000 7f62a2a00640 1 -- 172.21.15.175:0/1914940332 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f62a80043c0 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.810+0000 7f62c6a86740 10 client.26551 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.810+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561054c5f8b0 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.810+0000 7f62a2a00640 1 -- 172.21.15.175:0/1914940332 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b4003e50 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62a2a00640 1 -- 172.21.15.175:0/1914940332 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f62a8003b30 con 0x561054c430e0 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c6a86740 10 client.26551 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c6a86740 10 client.26551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c6a86740 20 client.26551 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054c60dc0 0x561054c63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c6a86740 4 client.26551 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:18.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561054bb5020 con 0x561054c60dc0 2024-07-29T00:42:18.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c2000640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054c60dc0 0x561054c63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62a2a00640 10 client.26551 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:42:18.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.811+0000 7f62c2000640 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054c60dc0 0x561054c63210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=164ed9a04c28626e server_cookie=de263c4ac7eb430a in_seq=0 out_seq=0 2024-07-29T00:42:18.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.812+0000 7f62a2a00640 1 -- 172.21.15.175:0/1914940332 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561054bb5020 con 0x561054c60dc0 2024-07-29T00:42:18.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.812+0000 7f62a2a00640 10 client.26551 handle_command_reply: tid=0 2024-07-29T00:42:18.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.812+0000 7f62c6a86740 1 client.26551 shutdown 2024-07-29T00:42:18.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 msgr2=0x561054c49da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c49da0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62a80127d0 tx=0x7f62a8012c90 comp rx=0 tx=0).stop 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 shutdown_connections 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054c60dc0 0x561054c63210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054c4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c430e0 0x561054c49da0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 --2- 172.21.15.175:0/1914940332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c41b70 0x561054c49860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 >> 172.21.15.175:0/1914940332 conn(0x561054c0a950 msgr2=0x561054c43f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62a2a00640 0 client.26551 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 shutdown_connections 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 1 -- 172.21.15.175:0/1914940332 wait complete. 2024-07-29T00:42:18.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.813+0000 7f62c6a86740 20 client.26551 trim_cache size 0 max 16384 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 Processor -- start 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 -- start start 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c4b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 0x561054c68b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c4c910 0x561054c69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1ab250 con 0x561054c4c910 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1aceb0 con 0x561054c430e0 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.828+0000 7f62c6a86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1ab920 con 0x561054c41b70 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 0x561054c68b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c4b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 0x561054c68b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58688/0 (socket says 172.21.15.175:58688) 2024-07-29T00:42:18.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 -- 172.21.15.175:0/2211995833 learned_addr learned my addr 172.21.15.175:0/2211995833 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:18.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c2000640 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c4b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46838/0 (socket says 172.21.15.175:46838) 2024-07-29T00:42:18.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c2000640 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c4c910 0x561054c69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 -- 172.21.15.175:0/2211995833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 msgr2=0x561054c4b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c4b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 -- 172.21.15.175:0/2211995833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c4c910 msgr2=0x561054c69060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c4c910 0x561054c69060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 -- 172.21.15.175:0/2211995833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561054c47db0 con 0x561054c430e0 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.829+0000 7f62c0c00640 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 0x561054c68b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005860 tx=0x7f62a8005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccbc279176a6470e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.830+0000 7f62adc00640 1 -- 172.21.15.175:0/2211995833 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8029030 con 0x561054c430e0 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.830+0000 7f62adc00640 1 -- 172.21.15.175:0/2211995833 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004020 con 0x561054c430e0 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.830+0000 7f62adc00640 1 -- 172.21.15.175:0/2211995833 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a800f920 con 0x561054c430e0 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.830+0000 7f62c6a86740 1 -- 172.21.15.175:0/2211995833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 msgr2=0x561054c68b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.830+0000 7f62c6a86740 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 0x561054c68b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005860 tx=0x7f62a8005db0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 -- 172.21.15.175:0/2211995833 shutdown_connections 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c4c910 0x561054c69060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c430e0 0x561054c68b20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 --2- 172.21.15.175:0/2211995833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c4b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 -- 172.21.15.175:0/2211995833 >> 172.21.15.175:0/2211995833 conn(0x561054c0a950 msgr2=0x561054c4b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 -- 172.21.15.175:0/2211995833 shutdown_connections 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 -- 172.21.15.175:0/2211995833 wait complete. 2024-07-29T00:42:18.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 -- start start 2024-07-29T00:42:18.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.831+0000 7f62c6a86740 1 Processor -- start 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 0x561054c54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054cc3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1ac720 con 0x561054c430e0 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1abff0 con 0x561054c4c910 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1a9230 con 0x561054c41b70 2024-07-29T00:42:18.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c6a86740 10 client.0 _ll_register_callbacks cb 0x7f62a8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 0x561054c54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 0x561054c54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40298/0 (socket says 172.21.15.175:40298) 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c0c00640 1 -- 172.21.15.175:0/3862890710 learned_addr learned my addr 172.21.15.175:0/3862890710 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c2000640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.832+0000 7f62c2000640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054cc3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c0c00640 1 -- 172.21.15.175:0/3862890710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 msgr2=0x561054c546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c0c00640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c0c00640 1 -- 172.21.15.175:0/3862890710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 msgr2=0x561054cc3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c0c00640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054cc3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c0c00640 1 -- 172.21.15.175:0/3862890710 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561054cc4110 con 0x561054c430e0 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c0c00640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 0x561054c54c10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f62a801c960 tx=0x7f62a801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567d8704bc61f5c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:18.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62a2a00640 1 -- 172.21.15.175:0/3862890710 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8029070 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a8004020 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62a2a00640 1 -- 172.21.15.175:0/3862890710 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f62a800f920 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62a2a00640 1 -- 172.21.15.175:0/3862890710 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f62a803c050 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.833+0000 7f62c6a86740 10 client.16461 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561054ccb490 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62a2a00640 1 -- 172.21.15.175:0/3862890710 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004020 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62a2a00640 1 -- 172.21.15.175:0/3862890710 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f62a80037b0 con 0x561054c430e0 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 10 client.16461 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 10 client.16461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:42:18.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 20 client.16461 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054ccbb80 0x561054ccdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 4 client.16461 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561054c635a0 con 0x561054ccbb80 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.834+0000 7f62c2000640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054ccbb80 0x561054ccdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62c2000640 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054ccbb80 0x561054ccdfa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a7a4ac77b9e7373 server_cookie=43fa53b13f31107d in_seq=0 out_seq=0 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62a2a00640 10 client.16461 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62a2a00640 1 -- 172.21.15.175:0/3862890710 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561054c635a0 con 0x561054ccbb80 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62a2a00640 10 client.16461 handle_command_reply: tid=0 2024-07-29T00:42:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62c6a86740 1 client.16461 shutdown 2024-07-29T00:42:18.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 msgr2=0x561054c54c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.835+0000 7f62c6a86740 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 0x561054c54c10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f62a801c960 tx=0x7f62a801cbc0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 shutdown_connections 2024-07-29T00:42:18.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x561054ccbb80 0x561054ccdfa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561054c4c910 0x561054cc3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561054c430e0 0x561054c54c10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 --2- 172.21.15.175:0/3862890710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561054c41b70 0x561054c546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 >> 172.21.15.175:0/3862890710 conn(0x561054c0a950 msgr2=0x561054c5c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62a2a00640 0 client.16461 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 shutdown_connections 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 1 -- 172.21.15.175:0/3862890710 wait complete. 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.836+0000 7f62c6a86740 20 client.16461 trim_cache size 0 max 16384 2024-07-29T00:42:18.840 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f62a40a2d70 msgr2=0x7f62a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f62a40a2d70 0x7f62a40a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f62b000c6a0 tx=0x7f62b00d5530 comp rx=0 tx=0).stop 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 msgr2=0x7f62bc197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 0x7f62bc197f60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62b000e9f0 tx=0x7f62b000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 shutdown_connections 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc10bce0 0x7f62bc19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f62a40a2d70 0x7f62a40a5230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62bc103260 0x7f62bc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 --2- 172.21.15.175:0/1132946979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62bc102940 0x7f62bc197f60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:18.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 >> 172.21.15.175:0/1132946979 conn(0x7f62bc0fc300 msgr2=0x7f62bc0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:18.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 shutdown_connections 2024-07-29T00:42:18.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:18.837+0000 7f62c2a00640 1 -- 172.21.15.175:0/1132946979 wait complete. 2024-07-29T00:42:18.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 3.06 MB/sec execute 129 sec latency 3689.531 ms 2024-07-29T00:42:19.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:42:19.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 3.04 MB/sec execute 130 sec latency 4689.604 ms 2024-07-29T00:42:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:19 smithi181 ceph-mon[36706]: pgmap v177: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:19 smithi175 ceph-mon[32138]: pgmap v177: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:19 smithi204 ceph-mon[37368]: pgmap v177: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:20.198 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:20.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.874+0000 7facd2400640 1 -- 172.21.15.175:0/1653027070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc10dd50 msgr2=0x7faccc10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.874+0000 7facd2400640 1 --2- 172.21.15.175:0/1653027070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc10dd50 0x7faccc10e130 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7facbc009cd0 tx=0x7facbc02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.874+0000 7facd2400640 1 -- 172.21.15.175:0/1653027070 shutdown_connections 2024-07-29T00:42:20.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.874+0000 7facd2400640 1 --2- 172.21.15.175:0/1653027070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc108670 0x7faccc114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.874+0000 7facd2400640 1 --2- 172.21.15.175:0/1653027070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc107d10 0x7faccc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.875+0000 7facd2400640 1 --2- 172.21.15.175:0/1653027070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc10dd50 0x7faccc10e130 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.875+0000 7facd2400640 1 -- 172.21.15.175:0/1653027070 >> 172.21.15.175:0/1653027070 conn(0x7faccc077670 msgr2=0x7faccc079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:20.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.875+0000 7facd2400640 1 -- 172.21.15.175:0/1653027070 shutdown_connections 2024-07-29T00:42:20.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.875+0000 7facd2400640 1 -- 172.21.15.175:0/1653027070 wait complete. 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 Processor -- start 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 -- start start 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc107d10 0x7faccc19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 0x7faccc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc10dd50 0x7faccc1a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1180b0 con 0x7faccc107d10 2024-07-29T00:42:20.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faccc117f30 con 0x7faccc10dd50 2024-07-29T00:42:20.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.876+0000 7facd2400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faccc118230 con 0x7faccc108670 2024-07-29T00:42:20.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 0x7faccc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7facd1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc107d10 0x7faccc19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 0x7faccc19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46856/0 (socket says 172.21.15.175:46856) 2024-07-29T00:42:20.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 -- 172.21.15.175:0/1197535195 learned_addr learned my addr 172.21.15.175:0/1197535195 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:20.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7facd1a00640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc10dd50 0x7faccc1a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 -- 172.21.15.175:0/1197535195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc10dd50 msgr2=0x7faccc1a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc10dd50 0x7faccc1a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 -- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc107d10 msgr2=0x7faccc19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc107d10 0x7faccc19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.877+0000 7faccbe00640 1 -- 172.21.15.175:0/1197535195 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc1a3520 con 0x7faccc108670 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.878+0000 7faccbe00640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 0x7faccc19eab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7facb400d950 tx=0x7facb400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.878+0000 7facc9600640 1 -- 172.21.15.175:0/1197535195 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4017030 con 0x7faccc108670 2024-07-29T00:42:20.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.878+0000 7facc9600640 1 -- 172.21.15.175:0/1197535195 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4004020 con 0x7faccc108670 2024-07-29T00:42:20.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.878+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc1a3810 con 0x7faccc108670 2024-07-29T00:42:20.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.878+0000 7facc9600640 1 -- 172.21.15.175:0/1197535195 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb40047a0 con 0x7faccc108670 2024-07-29T00:42:20.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.878+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc1ab0f0 con 0x7faccc108670 2024-07-29T00:42:20.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.881+0000 7facc9600640 1 -- 172.21.15.175:0/1197535195 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7facb40041d0 con 0x7faccc108670 2024-07-29T00:42:20.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.881+0000 7facd644d740 1 Processor -- start 2024-07-29T00:42:20.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.881+0000 7facd644d740 1 -- start start 2024-07-29T00:42:20.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd644d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e97153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd644d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e97153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e97160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557e970b9ab0 con 0x557e9715e910 2024-07-29T00:42:20.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd644d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557e97051a80 con 0x557e971550e0 2024-07-29T00:42:20.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd644d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557e970b5a70 con 0x557e97153b70 2024-07-29T00:42:20.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd1a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e97153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd1a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e97153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58736/0 (socket says 172.21.15.175:58736) 2024-07-29T00:42:20.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd1a00640 1 -- 172.21.15.175:0/1970056738 learned_addr learned my addr 172.21.15.175:0/1970056738 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:20.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7facd1000640 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e97153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.882+0000 7faccbe00640 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e97160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1a00640 1 -- 172.21.15.175:0/1970056738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 msgr2=0x557e97153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1a00640 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e97153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1a00640 1 -- 172.21.15.175:0/1970056738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 msgr2=0x557e97160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1a00640 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e97160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1a00640 1 -- 172.21.15.175:0/1970056738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e96e2b3c0 con 0x557e971550e0 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1a00640 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e97153630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facc400b680 tx=0x7facc4004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293de0d515f0aa2c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facc9600640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7facb00a2cf0 0x7facb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facc9600640 1 -- 172.21.15.175:0/1197535195 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7facb40cee20 con 0x7faccc108670 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facc2000640 1 -- 172.21.15.175:0/1970056738 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4003af0 con 0x557e971550e0 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facd1000640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7facb00a2cf0 0x7facb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facc2000640 1 -- 172.21.15.175:0/1970056738 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc4015070 con 0x557e971550e0 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.883+0000 7facc2000640 1 -- 172.21.15.175:0/1970056738 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4004d10 con 0x557e971550e0 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 -- 172.21.15.175:0/1970056738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 msgr2=0x557e97153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e97153630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facc400b680 tx=0x7facc4004610 comp rx=0 tx=0).stop 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd1000640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7facb00a2cf0 0x7facb00a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7facbc009ca0 tx=0x7facbc005ef0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 -- 172.21.15.175:0/1970056738 shutdown_connections 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e97160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e97153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 --2- 172.21.15.175:0/1970056738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e97153630 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 -- 172.21.15.175:0/1970056738 >> 172.21.15.175:0/1970056738 conn(0x557e9711c950 msgr2=0x557e97150200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.884+0000 7facd644d740 1 -- 172.21.15.175:0/1970056738 shutdown_connections 2024-07-29T00:42:20.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.885+0000 7facd644d740 1 -- 172.21.15.175:0/1970056738 wait complete. 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.885+0000 7facd644d740 1 -- start start 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.885+0000 7facd644d740 1 Processor -- start 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e9715b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e9715be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e9715c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557e971616b0 con 0x557e9715e910 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557e97074c10 con 0x557e971550e0 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557e97161830 con 0x557e97153b70 2024-07-29T00:42:20.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd644d740 10 client.0 _ll_register_callbacks cb 0x557e9703da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e9715c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e9715c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40324/0 (socket says 172.21.15.175:40324) 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd1a00640 1 -- 172.21.15.175:0/2130669266 learned_addr learned my addr 172.21.15.175:0/2130669266 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7faccbe00640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e9715be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.886+0000 7facd1000640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e9715b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1a00640 1 -- 172.21.15.175:0/2130669266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 msgr2=0x557e9715b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1a00640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e9715b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1a00640 1 -- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 msgr2=0x557e9715be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1a00640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e9715be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1a00640 1 -- 172.21.15.175:0/2130669266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e970781e0 con 0x557e9715e910 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1000640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e9715b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7faccbe00640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e9715be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd1a00640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e9715c370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7facc400c010 tx=0x7facc4003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd9a1be12afac14 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:20.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facbaa00640 1 -- 172.21.15.175:0/2130669266 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4028060 con 0x557e9715e910 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facbaa00640 1 -- 172.21.15.175:0/2130669266 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7facc4011440 con 0x557e9715e910 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.887+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facc4015070 con 0x557e9715e910 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facbaa00640 1 -- 172.21.15.175:0/2130669266 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7facc4011640 con 0x557e9715e910 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facd644d740 10 client.16473 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557e9716d480 con 0x557e9715e910 2024-07-29T00:42:20.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facbaa00640 1 -- 172.21.15.175:0/2130669266 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc4015070 con 0x557e9715e910 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facbaa00640 1 -- 172.21.15.175:0/2130669266 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7facc4022070 con 0x557e9715e910 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facd644d740 10 client.16473 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facd644d740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facd644d740 20 client.16473 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.888+0000 7facd644d740 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e9716e880 0x557e97170c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.889+0000 7facd644d740 4 client.16473 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.889+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e970c7020 con 0x557e9716e880 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.889+0000 7facd1000640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e9716e880 0x557e97170c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.890+0000 7facd1000640 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e9716e880 0x557e97170c70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7763794d69d57b9c server_cookie=6e7a8478a135ade6 in_seq=0 out_seq=0 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.890+0000 7facbaa00640 10 client.16473 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.890+0000 7facbaa00640 1 -- 172.21.15.175:0/2130669266 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557e970c7020 con 0x557e9716e880 2024-07-29T00:42:20.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.890+0000 7facbaa00640 10 client.16473 handle_command_reply: tid=0 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 client.16473 shutdown 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 msgr2=0x557e9715c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e9715c370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7facc400c010 tx=0x7facc4003b20 comp rx=0 tx=0).stop 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 shutdown_connections 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e9716e880 0x557e97170c70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e9715e910 0x557e9715c370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e9715be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 --2- 172.21.15.175:0/2130669266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e97153b70 0x557e9715b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 >> 172.21.15.175:0/2130669266 conn(0x557e9711c950 msgr2=0x557e97155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facbaa00640 0 client.16473 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 shutdown_connections 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 1 -- 172.21.15.175:0/2130669266 wait complete. 2024-07-29T00:42:20.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.891+0000 7facd644d740 20 client.16473 trim_cache size 0 max 16384 2024-07-29T00:42:20.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 3.01 MB/sec execute 131 sec latency 5689.673 ms 2024-07-29T00:42:20.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 Processor -- start 2024-07-29T00:42:20.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 -- start start 2024-07-29T00:42:20.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e97153b70 0x557e9715d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e971762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e97176810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1b3bc0 con 0x557e97153b70 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1b5820 con 0x557e971550e0 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.907+0000 7facd644d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1b4290 con 0x557e9715e910 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7facd1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e97153b70 0x557e9715d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7faccbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e971762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7facd1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e97153b70 0x557e9715d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40328/0 (socket says 172.21.15.175:40328) 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7facd1000640 1 -- 172.21.15.175:0/3684597282 learned_addr learned my addr 172.21.15.175:0/3684597282 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7facd1a00640 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e97176810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7faccbe00640 1 -- 172.21.15.175:0/3684597282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 msgr2=0x557e97176810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7faccbe00640 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e97176810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7faccbe00640 1 -- 172.21.15.175:0/3684597282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e97153b70 msgr2=0x557e9715d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7faccbe00640 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e97153b70 0x557e9715d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7faccbe00640 1 -- 172.21.15.175:0/3684597282 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e97159e40 con 0x557e971550e0 2024-07-29T00:42:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.908+0000 7facd1a00640 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e97176810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:20.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.909+0000 7faccbe00640 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e971762d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facb40cc4f0 tx=0x7facb4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5c1e358a165bee4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:20.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.909+0000 7facc1600640 1 -- 172.21.15.175:0/3684597282 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb409ae00 con 0x557e971550e0 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.909+0000 7facc1600640 1 -- 172.21.15.175:0/3684597282 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb409b9c0 con 0x557e971550e0 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.909+0000 7facc1600640 1 -- 172.21.15.175:0/3684597282 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb409ac20 con 0x557e971550e0 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 -- 172.21.15.175:0/3684597282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 msgr2=0x557e971762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e971762d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facb40cc4f0 tx=0x7facb4000bc0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 -- 172.21.15.175:0/3684597282 shutdown_connections 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e97176810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e971550e0 0x557e971762d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 --2- 172.21.15.175:0/3684597282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e97153b70 0x557e9715d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 -- 172.21.15.175:0/3684597282 >> 172.21.15.175:0/3684597282 conn(0x557e9711c950 msgr2=0x557e9715d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 -- 172.21.15.175:0/3684597282 shutdown_connections 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.910+0000 7facd644d740 1 -- 172.21.15.175:0/3684597282 wait complete. 2024-07-29T00:42:20.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 -- start start 2024-07-29T00:42:20.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 Processor -- start 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 0x557e97162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e971550e0 0x557e97162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e971d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1b5090 con 0x557e971550e0 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1b4960 con 0x557e97153b70 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1b1ba0 con 0x557e9715e910 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.911+0000 7facd644d740 10 client.0 _ll_register_callbacks cb 0x7facbc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 0x557e97162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7faccbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e971550e0 0x557e97162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 0x557e97162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58772/0 (socket says 172.21.15.175:58772) 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 -- 172.21.15.175:0/815423445 learned_addr learned my addr 172.21.15.175:0/815423445 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1a00640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e971d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 -- 172.21.15.175:0/815423445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 msgr2=0x557e971d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e971d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 -- 172.21.15.175:0/815423445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e971550e0 msgr2=0x557e97162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:20.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e971550e0 0x557e97162560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 -- 172.21.15.175:0/815423445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e971d78a0 con 0x557e97153b70 2024-07-29T00:42:20.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7faccbe00640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e971550e0 0x557e97162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:20.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facd1000640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 0x557e97162020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7facbc003ab0 tx=0x7facbc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc0af5ab36fa812 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:20.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.912+0000 7facbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:20.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facbaa00640 1 -- 172.21.15.175:0/815423445 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0045c0 con 0x557e97153b70 2024-07-29T00:42:20.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb409b9c0 con 0x557e97153b70 2024-07-29T00:42:20.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facbaa00640 1 -- 172.21.15.175:0/815423445 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7facbc004760 con 0x557e97153b70 2024-07-29T00:42:20.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facbaa00640 1 -- 172.21.15.175:0/815423445 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7facbc061050 con 0x557e97153b70 2024-07-29T00:42:20.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facd644d740 10 client.26618 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:20.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557e971d8bc0 con 0x557e97153b70 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.913+0000 7facbaa00640 1 -- 172.21.15.175:0/815423445 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb409b9c0 con 0x557e97153b70 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facbaa00640 1 -- 172.21.15.175:0/815423445 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7facbc030070 con 0x557e97153b70 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facd644d740 10 client.26618 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facd644d740 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facd644d740 20 client.26618 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facd644d740 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e971d9380 0x557e971db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facd644d740 4 client.26618 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e970c1530 con 0x557e971d9380 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.914+0000 7faccbe00640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e971d9380 0x557e971db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7faccbe00640 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e971d9380 0x557e971db7a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6d48d60fddf336 server_cookie=1e445bf83ad32008 in_seq=0 out_seq=0 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facbaa00640 10 client.26618 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facbaa00640 1 -- 172.21.15.175:0/815423445 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e970c1530 con 0x557e971d9380 2024-07-29T00:42:20.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facbaa00640 10 client.26618 handle_command_reply: tid=0 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 client.26618 shutdown 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 msgr2=0x557e97162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 0x557e97162020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7facbc003ab0 tx=0x7facbc0095c0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 shutdown_connections 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x557e971d9380 0x557e971db7a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e9715e910 0x557e971d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e971550e0 0x557e97162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facbaa00640 0 client.26618 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 --2- 172.21.15.175:0/815423445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e97153b70 0x557e97162020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 >> 172.21.15.175:0/815423445 conn(0x557e9711c950 msgr2=0x557e97155ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 shutdown_connections 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.915+0000 7facd644d740 1 -- 172.21.15.175:0/815423445 wait complete. 2024-07-29T00:42:20.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd644d740 20 client.26618 trim_cache size 0 max 16384 2024-07-29T00:42:20.918 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:42:20.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7facb00a2cf0 msgr2=0x7facb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7facb00a2cf0 0x7facb00a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7facbc009ca0 tx=0x7facbc005ef0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 msgr2=0x7faccc19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 0x7faccc19eab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7facb400d950 tx=0x7facb400de10 comp rx=0 tx=0).stop 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 shutdown_connections 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc10dd50 0x7faccc1a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc108670 0x7faccc19eab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7facb00a2cf0 0x7facb00a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 --2- 172.21.15.175:0/1197535195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc107d10 0x7faccc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.916+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 >> 172.21.15.175:0/1197535195 conn(0x7faccc077670 msgr2=0x7faccc078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.917+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 shutdown_connections 2024-07-29T00:42:20.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:20.917+0000 7facd2400640 1 -- 172.21.15.175:0/1197535195 wait complete. 2024-07-29T00:42:21.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.99 MB/sec execute 132 sec latency 6689.794 ms 2024-07-29T00:42:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:21 smithi181 ceph-mon[36706]: pgmap v178: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:22.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:21 smithi175 ceph-mon[32138]: pgmap v178: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:21 smithi204 ceph-mon[37368]: pgmap v178: 97 pgs: 97 active+clean; 117 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-07-29T00:42:22.898 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.97 MB/sec execute 133 sec latency 7689.920 ms 2024-07-29T00:42:23.227 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:42:23.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:42:23.665 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:23.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.95 MB/sec execute 134 sec latency 8690.051 ms 2024-07-29T00:42:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:23 smithi181 ceph-mon[36706]: pgmap v179: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-07-29T00:42:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:23 smithi175 ceph-mon[32138]: pgmap v179: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-07-29T00:42:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:23 smithi204 ceph-mon[37368]: pgmap v179: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 -- 172.21.15.175:0/962208011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8104420 msgr2=0x7feff8110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 --2- 172.21.15.175:0/962208011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8104420 0x7feff8110950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feff0009be0 tx=0x7feff002a6c0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 -- 172.21.15.175:0/962208011 shutdown_connections 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 --2- 172.21.15.175:0/962208011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8104420 0x7feff8110950 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 --2- 172.21.15.175:0/962208011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff8103a80 0x7feff8103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 --2- 172.21.15.175:0/962208011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8109a80 0x7feff8109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 -- 172.21.15.175:0/962208011 >> 172.21.15.175:0/962208011 conn(0x7feff80ff760 msgr2=0x7feff8101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 -- 172.21.15.175:0/962208011 shutdown_connections 2024-07-29T00:42:24.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.683+0000 7fefff000640 1 -- 172.21.15.175:0/962208011 wait complete. 2024-07-29T00:42:24.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.685+0000 7fefff000640 1 Processor -- start 2024-07-29T00:42:24.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.685+0000 7fefff000640 1 -- start start 2024-07-29T00:42:24.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.685+0000 7fefff000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 0x7feff819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.685+0000 7fefff000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8104420 0x7feff819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7fefff000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff8109a80 0x7feff819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7fefff000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff8113e80 con 0x7feff8109a80 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7fefff000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feff8113d00 con 0x7feff8103a80 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7fefff000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feff8114000 con 0x7feff8104420 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 0x7feff819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 0x7feff819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58800/0 (socket says 172.21.15.175:58800) 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7e00640 1 -- 172.21.15.175:0/511157859 learned_addr learned my addr 172.21.15.175:0/511157859 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7400640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8104420 0x7feff819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feffd000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff8109a80 0x7feff819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7400640 1 -- 172.21.15.175:0/511157859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 msgr2=0x7feff819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7400640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 0x7feff819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7400640 1 -- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff8109a80 msgr2=0x7feff819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7400640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff8109a80 0x7feff819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7400640 1 -- 172.21.15.175:0/511157859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff819f290 con 0x7feff8104420 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.686+0000 7feff7e00640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 0x7feff819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.687+0000 7feff7400640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8104420 0x7feff819a820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fefe4005fd0 tx=0x7fefe400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.687+0000 7feff4c00640 1 -- 172.21.15.175:0/511157859 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe400cd00 con 0x7feff8104420 2024-07-29T00:42:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.687+0000 7feff4c00640 1 -- 172.21.15.175:0/511157859 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe40035f0 con 0x7feff8104420 2024-07-29T00:42:24.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.687+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff819f580 con 0x7feff8104420 2024-07-29T00:42:24.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.687+0000 7feff4c00640 1 -- 172.21.15.175:0/511157859 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4010690 con 0x7feff8104420 2024-07-29T00:42:24.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.687+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81a6e60 con 0x7feff8104420 2024-07-29T00:42:24.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.690+0000 7ff001ab3740 1 Processor -- start 2024-07-29T00:42:24.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7feff4c00640 1 -- 172.21.15.175:0/511157859 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fefe40037a0 con 0x7feff8104420 2024-07-29T00:42:24.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 -- start start 2024-07-29T00:42:24.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cbdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cc8d20 0x55c2a0ccb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0c238a0 con 0x55c2a0cbdb40 2024-07-29T00:42:24.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0bbb710 con 0x55c2a0cc8d20 2024-07-29T00:42:24.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7ff001ab3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0c1f860 con 0x55c2a0cbe460 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.691+0000 7feffd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cbdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cc8d20 0x55c2a0ccb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feffd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cbdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40362/0 (socket says 172.21.15.175:40362) 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46918/0 (socket says 172.21.15.175:46918) 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feffd000640 1 -- 172.21.15.175:0/2578773530 learned_addr learned my addr 172.21.15.175:0/2578773530 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:24.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 -- 172.21.15.175:0/2578773530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cc8d20 msgr2=0x55c2a0ccb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cc8d20 0x55c2a0ccb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 -- 172.21.15.175:0/2578773530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 msgr2=0x55c2a0cbdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cbdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 -- 172.21.15.175:0/2578773530 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2a09956a0 con 0x55c2a0cbe460 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.692+0000 7feff7e00640 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc87e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fefe8011440 tx=0x7fefe8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82570bb6386595f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.693+0000 7feff4c00640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fefdc0a2d70 0x7fefdc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.693+0000 7fefed600640 1 -- 172.21.15.175:0/2578773530 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8004030 con 0x55c2a0cbe460 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.693+0000 7fefed600640 1 -- 172.21.15.175:0/2578773530 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe80041d0 con 0x55c2a0cbe460 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.693+0000 7fefed600640 1 -- 172.21.15.175:0/2578773530 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe800f930 con 0x55c2a0cbe460 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.693+0000 7feff4c00640 1 -- 172.21.15.175:0/511157859 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fefe40cf5f0 con 0x7feff8104420 2024-07-29T00:42:24.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.693+0000 7feffd000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fefdc0a2d70 0x7fefdc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 -- 172.21.15.175:0/2578773530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 msgr2=0x55c2a0cc87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7feffd000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fefdc0a2d70 0x7fefdc0a5230 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feff002acb0 tx=0x7feff0005fd0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc87e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fefe8011440 tx=0x7fefe8011900 comp rx=0 tx=0).stop 2024-07-29T00:42:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 -- 172.21.15.175:0/2578773530 shutdown_connections 2024-07-29T00:42:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cc8d20 0x55c2a0ccb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc87e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2578773530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cbdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 -- 172.21.15.175:0/2578773530 >> 172.21.15.175:0/2578773530 conn(0x55c2a0cba0d0 msgr2=0x55c2a0cba4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 -- 172.21.15.175:0/2578773530 shutdown_connections 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.694+0000 7ff001ab3740 1 -- 172.21.15.175:0/2578773530 wait complete. 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 -- start start 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 Processor -- start 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cc5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cc8d20 0x55c2a0cc65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0ccb8f0 con 0x55c2a0cbe460 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0bde8c0 con 0x55c2a0cbdb40 2024-07-29T00:42:24.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0ccba70 con 0x55c2a0cc8d20 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.695+0000 7ff001ab3740 10 client.0 _ll_register_callbacks cb 0x55c2a0bcaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40364/0 (socket says 172.21.15.175:40364) 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 -- 172.21.15.175:0/2059073057 learned_addr learned my addr 172.21.15.175:0/2059073057 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7e00640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cc5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feffd000640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cc8d20 0x55c2a0cc65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 -- 172.21.15.175:0/2059073057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cc8d20 msgr2=0x55c2a0cc65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cc8d20 0x55c2a0cc65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 -- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbdb40 msgr2=0x55c2a0cc5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cc5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7400640 1 -- 172.21.15.175:0/2059073057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2a0bc1bc0 con 0x55c2a0cbe460 2024-07-29T00:42:24.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.696+0000 7feff7e00640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cc5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:24.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7feff7400640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc6090 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c6a0 tx=0x7fefe409b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10350d7dd57afcb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7fefe2000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7fefe2000640 1 -- 172.21.15.175:0/2059073057 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40cd140 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7fefe2000640 1 -- 172.21.15.175:0/2059073057 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fefe40d1b60 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe80041d0 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7fefe2000640 1 -- 172.21.15.175:0/2059073057 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fefe409c190 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7ff001ab3740 10 client.16491 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c2a0cd7900 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.697+0000 7fefe2000640 1 -- 172.21.15.175:0/2059073057 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe80041d0 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7fefe2000640 1 -- 172.21.15.175:0/2059073057 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fefe40cf0c0 con 0x55c2a0cbe460 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7ff001ab3740 10 client.16491 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7ff001ab3740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:24.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7ff001ab3740 20 client.16491 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cd8d60 0x55c2a0cdb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7ff001ab3740 4 client.16491 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2a0be3c90 con 0x55c2a0cd8d60 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.698+0000 7feff7e00640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cd8d60 0x55c2a0cdb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.699+0000 7feff7e00640 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cd8d60 0x55c2a0cdb1b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61feb51faa7c3b5 server_cookie=cb9af5a0267f4196 in_seq=0 out_seq=0 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.699+0000 7fefe2000640 10 client.16491 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7fefe2000640 1 -- 172.21.15.175:0/2059073057 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2a0be3c90 con 0x55c2a0cd8d60 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7fefe2000640 10 client.16491 handle_command_reply: tid=0 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 client.16491 shutdown 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 msgr2=0x55c2a0cc6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc6090 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c6a0 tx=0x7fefe409b920 comp rx=0 tx=0).stop 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 shutdown_connections 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cd8d60 0x55c2a0cdb1b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cc8d20 0x55c2a0cc65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0cc6090 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2059073057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbdb40 0x55c2a0cc5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 >> 172.21.15.175:0/2059073057 conn(0x55c2a0cba0d0 msgr2=0x55c2a0cbfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:24.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7fefe2000640 0 client.16491 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:24.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 shutdown_connections 2024-07-29T00:42:24.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.700+0000 7ff001ab3740 1 -- 172.21.15.175:0/2059073057 wait complete. 2024-07-29T00:42:24.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.701+0000 7ff001ab3740 20 client.16491 trim_cache size 0 max 16384 2024-07-29T00:42:24.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.719+0000 7ff001ab3740 1 Processor -- start 2024-07-29T00:42:24.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 -- start start 2024-07-29T00:42:24.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cc7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ce0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbe460 0x55c2a0ce1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff81af8f0 con 0x55c2a0cbdb40 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feff81b1550 con 0x55c2a0cbe460 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7ff001ab3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feff81affc0 con 0x55c2a0cba0d0 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7feff7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ce0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7feff7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cc7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7feff7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ce0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40372/0 (socket says 172.21.15.175:40372) 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7feff7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cc7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46934/0 (socket says 172.21.15.175:46934) 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7feff7400640 1 -- 172.21.15.175:0/2365547290 learned_addr learned my addr 172.21.15.175:0/2365547290 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:24.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.720+0000 7feffd000640 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbe460 0x55c2a0ce1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7feff7e00640 1 -- 172.21.15.175:0/2365547290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbe460 msgr2=0x55c2a0ce1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7feff7e00640 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbe460 0x55c2a0ce1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7feff7e00640 1 -- 172.21.15.175:0/2365547290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 msgr2=0x55c2a0ce0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7feff7e00640 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ce0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7feff7e00640 1 -- 172.21.15.175:0/2365547290 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2a0cb9f00 con 0x55c2a0cba0d0 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7feff7e00640 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cc7ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fefe801b040 tx=0x7fefe80040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b3bd3725c800a5e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7fefecc00640 1 -- 172.21.15.175:0/2365547290 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8003900 con 0x55c2a0cba0d0 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7fefecc00640 1 -- 172.21.15.175:0/2365547290 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe8005520 con 0x55c2a0cba0d0 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.721+0000 7fefecc00640 1 -- 172.21.15.175:0/2365547290 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe800f790 con 0x55c2a0cba0d0 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 -- 172.21.15.175:0/2365547290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 msgr2=0x55c2a0cc7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cc7ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fefe801b040 tx=0x7fefe80040f0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 -- 172.21.15.175:0/2365547290 shutdown_connections 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cbe460 0x55c2a0ce1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ce0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 --2- 172.21.15.175:0/2365547290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cc7ac0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 -- 172.21.15.175:0/2365547290 >> 172.21.15.175:0/2365547290 conn(0x55c2a0cc1540 msgr2=0x55c2a0cc73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:24.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 -- 172.21.15.175:0/2365547290 shutdown_connections 2024-07-29T00:42:24.723 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.722+0000 7ff001ab3740 1 -- 172.21.15.175:0/2365547290 wait complete. 2024-07-29T00:42:24.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 -- start start 2024-07-29T00:42:24.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 Processor -- start 2024-07-29T00:42:24.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ccd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0ccd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff81b0dc0 con 0x55c2a0cbe460 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feff81b0690 con 0x55c2a0cba0d0 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.723+0000 7ff001ab3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feff81ad8d0 con 0x55c2a0cbdb40 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7ff001ab3740 10 client.0 _ll_register_callbacks cb 0x7fefe4097d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58828/0 (socket says 172.21.15.175:58828) 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 -- 172.21.15.175:0/545748327 learned_addr learned my addr 172.21.15.175:0/545748327 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7400640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ccd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feffd000640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0ccd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 -- 172.21.15.175:0/545748327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbdb40 msgr2=0x55c2a0ccd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ccd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 -- 172.21.15.175:0/545748327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 msgr2=0x55c2a0ccd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0ccd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7e00640 1 -- 172.21.15.175:0/545748327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2a0d42bf0 con 0x55c2a0cba0d0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.724+0000 7feff7400640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ccd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7feff7e00640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cccba0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009db0 tx=0x7fefe80056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1437100005dc93 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7fefe2000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7fefe2000640 1 -- 172.21.15.175:0/545748327 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe803c060 con 0x55c2a0cba0d0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7fefe2000640 1 -- 172.21.15.175:0/545748327 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fefe8003900 con 0x55c2a0cba0d0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe8005520 con 0x55c2a0cba0d0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7fefe2000640 1 -- 172.21.15.175:0/545748327 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fefe8003b00 con 0x55c2a0cba0d0 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7ff001ab3740 10 client.26636 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c2a0d43ef0 con 0x55c2a0cba0d0 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.725+0000 7fefe2000640 1 -- 172.21.15.175:0/545748327 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe8005520 con 0x55c2a0cba0d0 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7fefe2000640 1 -- 172.21.15.175:0/545748327 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fefe802d070 con 0x55c2a0cba0d0 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7ff001ab3740 10 client.26636 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7ff001ab3740 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7ff001ab3740 20 client.26636 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7ff001ab3740 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cc8d20 0x55c2a0d46460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7ff001ab3740 4 client.26636 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c2a0c30e10 con 0x55c2a0cc8d20 2024-07-29T00:42:24.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.726+0000 7feff7400640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cc8d20 0x55c2a0d46460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.727+0000 7fefe2000640 10 client.26636 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.727+0000 7feff7400640 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cc8d20 0x55c2a0d46460 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4b0a9b801f0012 server_cookie=630779617349b5c in_seq=0 out_seq=0 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7fefe2000640 1 -- 172.21.15.175:0/545748327 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c2a0c30e10 con 0x55c2a0cc8d20 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7fefe2000640 10 client.26636 handle_command_reply: tid=0 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 client.26636 shutdown 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 msgr2=0x55c2a0cccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cccba0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009db0 tx=0x7fefe80056d0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 shutdown_connections 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c2a0cc8d20 0x55c2a0d46460 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2a0cbe460 0x55c2a0ccd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2a0cbdb40 0x55c2a0ccd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 --2- 172.21.15.175:0/545748327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2a0cba0d0 0x55c2a0cccba0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7fefe2000640 0 client.26636 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 >> 172.21.15.175:0/545748327 conn(0x55c2a0cc1540 msgr2=0x55c2a0cc6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.728+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 shutdown_connections 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.729+0000 7ff001ab3740 1 -- 172.21.15.175:0/545748327 wait complete. 2024-07-29T00:42:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.729+0000 7ff001ab3740 20 client.26636 trim_cache size 0 max 16384 2024-07-29T00:42:24.731 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:42:24.731 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:42:24.731 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "8a3e9f56-8f41-400d-a117-ff68f0d345a2", 2024-07-29T00:42:24.731 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:42:24.731 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.729+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fefdc0a2d70 msgr2=0x7fefdc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.729+0000 7fefff000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fefdc0a2d70 0x7fefdc0a5230 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feff002acb0 tx=0x7feff0005fd0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8104420 msgr2=0x7feff819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8104420 0x7feff819a820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fefe4005fd0 tx=0x7fefe400ee50 comp rx=0 tx=0).stop 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 shutdown_connections 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fefdc0a2d70 0x7fefdc0a5230 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff8109a80 0x7feff819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feff8104420 0x7feff819a820 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 --2- 172.21.15.175:0/511157859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff8103a80 0x7feff819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 >> 172.21.15.175:0/511157859 conn(0x7feff80ff760 msgr2=0x7feff80ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 shutdown_connections 2024-07-29T00:42:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:24.730+0000 7fefff000640 1 -- 172.21.15.175:0/511157859 wait complete. 2024-07-29T00:42:24.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.92 MB/sec execute 135 sec latency 9690.200 ms 2024-07-29T00:42:25.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:42:25.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.90 MB/sec execute 136 sec latency 10690.342 ms 2024-07-29T00:42:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: pgmap v180: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 819 KiB/s wr, 81 op/s 2024-07-29T00:42:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:25 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: pgmap v180: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 819 KiB/s wr, 81 op/s 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.036 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:25 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: pgmap v180: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 819 KiB/s wr, 81 op/s 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:25 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.263 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:26.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.88 MB/sec execute 137 sec latency 11690.469 ms 2024-07-29T00:42:26.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.950+0000 7fa8ef200640 1 -- 172.21.15.175:0/3389835835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 msgr2=0x7fa8f0101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.950+0000 7fa8ef200640 1 --2- 172.21.15.175:0/3389835835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f0101b10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009930 tx=0x7fa8dc02a420 comp rx=0 tx=0).stop 2024-07-29T00:42:26.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 -- 172.21.15.175:0/3389835835 shutdown_connections 2024-07-29T00:42:26.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 --2- 172.21.15.175:0/3389835835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 0x7fa8f010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 --2- 172.21.15.175:0/3389835835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f0101b10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 --2- 172.21.15.175:0/3389835835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 0x7fa8f0107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 -- 172.21.15.175:0/3389835835 >> 172.21.15.175:0/3389835835 conn(0x7fa8f00fd430 msgr2=0x7fa8f00ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:26.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 -- 172.21.15.175:0/3389835835 shutdown_connections 2024-07-29T00:42:26.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.951+0000 7fa8ef200640 1 -- 172.21.15.175:0/3389835835 wait complete. 2024-07-29T00:42:26.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.952+0000 7fa8ef200640 1 Processor -- start 2024-07-29T00:42:26.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.952+0000 7fa8ef200640 1 -- start start 2024-07-29T00:42:26.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ef200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ef200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 0x7fa8f019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ef200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 0x7fa8f019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ef200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0111a70 con 0x7fa8f01076b0 2024-07-29T00:42:26.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ef200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01118f0 con 0x7fa8f0102050 2024-07-29T00:42:26.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ef200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0111bf0 con 0x7fa8f01016b0 2024-07-29T00:42:26.953 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.953 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ee800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 0x7fa8f019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.953 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 0x7fa8f019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.953 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46966/0 (socket says 172.21.15.175:46966) 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ede00640 1 -- 172.21.15.175:0/9034987 learned_addr learned my addr 172.21.15.175:0/9034987 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.953+0000 7fa8ed400640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 0x7fa8f019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58850/0 (socket says 172.21.15.175:58850) 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ee800640 1 -- 172.21.15.175:0/9034987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 msgr2=0x7fa8f019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ee800640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ee800640 1 -- 172.21.15.175:0/9034987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 msgr2=0x7fa8f019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ee800640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 0x7fa8f019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ee800640 1 -- 172.21.15.175:0/9034987 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f019f190 con 0x7fa8f01076b0 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ede00640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ee800640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 0x7fa8f019eab0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800ec50 tx=0x7fa8e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8e6a00640 1 -- 172.21.15.175:0/9034987 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8003b70 con 0x7fa8f01076b0 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f019f480 con 0x7fa8f01076b0 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8e6a00640 1 -- 172.21.15.175:0/9034987 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e8003d10 con 0x7fa8f01076b0 2024-07-29T00:42:26.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8e6a00640 1 -- 172.21.15.175:0/9034987 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8010890 con 0x7fa8f01076b0 2024-07-29T00:42:26.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.954+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f01a6d60 con 0x7fa8f01076b0 2024-07-29T00:42:26.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.958+0000 7fa8e6a00640 1 -- 172.21.15.175:0/9034987 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa8e8003fc0 con 0x7fa8f01076b0 2024-07-29T00:42:26.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.958+0000 7fa8f79fc740 1 Processor -- start 2024-07-29T00:42:26.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.958+0000 7fa8f79fc740 1 -- start start 2024-07-29T00:42:26.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df4f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 0x55b5df5037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df5060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b5df45dad0 con 0x55b5df4f9430 2024-07-29T00:42:26.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5df3f5af0 con 0x55b5df4f8b10 2024-07-29T00:42:26.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b5df459a90 con 0x55b5df503cf0 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 0x55b5df5037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 0x55b5df5037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40408/0 (socket says 172.21.15.175:40408) 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8ed400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df5060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8ede00640 1 -- 172.21.15.175:0/1657053208 learned_addr learned my addr 172.21.15.175:0/1657053208 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.959+0000 7fa8ed400640 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df4f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ede00640 1 -- 172.21.15.175:0/1657053208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 msgr2=0x55b5df5060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ede00640 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df5060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ede00640 1 -- 172.21.15.175:0/1657053208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 msgr2=0x55b5df4f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ede00640 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df4f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ede00640 1 -- 172.21.15.175:0/1657053208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5df1cf3c0 con 0x55b5df4f9430 2024-07-29T00:42:26.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ed400640 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df4f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:26.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8e6a00640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8d40a2d70 0x7fa8d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ede00640 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 0x55b5df5037b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00115c0 tx=0x7fa8e0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4be9dada9f7ab1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8e6a00640 1 -- 172.21.15.175:0/9034987 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa8e80cfa50 con 0x7fa8f01076b0 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8ed400640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8d40a2d70 0x7fa8d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.960+0000 7fa8db400640 1 -- 172.21.15.175:0/1657053208 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000f660 con 0x55b5df4f9430 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.961+0000 7fa8db400640 1 -- 172.21.15.175:0/1657053208 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e000fce0 con 0x55b5df4f9430 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.961+0000 7fa8db400640 1 -- 172.21.15.175:0/1657053208 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0019e00 con 0x55b5df4f9430 2024-07-29T00:42:26.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.961+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1657053208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 msgr2=0x55b5df5037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.961+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 0x55b5df5037b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00115c0 tx=0x7fa8e0011a80 comp rx=0 tx=0).stop 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.961+0000 7fa8ed400640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8d40a2d70 0x7fa8d40a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009900 tx=0x7fa8dc02a420 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1657053208 shutdown_connections 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df5060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f9430 0x55b5df5037b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1657053208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df4f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1657053208 >> 172.21.15.175:0/1657053208 conn(0x55b5df4f50a0 msgr2=0x55b5df4f5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1657053208 shutdown_connections 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1657053208 wait complete. 2024-07-29T00:42:26.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.962+0000 7fa8f79fc740 1 -- start start 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 Processor -- start 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df500bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df501110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df503cf0 0x55b5df501650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b5df5068c0 con 0x55b5df4f8b10 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5df418bc0 con 0x55b5df503cf0 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b5df506a40 con 0x55b5df4f9430 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8f79fc740 10 client.0 _ll_register_callbacks cb 0x55b5df404d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df500bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8ed400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df501110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df500bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40418/0 (socket says 172.21.15.175:40418) 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8ede00640 1 -- 172.21.15.175:0/1222133351 learned_addr learned my addr 172.21.15.175:0/1222133351 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8ed400640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df501110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:46994/0 (socket says 172.21.15.175:46994) 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.963+0000 7fa8ee800640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df503cf0 0x55b5df501650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ee800640 1 -- 172.21.15.175:0/1222133351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 msgr2=0x55b5df501110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ee800640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df501110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ee800640 1 -- 172.21.15.175:0/1222133351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 msgr2=0x55b5df500bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ee800640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df500bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ee800640 1 -- 172.21.15.175:0/1222133351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5df50fdb0 con 0x55b5df503cf0 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ed400640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df501110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ede00640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df500bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8ee800640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df503cf0 0x55b5df501650 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80cd120 tx=0x7fa8e809ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f9acc25f96f108 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.964+0000 7fa8cfe00640 1 -- 172.21.15.175:0/1222133351 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e809c350 con 0x55b5df503cf0 2024-07-29T00:42:26.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8cfe00640 1 -- 172.21.15.175:0/1222133351 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa8e80a46d0 con 0x55b5df503cf0 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e000fce0 con 0x55b5df503cf0 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8cfe00640 1 -- 172.21.15.175:0/1222133351 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8e809c5f0 con 0x55b5df503cf0 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8f79fc740 10 client.26648 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b5df5127a0 con 0x55b5df503cf0 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8cfe00640 1 -- 172.21.15.175:0/1222133351 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e000fce0 con 0x55b5df503cf0 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.965+0000 7fa8cfe00640 1 -- 172.21.15.175:0/1222133351 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fa8e8004b40 con 0x55b5df503cf0 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8f79fc740 10 client.26648 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8f79fc740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8f79fc740 20 client.26648 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df513c60 0x55b5df516100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8f79fc740 4 client.26648 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5df46b040 con 0x55b5df513c60 2024-07-29T00:42:26.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.966+0000 7fa8ede00640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df513c60 0x55b5df516100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.967+0000 7fa8ede00640 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df513c60 0x55b5df516100 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195b68319abe101e server_cookie=c0b137e3e2d6e867 in_seq=0 out_seq=0 2024-07-29T00:42:26.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.967+0000 7fa8cfe00640 10 client.26648 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:26.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.967+0000 7fa8cfe00640 1 -- 172.21.15.175:0/1222133351 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5df46b040 con 0x55b5df513c60 2024-07-29T00:42:26.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.967+0000 7fa8cfe00640 10 client.26648 handle_command_reply: tid=0 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.967+0000 7fa8f79fc740 1 client.26648 shutdown 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df503cf0 msgr2=0x55b5df501650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df503cf0 0x55b5df501650 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80cd120 tx=0x7fa8e809ba30 comp rx=0 tx=0).stop 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 shutdown_connections 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df513c60 0x55b5df516100 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df503cf0 0x55b5df501650 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df501110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/1222133351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df500bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 >> 172.21.15.175:0/1222133351 conn(0x55b5df4f50a0 msgr2=0x55b5df4f4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8cfe00640 0 client.26648 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 shutdown_connections 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 1 -- 172.21.15.175:0/1222133351 wait complete. 2024-07-29T00:42:26.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.968+0000 7fa8f79fc740 20 client.26648 trim_cache size 0 max 16384 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 Processor -- start 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 -- start start 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df4fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f9430 0x55b5df501120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df501660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01af6f0 con 0x55b5df4f8b10 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b1350 con 0x55b5df4f9430 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01afdc0 con 0x55b5df503cf0 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8ee800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df501660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8ee800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df501660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:47006/0 (socket says 172.21.15.175:47006) 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8ee800640 1 -- 172.21.15.175:0/4158473214 learned_addr learned my addr 172.21.15.175:0/4158473214 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.987+0000 7fa8ede00640 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df4fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ed400640 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f9430 0x55b5df501120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ee800640 1 -- 172.21.15.175:0/4158473214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f9430 msgr2=0x55b5df501120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ee800640 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f9430 0x55b5df501120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ee800640 1 -- 172.21.15.175:0/4158473214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 msgr2=0x55b5df4fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ee800640 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df4fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ee800640 1 -- 172.21.15.175:0/4158473214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5df4f8940 con 0x55b5df503cf0 2024-07-29T00:42:26.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8ee800640 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df501660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80099a0 tx=0x7fa8e809bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a44fade46fb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:26.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8daa00640 1 -- 172.21.15.175:0/4158473214 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80a02a0 con 0x55b5df503cf0 2024-07-29T00:42:26.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8daa00640 1 -- 172.21.15.175:0/4158473214 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e80a4b60 con 0x55b5df503cf0 2024-07-29T00:42:26.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.988+0000 7fa8daa00640 1 -- 172.21.15.175:0/4158473214 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e809c3b0 con 0x55b5df503cf0 2024-07-29T00:42:26.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4158473214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 msgr2=0x55b5df501660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df501660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80099a0 tx=0x7fa8e809bb60 comp rx=0 tx=0).stop 2024-07-29T00:42:26.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4158473214 shutdown_connections 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df503cf0 0x55b5df501660 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f9430 0x55b5df501120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4158473214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df4f8b10 0x55b5df4fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4158473214 >> 172.21.15.175:0/4158473214 conn(0x55b5df4f50a0 msgr2=0x55b5df502c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4158473214 shutdown_connections 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.989+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4158473214 wait complete. 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.990+0000 7fa8f79fc740 1 -- start start 2024-07-29T00:42:26.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.990+0000 7fa8f79fc740 1 Processor -- start 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df5071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df507730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df503cf0 0x55b5df57c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b0bc0 con 0x55b5df503cf0 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b0490 con 0x55b5df4f8b10 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01ad6d0 con 0x55b5df4f9430 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8f79fc740 10 client.0 _ll_register_callbacks cb 0x7fa8e8098190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df507730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ee800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df503cf0 0x55b5df57c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df507730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:47010/0 (socket says 172.21.15.175:47010) 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 -- 172.21.15.175:0/4175170857 learned_addr learned my addr 172.21.15.175:0/4175170857 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ede00640 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df5071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 -- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 msgr2=0x55b5df5071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df5071f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 -- 172.21.15.175:0/4175170857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df503cf0 msgr2=0x55b5df57c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df503cf0 0x55b5df57c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.991+0000 7fa8ed400640 1 -- 172.21.15.175:0/4175170857 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5df57ced0 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8ed400640 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df507730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009930 tx=0x7fa8dc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2db2547377266e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8cfe00640 1 -- 172.21.15.175:0/4175170857 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc035040 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8cfe00640 1 -- 172.21.15.175:0/4175170857 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fa8dc003980 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e80a4b60 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8cfe00640 1 -- 172.21.15.175:0/4175170857 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8dc003b80 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8f79fc740 10 client.26611 fetch_fsmap learned FSMap version 38 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.992+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b5df57e120 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8cfe00640 1 -- 172.21.15.175:0/4175170857 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e80a4b60 con 0x55b5df4f9430 2024-07-29T00:42:26.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8cfe00640 1 -- 172.21.15.175:0/4175170857 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fa8dc004ad0 con 0x55b5df4f9430 2024-07-29T00:42:26.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8f79fc740 10 client.26611 fetch_fsmap finished waiting for FSMap version 38 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8f79fc740 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8f79fc740 20 client.26611 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df57e9b0 0x55b5df580dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8f79fc740 4 client.26611 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5df465550 con 0x55b5df57e9b0 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.993+0000 7fa8ede00640 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df57e9b0 0x55b5df580dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8ede00640 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df57e9b0 0x55b5df580dd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e0c2da18b09547 server_cookie=e095858fee94c27b in_seq=0 out_seq=0 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8cfe00640 10 client.26611 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8cfe00640 1 -- 172.21.15.175:0/4175170857 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b5df465550 con 0x55b5df57e9b0 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8cfe00640 10 client.26611 handle_command_reply: tid=0 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8f79fc740 1 client.26611 shutdown 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 msgr2=0x55b5df507730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df507730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009930 tx=0x7fa8dc009260 comp rx=0 tx=0).stop 2024-07-29T00:42:26.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.994+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 shutdown_connections 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55b5df57e9b0 0x55b5df580dd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b5df503cf0 0x55b5df57c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5df4f9430 0x55b5df507730 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8cfe00640 0 client.26611 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 --2- 172.21.15.175:0/4175170857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5df4f8b10 0x55b5df5071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 >> 172.21.15.175:0/4175170857 conn(0x55b5df4f50a0 msgr2=0x55b5df4f39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 shutdown_connections 2024-07-29T00:42:26.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 1 -- 172.21.15.175:0/4175170857 wait complete. 2024-07-29T00:42:26.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:26 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:42:26.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:26 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.995+0000 7fa8f79fc740 20 client.26611 trim_cache size 0 max 16384 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "8a3e9f56-8f41-400d-a117-ff68f0d345a2": { 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "8a3e9f56-8f41-400d-a117-ff68f0d345a2", 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:26.997 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:42:26.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8d40a2d70 msgr2=0x7fa8d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8d40a2d70 0x7fa8d40a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009900 tx=0x7fa8dc02a420 comp rx=0 tx=0).stop 2024-07-29T00:42:26.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 msgr2=0x7fa8f019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:26.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 0x7fa8f019eab0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800ec50 tx=0x7fa8e800c6a0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 shutdown_connections 2024-07-29T00:42:26.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8d40a2d70 0x7fa8d40a5230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f01076b0 0x7fa8f019eab0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0102050 0x7fa8f019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 --2- 172.21.15.175:0/9034987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8f01016b0 0x7fa8f019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 >> 172.21.15.175:0/9034987 conn(0x7fa8f00fd430 msgr2=0x7fa8f010cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 shutdown_connections 2024-07-29T00:42:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:26.996+0000 7fa8ef200640 1 -- 172.21.15.175:0/9034987 wait complete. 2024-07-29T00:42:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:26 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:42:26.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:26 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:26 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:42:27.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:26 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:42:27.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.86 MB/sec execute 138 sec latency 12690.546 ms 2024-07-29T00:42:27.957 INFO:tasks.cephfs.filesystem:scrub status for tag:8a3e9f56-8f41-400d-a117-ff68f0d345a2 - {'path': '/', 'tag': '8a3e9f56-8f41-400d-a117-ff68f0d345a2', 'options': 'recursive,force'} 2024-07-29T00:42:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:27 smithi181 ceph-mon[36706]: pgmap v181: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-07-29T00:42:28.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:27 smithi175 ceph-mon[32138]: pgmap v181: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-07-29T00:42:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:27 smithi204 ceph-mon[37368]: pgmap v181: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-07-29T00:42:28.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.84 MB/sec execute 139 sec latency 13690.675 ms 2024-07-29T00:42:29.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.82 MB/sec execute 140 sec latency 14690.820 ms 2024-07-29T00:42:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:29 smithi175 ceph-mon[32138]: pgmap v182: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 540 KiB/s wr, 40 op/s 2024-07-29T00:42:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:29 smithi204 ceph-mon[37368]: pgmap v182: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 540 KiB/s wr, 40 op/s 2024-07-29T00:42:30.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:29 smithi181 ceph-mon[36706]: pgmap v182: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 540 KiB/s wr, 40 op/s 2024-07-29T00:42:30.899 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.80 MB/sec execute 141 sec latency 15690.962 ms 2024-07-29T00:42:31.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.78 MB/sec execute 142 sec latency 16691.113 ms 2024-07-29T00:42:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:31 smithi175 ceph-mon[32138]: pgmap v183: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 540 KiB/s wr, 40 op/s 2024-07-29T00:42:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:31 smithi204 ceph-mon[37368]: pgmap v183: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 540 KiB/s wr, 40 op/s 2024-07-29T00:42:32.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:31 smithi181 ceph-mon[36706]: pgmap v183: 97 pgs: 97 active+clean; 116 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 540 KiB/s wr, 40 op/s 2024-07-29T00:42:32.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.76 MB/sec execute 143 sec latency 17691.258 ms 2024-07-29T00:42:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:32 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:42:33.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:32 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:42:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:32 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:42:33.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:32 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:42:33.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:32 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:42:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:32 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:42:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:32 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:42:33.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:32 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:42:33.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:32 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:42:33.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:32 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:42:33.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:32 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:42:33.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:32 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:42:33.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.74 MB/sec execute 144 sec latency 18691.452 ms 2024-07-29T00:42:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:33 smithi175 ceph-mon[32138]: pgmap v184: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 803 KiB/s wr, 54 op/s 2024-07-29T00:42:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:33 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:42:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:33 smithi204 ceph-mon[37368]: pgmap v184: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 803 KiB/s wr, 54 op/s 2024-07-29T00:42:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:33 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:42:34.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:33 smithi181 ceph-mon[36706]: pgmap v184: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 803 KiB/s wr, 54 op/s 2024-07-29T00:42:34.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:33 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:42:34.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.72 MB/sec execute 145 sec latency 19691.604 ms 2024-07-29T00:42:35.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.70 MB/sec execute 146 sec latency 20691.753 ms 2024-07-29T00:42:36.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:35 smithi175 ceph-mon[32138]: pgmap v185: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 38 op/s 2024-07-29T00:42:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:35 smithi204 ceph-mon[37368]: pgmap v185: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 38 op/s 2024-07-29T00:42:36.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:35 smithi181 ceph-mon[36706]: pgmap v185: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 38 op/s 2024-07-29T00:42:36.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.68 MB/sec execute 147 sec latency 21691.896 ms 2024-07-29T00:42:37.900 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.67 MB/sec execute 148 sec latency 22692.041 ms 2024-07-29T00:42:38.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:37 smithi175 ceph-mon[32138]: pgmap v186: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 845 KiB/s wr, 55 op/s 2024-07-29T00:42:38.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:37 smithi204 ceph-mon[37368]: pgmap v186: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 845 KiB/s wr, 55 op/s 2024-07-29T00:42:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:37 smithi181 ceph-mon[36706]: pgmap v186: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 845 KiB/s wr, 55 op/s 2024-07-29T00:42:38.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.65 MB/sec execute 149 sec latency 23692.216 ms 2024-07-29T00:42:39.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.63 MB/sec execute 150 sec latency 24692.356 ms 2024-07-29T00:42:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:39 smithi204 ceph-mon[37368]: pgmap v187: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 571 KiB/s wr, 31 op/s 2024-07-29T00:42:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:39 smithi181 ceph-mon[36706]: pgmap v187: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 571 KiB/s wr, 31 op/s 2024-07-29T00:42:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:39 smithi175 ceph-mon[32138]: pgmap v187: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 571 KiB/s wr, 31 op/s 2024-07-29T00:42:40.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.61 MB/sec execute 151 sec latency 25692.506 ms 2024-07-29T00:42:41.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.60 MB/sec execute 152 sec latency 26692.655 ms 2024-07-29T00:42:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:41 smithi204 ceph-mon[37368]: pgmap v188: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 571 KiB/s wr, 31 op/s 2024-07-29T00:42:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:41 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:42:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:41 smithi181 ceph-mon[36706]: pgmap v188: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 571 KiB/s wr, 31 op/s 2024-07-29T00:42:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:41 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:42:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:41 smithi175 ceph-mon[32138]: pgmap v188: 97 pgs: 97 active+clean; 118 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 571 KiB/s wr, 31 op/s 2024-07-29T00:42:42.341 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:41 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:42:42.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.58 MB/sec execute 153 sec latency 27692.798 ms 2024-07-29T00:42:43.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:42:43.784 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:43.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.56 MB/sec execute 154 sec latency 28692.939 ms 2024-07-29T00:42:44.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:43 smithi204 ceph-mon[37368]: pgmap v189: 97 pgs: 97 active+clean; 117 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.0 MiB/s wr, 51 op/s 2024-07-29T00:42:44.203 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:43 smithi175 ceph-mon[32138]: pgmap v189: 97 pgs: 97 active+clean; 117 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.0 MiB/s wr, 51 op/s 2024-07-29T00:42:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:43 smithi181 ceph-mon[36706]: pgmap v189: 97 pgs: 97 active+clean; 117 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.0 MiB/s wr, 51 op/s 2024-07-29T00:42:44.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 -- 172.21.15.175:0/1020655374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 msgr2=0x7fbc40114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:44.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 --2- 172.21.15.175:0/1020655374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc40114b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009be0 tx=0x7fbc3802a6c0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 -- 172.21.15.175:0/1020655374 shutdown_connections 2024-07-29T00:42:44.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 --2- 172.21.15.175:0/1020655374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc40114b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 --2- 172.21.15.175:0/1020655374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc40108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 --2- 172.21.15.175:0/1020655374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 0x7fbc4010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 -- 172.21.15.175:0/1020655374 >> 172.21.15.175:0/1020655374 conn(0x7fbc40077670 msgr2=0x7fbc40079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:44.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 -- 172.21.15.175:0/1020655374 shutdown_connections 2024-07-29T00:42:44.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.509+0000 7fbc46e00640 1 -- 172.21.15.175:0/1020655374 wait complete. 2024-07-29T00:42:44.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.510+0000 7fbc46e00640 1 Processor -- start 2024-07-29T00:42:44.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 -- start start 2024-07-29T00:42:44.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc4019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:44.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc4019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:44.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 0x7fbc401a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:44.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40118070 con 0x7fbc40107d10 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40117ef0 con 0x7fbc40108670 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc46e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc401181f0 con 0x7fbc4010dd50 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc45a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc4019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc45a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc4019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38574/0 (socket says 172.21.15.175:38574) 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.511+0000 7fbc45a00640 1 -- 172.21.15.175:0/2025797635 learned_addr learned my addr 172.21.15.175:0/2025797635 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45000640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc4019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc46400640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 0x7fbc401a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45a00640 1 -- 172.21.15.175:0/2025797635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 msgr2=0x7fbc401a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45a00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 0x7fbc401a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45a00640 1 -- 172.21.15.175:0/2025797635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 msgr2=0x7fbc4019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45a00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc4019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45a00640 1 -- 172.21.15.175:0/2025797635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc401a34c0 con 0x7fbc40107d10 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45000640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc4019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:44.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc46400640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 0x7fbc401a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.512+0000 7fbc45a00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc4019e510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e9f0 tx=0x7fbc3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.513+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400cd80 con 0x7fbc40107d10 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.513+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc34005500 con 0x7fbc40107d10 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.513+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc401a37b0 con 0x7fbc40107d10 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.513+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34010690 con 0x7fbc40107d10 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.513+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401ab090 con 0x7fbc40107d10 2024-07-29T00:42:44.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.514+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc40108130 con 0x7fbc40107d10 2024-07-29T00:42:44.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.516+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fbc340046c0 con 0x7fbc40107d10 2024-07-29T00:42:44.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.517+0000 7fbc3e000640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbc280a2cf0 0x7fbc280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:44.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.517+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fbc340cf920 con 0x7fbc40107d10 2024-07-29T00:42:44.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.517+0000 7fbc45000640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbc280a2cf0 0x7fbc280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:44.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.518+0000 7fbc45000640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbc280a2cf0 0x7fbc280a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0096c0 tx=0x7fbc2c009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:44.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.522+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbc3409c4c0 con 0x7fbc40107d10 2024-07-29T00:42:44.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.732+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc4019f870 con 0x7fbc40107d10 2024-07-29T00:42:44.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.733+0000 7fbc3e000640 1 -- 172.21.15.175:0/2025797635 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12002 (secure 0 0 0) 0x7fbc3409bdc0 con 0x7fbc40107d10 2024-07-29T00:42:44.735 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:42:44.735 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":39,"btime":"2024-07-29T00:42:40:791346+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:42:40.665517+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:42:44.735 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 39 2024-07-29T00:42:44.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbc280a2cf0 msgr2=0x7fbc280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:44.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbc280a2cf0 0x7fbc280a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0096c0 tx=0x7fbc2c009260 comp rx=0 tx=0).stop 2024-07-29T00:42:44.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 msgr2=0x7fbc4019e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:44.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc4019e510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e9f0 tx=0x7fbc3400edf0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 shutdown_connections 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc4010dd50 0x7fbc401a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc40108670 0x7fbc4019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.736+0000 7fbc46e00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fbc280a2cf0 0x7fbc280a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.737+0000 7fbc46e00640 1 --2- 172.21.15.175:0/2025797635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc40107d10 0x7fbc4019e510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.737+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 >> 172.21.15.175:0/2025797635 conn(0x7fbc40077670 msgr2=0x7fbc40078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.737+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 shutdown_connections 2024-07-29T00:42:44.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:44.737+0000 7fbc46e00640 1 -- 172.21.15.175:0/2025797635 wait complete. 2024-07-29T00:42:44.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.55 MB/sec execute 155 sec latency 29693.072 ms 2024-07-29T00:42:45.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:44 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2025797635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:45.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:44 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2025797635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:45.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:44 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2025797635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:42:45.741 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id 2caef02d --release --await' 2024-07-29T00:42:45.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db --set-id 2caef02d --release --await 2024-07-29T00:42:45.901 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.53 MB/sec execute 156 sec latency 30693.198 ms 2024-07-29T00:42:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:45 smithi175 ceph-mon[32138]: pgmap v190: 97 pgs: 97 active+clean; 117 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 784 KiB/s wr, 36 op/s 2024-07-29T00:42:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:45 smithi204 ceph-mon[37368]: pgmap v190: 97 pgs: 97 active+clean; 117 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 784 KiB/s wr, 36 op/s 2024-07-29T00:42:46.208 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:46.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:45 smithi181 ceph-mon[36706]: pgmap v190: 97 pgs: 97 active+clean; 117 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 784 KiB/s wr, 36 op/s 2024-07-29T00:42:46.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 176808 2.51 MB/sec execute 157 sec latency 31693.339 ms 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 -- 172.21.15.175:0/1556618293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076d30 msgr2=0x7f6460112020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 --2- 172.21.15.175:0/1556618293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076d30 0x7f6460112020 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f644c009900 tx=0x7f644c02a3f0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 -- 172.21.15.175:0/1556618293 shutdown_connections 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 --2- 172.21.15.175:0/1556618293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460112560 0x7f6460114950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 --2- 172.21.15.175:0/1556618293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076d30 0x7f6460112020 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 --2- 172.21.15.175:0/1556618293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460076410 0x7f64600767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 -- 172.21.15.175:0/1556618293 >> 172.21.15.175:0/1556618293 conn(0x7f64600ff5c0 msgr2=0x7f64601019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 -- 172.21.15.175:0/1556618293 shutdown_connections 2024-07-29T00:42:46.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.978+0000 7f6466e00640 1 -- 172.21.15.175:0/1556618293 wait complete. 2024-07-29T00:42:46.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.979+0000 7f6466e00640 1 Processor -- start 2024-07-29T00:42:46.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.980+0000 7f6466e00640 1 -- start start 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.980+0000 7f6466e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076410 0x7f646019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.980+0000 7f6466e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 0x7f646019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6466e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460112560 0x7f64601a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6466e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6460118140 con 0x7f6460076410 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6466e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6460117fc0 con 0x7f6460076d30 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6466e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64601182c0 con 0x7f6460112560 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076410 0x7f646019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 0x7f646019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 0x7f646019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51644/0 (socket says 172.21.15.175:51644) 2024-07-29T00:42:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 -- 172.21.15.175:0/512117150 learned_addr learned my addr 172.21.15.175:0/512117150 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:46.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6466400640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460112560 0x7f64601a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 -- 172.21.15.175:0/512117150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460112560 msgr2=0x7f64601a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460112560 0x7f64601a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 -- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076410 msgr2=0x7f646019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076410 0x7f646019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.981+0000 7f6465000640 1 -- 172.21.15.175:0/512117150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64601a3570 con 0x7f6460076d30 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f6465a00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076410 0x7f646019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f6466400640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460112560 0x7f64601a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f6465000640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 0x7f646019eb00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f644c004770 tx=0x7f644c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f645e000640 1 -- 172.21.15.175:0/512117150 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c03f070 con 0x7f6460076d30 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64601a3800 con 0x7f6460076d30 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f645e000640 1 -- 172.21.15.175:0/512117150 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c031ca0 con 0x7f6460076d30 2024-07-29T00:42:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f645e000640 1 -- 172.21.15.175:0/512117150 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c035030 con 0x7f6460076d30 2024-07-29T00:42:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.982+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64601ab140 con 0x7f6460076d30 2024-07-29T00:42:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f645e000640 1 -- 172.21.15.175:0/512117150 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f644c042070 con 0x7f6460076d30 2024-07-29T00:42:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 Processor -- start 2024-07-29T00:42:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 -- start start 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59dfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59dffb0 0x55f6b59e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f6b593acc0 con 0x55f6b59d4dd0 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f6b58d2bc0 con 0x55f6b59dffb0 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.986+0000 7f646ae84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f6b5936c80 con 0x55f6b59d56f0 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59dfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59dfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55742/0 (socket says 172.21.15.175:55742) 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 -- 172.21.15.175:0/1349312105 learned_addr learned my addr 172.21.15.175:0/1349312105 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465000640 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59dffb0 0x55f6b59e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6466400640 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 -- 172.21.15.175:0/1349312105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59dffb0 msgr2=0x55f6b59e23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:46.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59dffb0 0x55f6b59e23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 -- 172.21.15.175:0/1349312105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 msgr2=0x55f6b59d51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 -- 172.21.15.175:0/1349312105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b564a580 con 0x55f6b59d56f0 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.987+0000 7f6465a00640 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59dfa70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64540115c0 tx=0x7f6454011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c4b13c678ef102 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.988+0000 7f645e000640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f64480a2cf0 0x7f64480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.988+0000 7f645e000640 1 -- 172.21.15.175:0/512117150 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f644c0efab0 con 0x7f6460076d30 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.988+0000 7f6452a00640 1 -- 172.21.15.175:0/1349312105 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645400f640 con 0x55f6b59d56f0 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.988+0000 7f6452a00640 1 -- 172.21.15.175:0/1349312105 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645400fcc0 con 0x55f6b59d56f0 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.988+0000 7f6452a00640 1 -- 172.21.15.175:0/1349312105 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454019da0 con 0x55f6b59d56f0 2024-07-29T00:42:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.988+0000 7f6466400640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f64480a2cf0 0x7f64480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 -- 172.21.15.175:0/1349312105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 msgr2=0x55f6b59dfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:46.990 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59dfa70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64540115c0 tx=0x7f6454011a80 comp rx=0 tx=0).stop 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f6466400640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f64480a2cf0 0x7f64480a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f646006b0c0 tx=0x7f6458009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 -- 172.21.15.175:0/1349312105 shutdown_connections 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59dffb0 0x55f6b59e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59dfa70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 --2- 172.21.15.175:0/1349312105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 -- 172.21.15.175:0/1349312105 >> 172.21.15.175:0/1349312105 conn(0x55f6b59d1360 msgr2=0x55f6b59d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 -- 172.21.15.175:0/1349312105 shutdown_connections 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.989+0000 7f646ae84740 1 -- 172.21.15.175:0/1349312105 wait complete. 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 -- start start 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 Processor -- start 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.991 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 0x55f6b59dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f6b59e2b80 con 0x55f6b59d4dd0 2024-07-29T00:42:46.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f6b58f5ca0 con 0x55f6b59d56f0 2024-07-29T00:42:46.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f6b59e2d00 con 0x55f6b59dffb0 2024-07-29T00:42:46.992 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.990+0000 7f646ae84740 10 client.0 _ll_register_callbacks cb 0x55f6b58e1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:46.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6465a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 0x55f6b59dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6465a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59dcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38620/0 (socket says 172.21.15.175:38620) 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 0x55f6b59dd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55748/0 (socket says 172.21.15.175:55748) 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6465a00640 1 -- 172.21.15.175:0/2709499438 learned_addr learned my addr 172.21.15.175:0/2709499438 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6465000640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 -- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 msgr2=0x55f6b59dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59dd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 -- 172.21.15.175:0/2709499438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 msgr2=0x55f6b59dcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59dcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6466400640 1 -- 172.21.15.175:0/2709499438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b5714960 con 0x55f6b59dffb0 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.991+0000 7f6465000640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:46.994 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f6466400640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 0x55f6b59dd940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6458003f00 tx=0x7f6458004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8592c0a0b5336e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f6447400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f6447400640 1 -- 172.21.15.175:0/2709499438 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580045c0 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f6447400640 1 -- 172.21.15.175:0/2709499438 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f6458004760 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f645400fcc0 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f6447400640 1 -- 172.21.15.175:0/2709499438 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6458029440 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f646ae84740 10 client.26629 fetch_fsmap learned FSMap version 39 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f6b59ee8c0 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.992+0000 7f6447400640 1 -- 172.21.15.175:0/2709499438 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645400fcc0 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f6447400640 1 -- 172.21.15.175:0/2709499438 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f6458027020 con 0x55f6b59dffb0 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f646ae84740 10 client.26629 fetch_fsmap finished waiting for FSMap version 39 2024-07-29T00:42:46.995 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f646ae84740 10 client.26629 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:42:46.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f646ae84740 20 client.26629 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:46.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f646ae84740 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b59efdd0 0x55f6b59f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:46.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f646ae84740 4 client.26629 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:46.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b5948230 con 0x55f6b59efdd0 2024-07-29T00:42:46.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.993+0000 7f6465a00640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b59efdd0 0x55f6b59f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.994+0000 7f6465a00640 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b59efdd0 0x55f6b59f2250 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da566c8b569f7c2 server_cookie=9bb3c083fd7b9d69 in_seq=0 out_seq=0 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.994+0000 7f6447400640 10 client.26629 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f6447400640 1 -- 172.21.15.175:0/2709499438 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f6b5948230 con 0x55f6b59efdd0 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f6447400640 10 client.26629 handle_command_reply: tid=0 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 client.26629 shutdown 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 msgr2=0x55f6b59dd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 0x55f6b59dd940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6458003f00 tx=0x7f6458004ab0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 shutdown_connections 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b59efdd0 0x55f6b59f2250 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59dffb0 0x55f6b59dd940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 --2- 172.21.15.175:0/2709499438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59d4dd0 0x55f6b59dcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f6447400640 0 client.26629 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 >> 172.21.15.175:0/2709499438 conn(0x55f6b59d1360 msgr2=0x55f6b59d7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 shutdown_connections 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.995+0000 7f646ae84740 1 -- 172.21.15.175:0/2709499438 wait complete. 2024-07-29T00:42:46.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:46.996+0000 7f646ae84740 20 client.26629 trim_cache size 0 max 16384 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.014+0000 7f646ae84740 1 Processor -- start 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.014+0000 7f646ae84740 1 -- start start 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f646ae84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f646ae84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f646ae84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b59f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f646ae84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64601b3ac0 con 0x55f6b59dffb0 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f646ae84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64601b56f0 con 0x55f6b59d4dd0 2024-07-29T00:42:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f646ae84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64601b4190 con 0x55f6b59d56f0 2024-07-29T00:42:47.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f6466400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b59f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f6466400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b59f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38628/0 (socket says 172.21.15.175:38628) 2024-07-29T00:42:47.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f6466400640 1 -- 172.21.15.175:0/1176851544 learned_addr learned my addr 172.21.15.175:0/1176851544 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f6465000640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.015+0000 7f6465a00640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465000640 1 -- 172.21.15.175:0/1176851544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d4dd0 msgr2=0x55f6b59d7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465000640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465000640 1 -- 172.21.15.175:0/1176851544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 msgr2=0x55f6b59f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465000640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b59f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465000640 1 -- 172.21.15.175:0/1176851544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b59d0e00 con 0x55f6b59d56f0 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465a00640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6466400640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b59f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6465000640 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59d03b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f644c0ed180 tx=0x7f644c0f3560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27dbb4a772f49b1a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6452000640 1 -- 172.21.15.175:0/1176851544 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c049040 con 0x55f6b59d56f0 2024-07-29T00:42:47.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6452000640 1 -- 172.21.15.175:0/1176851544 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c0bc0c0 con 0x55f6b59d56f0 2024-07-29T00:42:47.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.016+0000 7f6452000640 1 -- 172.21.15.175:0/1176851544 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0b8730 con 0x55f6b59d56f0 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 -- 172.21.15.175:0/1176851544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 msgr2=0x55f6b59d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59d03b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f644c0ed180 tx=0x7f644c0f3560 comp rx=0 tx=0).stop 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 -- 172.21.15.175:0/1176851544 shutdown_connections 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b59f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d56f0 0x55f6b59d03b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 --2- 172.21.15.175:0/1176851544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d4dd0 0x55f6b59d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 -- 172.21.15.175:0/1176851544 >> 172.21.15.175:0/1176851544 conn(0x55f6b59d1360 msgr2=0x55f6b59cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 -- 172.21.15.175:0/1176851544 shutdown_connections 2024-07-29T00:42:47.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 -- 172.21.15.175:0/1176851544 wait complete. 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.017+0000 7f646ae84740 1 -- start start 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 Processor -- start 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d4dd0 0x55f6b59e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b5a53b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64601b4f60 con 0x55f6b59dffb0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64601b4860 con 0x55f6b59d56f0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64601b1aa0 con 0x55f6b59d4dd0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.018+0000 7f646ae84740 10 client.0 _ll_register_callbacks cb 0x7f6458003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6466400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b5a53b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d4dd0 0x55f6b59e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6466400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b5a53b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38642/0 (socket says 172.21.15.175:38642) 2024-07-29T00:42:47.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6466400640 1 -- 172.21.15.175:0/3797145756 learned_addr learned my addr 172.21.15.175:0/3797145756 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:51690/0 (socket says 172.21.15.175:51690) 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d4dd0 0x55f6b59e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:55780/0 (socket says 172.21.15.175:55780) 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 -- 172.21.15.175:0/3797145756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d4dd0 msgr2=0x55f6b59e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d4dd0 0x55f6b59e4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 -- 172.21.15.175:0/3797145756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 msgr2=0x55f6b5a53b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b5a53b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 -- 172.21.15.175:0/3797145756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b5a541a0 con 0x55f6b59d56f0 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6465000640 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59e4b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f644c0bbd70 tx=0x7f644c03b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df865bd85de2424 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6447400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.019+0000 7f6447400640 1 -- 172.21.15.175:0/3797145756 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0da060 con 0x55f6b59d56f0 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f6447400640 1 -- 172.21.15.175:0/3797145756 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f644c0ed400 con 0x55f6b59d56f0 2024-07-29T00:42:47.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644c0bc0c0 con 0x55f6b59d56f0 2024-07-29T00:42:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f6447400640 1 -- 172.21.15.175:0/3797145756 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f644c049040 con 0x55f6b59d56f0 2024-07-29T00:42:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 10 client.26678 fetch_fsmap learned FSMap version 39 2024-07-29T00:42:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f6b5a541a0 con 0x55f6b59d56f0 2024-07-29T00:42:47.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f6447400640 1 -- 172.21.15.175:0/3797145756 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c0bc0c0 con 0x55f6b59d56f0 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f6447400640 1 -- 172.21.15.175:0/3797145756 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f644c0bc320 con 0x55f6b59d56f0 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 10 client.26678 fetch_fsmap finished waiting for FSMap version 39 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 10 client.26678 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 20 client.26678 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b5a5b8a0 0x55f6b5a5dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 4 client.26678 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2caef02d", "release": true, "await": true}] 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.020+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2caef02d", "release": true, "await": true}) -- 0x55f6b5942740 con 0x55f6b5a5b8a0 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.021+0000 7f6465a00640 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b5a5b8a0 0x55f6b5a5dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:47.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.021+0000 7f6465a00640 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b5a5b8a0 0x55f6b5a5dcc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74045ee1c5d2d294 server_cookie=273bfa7f888652e6 in_seq=0 out_seq=0 2024-07-29T00:42:47.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.021+0000 7f6447400640 10 client.26678 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:47.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.028+0000 7f6447400640 1 -- 172.21.15.175:0/3797145756 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f6b5942740 con 0x55f6b5a5b8a0 2024-07-29T00:42:47.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.028+0000 7f6447400640 10 client.26678 handle_command_reply: tid=0 2024-07-29T00:42:47.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.028+0000 7f646ae84740 1 client.26678 shutdown 2024-07-29T00:42:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.028+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 msgr2=0x55f6b59e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.028+0000 7f646ae84740 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59e4b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f644c0bbd70 tx=0x7f644c03b890 comp rx=0 tx=0).stop 2024-07-29T00:42:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 shutdown_connections 2024-07-29T00:42:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55f6b5a5b8a0 0x55f6b5a5dcc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6b59dffb0 0x55f6b5a53b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b59d56f0 0x55f6b59e4b90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 --2- 172.21.15.175:0/3797145756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6b59d4dd0 0x55f6b59e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f6447400640 0 client.26678 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 >> 172.21.15.175:0/3797145756 conn(0x55f6b59d1360 msgr2=0x55f6b59dd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 shutdown_connections 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 1 -- 172.21.15.175:0/3797145756 wait complete. 2024-07-29T00:42:47.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.029+0000 7f646ae84740 20 client.26678 trim_cache size 0 max 16384 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 39, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 20, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": { 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "2caef02d": { 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "version": 20, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "age_ref": 0.0, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "timeout": 90.0, 2024-07-29T00:42:47.031 INFO:teuthology.orchestra.run.smithi175.stdout: "expiration": 154.3, 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: "members": { 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: "file:/": { 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: "excluded": false, 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: "state": { 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RELEASED", 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:42:47.032 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f64480a2cf0 msgr2=0x7f64480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f64480a2cf0 0x7f64480a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f646006b0c0 tx=0x7f6458009500 comp rx=0 tx=0).stop 2024-07-29T00:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 msgr2=0x7f646019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 0x7f646019eb00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f644c004770 tx=0x7f644c003c10 comp rx=0 tx=0).stop 2024-07-29T00:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 shutdown_connections 2024-07-29T00:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6460112560 0x7f64601a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6460076d30 0x7f646019eb00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f64480a2cf0 0x7f64480a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 --2- 172.21.15.175:0/512117150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6460076410 0x7f646019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 >> 172.21.15.175:0/512117150 conn(0x7f64600ff5c0 msgr2=0x7f64601019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 shutdown_connections 2024-07-29T00:42:47.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:47.030+0000 7f6466e00640 1 -- 172.21.15.175:0/512117150 wait complete. 2024-07-29T00:42:47.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 177599 2.54 MB/sec execute 158 sec latency 31822.577 ms 2024-07-29T00:42:47.987 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2caef02d', total seconds elapsed: 38.9 2024-07-29T00:42:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:47 smithi204 ceph-mon[37368]: pgmap v191: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-07-29T00:42:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:47 smithi181 ceph-mon[36706]: pgmap v191: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-07-29T00:42:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:47 smithi175 ceph-mon[32138]: pgmap v191: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-07-29T00:42:48.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 178156 2.53 MB/sec execute 159 sec latency 27.252 ms 2024-07-29T00:42:49.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 179333 2.54 MB/sec execute 160 sec latency 20.112 ms 2024-07-29T00:42:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:49 smithi204 ceph-mon[37368]: pgmap v192: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 758 KiB/s wr, 36 op/s 2024-07-29T00:42:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:49 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:42:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:49 smithi181 ceph-mon[36706]: pgmap v192: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 758 KiB/s wr, 36 op/s 2024-07-29T00:42:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:49 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:42:50.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:49 smithi175 ceph-mon[32138]: pgmap v192: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 758 KiB/s wr, 36 op/s 2024-07-29T00:42:50.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:49 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:42:50.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 180973 2.60 MB/sec execute 161 sec latency 36.288 ms 2024-07-29T00:42:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:50 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:42:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:50 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:42:51.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:50 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:42:51.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:50 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:42:51.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:50 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:42:51.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:50 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:42:51.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:50 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:42:51.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:50 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:42:51.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:50 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:42:51.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 181353 2.59 MB/sec execute 162 sec latency 20.575 ms 2024-07-29T00:42:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:51 smithi175 ceph-mon[32138]: pgmap v193: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 758 KiB/s wr, 36 op/s 2024-07-29T00:42:52.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:51 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:42:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:51 smithi181 ceph-mon[36706]: pgmap v193: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 758 KiB/s wr, 36 op/s 2024-07-29T00:42:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:51 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:42:52.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:51 smithi204 ceph-mon[37368]: pgmap v193: 97 pgs: 97 active+clean; 117 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 758 KiB/s wr, 36 op/s 2024-07-29T00:42:52.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:51 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:42:52.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 182322 2.61 MB/sec execute 163 sec latency 22.768 ms 2024-07-29T00:42:53.902 INFO:tasks.workunit.client.0.smithi175.stdout: 1 183673 2.63 MB/sec execute 164 sec latency 21.330 ms 2024-07-29T00:42:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:53 smithi181 ceph-mon[36706]: pgmap v194: 97 pgs: 97 active+clean; 120 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-07-29T00:42:54.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:53 smithi175 ceph-mon[32138]: pgmap v194: 97 pgs: 97 active+clean; 120 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-07-29T00:42:54.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:53 smithi204 ceph-mon[37368]: pgmap v194: 97 pgs: 97 active+clean; 120 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-07-29T00:42:54.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 184734 2.66 MB/sec execute 165 sec latency 18.683 ms 2024-07-29T00:42:55.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 185207 2.65 MB/sec execute 166 sec latency 21.217 ms 2024-07-29T00:42:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:55 smithi181 ceph-mon[36706]: pgmap v195: 97 pgs: 97 active+clean; 120 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 778 KiB/s wr, 87 op/s 2024-07-29T00:42:56.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:55 smithi175 ceph-mon[32138]: pgmap v195: 97 pgs: 97 active+clean; 120 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 778 KiB/s wr, 87 op/s 2024-07-29T00:42:56.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:55 smithi204 ceph-mon[37368]: pgmap v195: 97 pgs: 97 active+clean; 120 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 778 KiB/s wr, 87 op/s 2024-07-29T00:42:56.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 186105 2.66 MB/sec execute 167 sec latency 17.440 ms 2024-07-29T00:42:57.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 187478 2.69 MB/sec execute 168 sec latency 21.639 ms 2024-07-29T00:42:57.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:42:58.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:57 smithi181 ceph-mon[36706]: pgmap v196: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-07-29T00:42:58.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:57 smithi175 ceph-mon[32138]: pgmap v196: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-07-29T00:42:58.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:57 smithi204 ceph-mon[37368]: pgmap v196: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-07-29T00:42:58.466 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:42:58.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 188368 2.71 MB/sec execute 169 sec latency 145.449 ms 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 -- 172.21.15.175:0/2133660296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 msgr2=0x7f135c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 --2- 172.21.15.175:0/2133660296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c102e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f134c009860 tx=0x7f134c02a3d0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 -- 172.21.15.175:0/2133660296 shutdown_connections 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 --2- 172.21.15.175:0/2133660296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c10be60 0x7f135c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 --2- 172.21.15.175:0/2133660296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f135c103350 0x7f135c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 --2- 172.21.15.175:0/2133660296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c102e10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 -- 172.21.15.175:0/2133660296 >> 172.21.15.175:0/2133660296 conn(0x7f135c0fc490 msgr2=0x7f135c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 -- 172.21.15.175:0/2133660296 shutdown_connections 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.104+0000 7f1362800640 1 -- 172.21.15.175:0/2133660296 wait complete. 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.105+0000 7f1362800640 1 Processor -- start 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.105+0000 7f1362800640 1 -- start start 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f1362800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f1362800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f135c103350 0x7f135c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f1362800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c10be60 0x7f135c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f1362800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111cb0 con 0x7f135c103350 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f1362800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111b30 con 0x7f135c10be60 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f1362800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111e30 con 0x7f135c102a30 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f135b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f135b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42848/0 (socket says 172.21.15.175:42848) 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f135b400640 1 -- 172.21.15.175:0/2415044456 learned_addr learned my addr 172.21.15.175:0/2415044456 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.106+0000 7f135be00640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c10be60 0x7f135c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135aa00640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f135c103350 0x7f135c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135b400640 1 -- 172.21.15.175:0/2415044456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c10be60 msgr2=0x7f135c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135b400640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c10be60 0x7f135c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135b400640 1 -- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f135c103350 msgr2=0x7f135c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135b400640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f135c103350 0x7f135c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135b400640 1 -- 172.21.15.175:0/2415044456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c19d0c0 con 0x7f135c102a30 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f135b400640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c198110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f134c009980 tx=0x7f134c0309a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.107+0000 7f1353e00640 1 -- 172.21.15.175:0/2415044456 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c040070 con 0x7f135c102a30 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.108+0000 7f1353e00640 1 -- 172.21.15.175:0/2415044456 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c005620 con 0x7f135c102a30 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.108+0000 7f1353e00640 1 -- 172.21.15.175:0/2415044456 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0049b0 con 0x7f135c102a30 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.108+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19d350 con 0x7f135c102a30 2024-07-29T00:42:59.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.108+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c19d7e0 con 0x7f135c102a30 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.111+0000 7f1353e00640 1 -- 172.21.15.175:0/2415044456 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f134c043070 con 0x7f135c102a30 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.112+0000 7f1365253740 1 Processor -- start 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.112+0000 7f1365253740 1 -- start start 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.112+0000 7f1365253740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 0x5636382cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1365253740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382cb170 0x5636382d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1365253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1365253740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56363822fad0 con 0x5636382cb170 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1365253740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5636381c7af0 con 0x5636382ca850 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1365253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56363822ba90 con 0x5636382d9f20 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 0x5636382cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 0x5636382cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54478/0 (socket says 172.21.15.175:54478) 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135be00640 1 -- 172.21.15.175:0/753444440 learned_addr learned my addr 172.21.15.175:0/753444440 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:59.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135aa00640 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1353e00640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f13400a2d70 0x7f13400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f1353e00640 1 -- 172.21.15.175:0/2415044456 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f134c0f01d0 con 0x7f135c102a30 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135aa00640 1 -- 172.21.15.175:0/753444440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 msgr2=0x5636382cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135aa00640 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 0x5636382cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135aa00640 1 -- 172.21.15.175:0/753444440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382cb170 msgr2=0x5636382d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135aa00640 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382cb170 0x5636382d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135aa00640 1 -- 172.21.15.175:0/753444440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563637fa13c0 con 0x5636382d9f20 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.113+0000 7f135be00640 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 0x5636382cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.114+0000 7f135be00640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f13400a2d70 0x7f13400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.114+0000 7f135aa00640 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382dc310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1348005fd0 tx=0x7f13480113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b140bc125cf697db server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.114+0000 7f1350c00640 1 -- 172.21.15.175:0/753444440 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348011e10 con 0x5636382d9f20 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.114+0000 7f1350c00640 1 -- 172.21.15.175:0/753444440 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1348012930 con 0x5636382d9f20 2024-07-29T00:42:59.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.114+0000 7f1350c00640 1 -- 172.21.15.175:0/753444440 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134800fb00 con 0x5636382d9f20 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 -- 172.21.15.175:0/753444440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 msgr2=0x5636382dc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382dc310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1348005fd0 tx=0x7f13480113e0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f135be00640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f13400a2d70 0x7f13400a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f13540107d0 tx=0x7f1354004a40 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 -- 172.21.15.175:0/753444440 shutdown_connections 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382dc310 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382cb170 0x5636382d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 --2- 172.21.15.175:0/753444440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382ca850 0x5636382cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.115+0000 7f1365253740 1 -- 172.21.15.175:0/753444440 >> 172.21.15.175:0/753444440 conn(0x5636382c6ec0 msgr2=0x5636382c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.116+0000 7f1365253740 1 -- 172.21.15.175:0/753444440 shutdown_connections 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.116+0000 7f1365253740 1 -- 172.21.15.175:0/753444440 wait complete. 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.116+0000 7f1365253740 1 -- start start 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.116+0000 7f1365253740 1 Processor -- start 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 0x5636382d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5636382dca90 con 0x5636382ca850 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5636381eabc0 con 0x5636382cb170 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636382dcc10 con 0x5636382d9f20 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f1365253740 10 client.0 _ll_register_callbacks cb 0x5636381d6d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f135aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 0x5636382d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f135aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 0x5636382d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54484/0 (socket says 172.21.15.175:54484) 2024-07-29T00:42:59.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.117+0000 7f135aa00640 1 -- 172.21.15.175:0/614616616 learned_addr learned my addr 172.21.15.175:0/614616616 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135b400640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135be00640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135aa00640 1 -- 172.21.15.175:0/614616616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 msgr2=0x5636382d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135aa00640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135aa00640 1 -- 172.21.15.175:0/614616616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 msgr2=0x5636382d6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135aa00640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382d6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135aa00640 1 -- 172.21.15.175:0/614616616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636382e6140 con 0x5636382cb170 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135be00640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f135aa00640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 0x5636382d74d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f134800f600 tx=0x7f1348012cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a014c1b46521cb6c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f1345600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f1345600640 1 -- 172.21.15.175:0/614616616 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348009770 con 0x5636382cb170 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1348012930 con 0x5636382cb170 2024-07-29T00:42:59.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.118+0000 7f1345600640 1 -- 172.21.15.175:0/614616616 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f1348005040 con 0x5636382cb170 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.119+0000 7f1345600640 1 -- 172.21.15.175:0/614616616 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1348033050 con 0x5636382cb170 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.119+0000 7f1365253740 10 client.26690 fetch_fsmap learned FSMap version 40 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.119+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5636382e8ad0 con 0x5636382cb170 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.119+0000 7f1345600640 1 -- 172.21.15.175:0/614616616 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1348012930 con 0x5636382cb170 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.119+0000 7f1345600640 1 -- 172.21.15.175:0/614616616 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f13480035f0 con 0x5636382cb170 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.119+0000 7f1365253740 10 client.26690 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f1365253740 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f1365253740 20 client.26690 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f1365253740 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636382ea200 0x5636382ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f1365253740 4 client.26690 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363823d040 con 0x5636382ea200 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f135b400640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636382ea200 0x5636382ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f135b400640 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636382ea200 0x5636382ec5f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3794f9fb4d34da9 server_cookie=3606ab09877407f6 in_seq=0 out_seq=0 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.120+0000 7f1345600640 10 client.26690 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.121+0000 7f1345600640 1 -- 172.21.15.175:0/614616616 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56363823d040 con 0x5636382ea200 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.121+0000 7f1345600640 10 client.26690 handle_command_reply: tid=0 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.121+0000 7f1365253740 1 client.26690 shutdown 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 msgr2=0x5636382d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 0x5636382d74d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f134800f600 tx=0x7f1348012cf0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 shutdown_connections 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636382ea200 0x5636382ec5f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382d9f20 0x5636382d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382cb170 0x5636382d74d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 --2- 172.21.15.175:0/614616616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382d6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 >> 172.21.15.175:0/614616616 conn(0x5636382c6ec0 msgr2=0x5636382ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1345600640 0 client.26690 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:59.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 shutdown_connections 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 1 -- 172.21.15.175:0/614616616 wait complete. 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.122+0000 7f1365253740 20 client.26690 trim_cache size 0 max 16384 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 Processor -- start 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 -- start start 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ab4d0 con 0x5636382ca850 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ad0d0 con 0x5636382d9f20 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.143+0000 7f1365253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1abba0 con 0x5636382cb170 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40766/0 (socket says 172.21.15.175:40766) 2024-07-29T00:42:59.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 -- 172.21.15.175:0/722052354 learned_addr learned my addr 172.21.15.175:0/722052354 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135aa00640 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135be00640 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 -- 172.21.15.175:0/722052354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 msgr2=0x5636382d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 -- 172.21.15.175:0/722052354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 msgr2=0x5636382ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135b400640 1 -- 172.21.15.175:0/722052354 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636382d5f40 con 0x5636382ca850 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.144+0000 7f135aa00640 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.145+0000 7f135b400640 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382c1b10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f134c0bc370 tx=0x7f134c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc497746a56d186 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.145+0000 7f1347e00640 1 -- 172.21.15.175:0/722052354 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c04a030 con 0x5636382ca850 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.145+0000 7f1347e00640 1 -- 172.21.15.175:0/722052354 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c0bcd70 con 0x5636382ca850 2024-07-29T00:42:59.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.145+0000 7f1347e00640 1 -- 172.21.15.175:0/722052354 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0ba970 con 0x5636382ca850 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 -- 172.21.15.175:0/722052354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 msgr2=0x5636382c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382c1b10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f134c0bc370 tx=0x7f134c000bc0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 -- 172.21.15.175:0/722052354 shutdown_connections 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 --2- 172.21.15.175:0/722052354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382c1b10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 -- 172.21.15.175:0/722052354 >> 172.21.15.175:0/722052354 conn(0x5636382c6ec0 msgr2=0x5636382d87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 -- 172.21.15.175:0/722052354 shutdown_connections 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.146+0000 7f1365253740 1 -- 172.21.15.175:0/722052354 wait complete. 2024-07-29T00:42:59.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.147+0000 7f1365253740 1 -- start start 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.147+0000 7f1365253740 1 Processor -- start 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ac940 con 0x5636382ca850 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ac270 con 0x5636382d9f20 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1a94b0 con 0x5636382cb170 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f1365253740 10 client.0 _ll_register_callbacks cb 0x7f134800f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f135b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f135be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f135be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ecef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54496/0 (socket says 172.21.15.175:54496) 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f135b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382ef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40772/0 (socket says 172.21.15.175:40772) 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f135be00640 1 -- 172.21.15.175:0/636975729 learned_addr learned my addr 172.21.15.175:0/636975729 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.148+0000 7f135aa00640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135b400640 1 -- 172.21.15.175:0/636975729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 msgr2=0x5636382ec9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135b400640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382ec9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135b400640 1 -- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 msgr2=0x5636382ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135b400640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135b400640 1 -- 172.21.15.175:0/636975729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636382ed4b0 con 0x5636382ca850 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135aa00640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135be00640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f135b400640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382ef610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5636381eac40 tx=0x7f134c0f0450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b5ec09f66115c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f1345600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f1345600640 1 -- 172.21.15.175:0/636975729 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0da060 con 0x5636382ca850 2024-07-29T00:42:59.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f1345600640 1 -- 172.21.15.175:0/636975729 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7f134c031c30 con 0x5636382ca850 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.149+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134c0bcd70 con 0x5636382ca850 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1345600640 1 -- 172.21.15.175:0/636975729 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f134c04a040 con 0x5636382ca850 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 10 client.16554 fetch_fsmap learned FSMap version 40 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5636383548e0 con 0x5636382ca850 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1345600640 1 -- 172.21.15.175:0/636975729 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c0bcd70 con 0x5636382ca850 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1345600640 1 -- 172.21.15.175:0/636975729 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f134c0b8ce0 con 0x5636382ca850 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 10 client.16554 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 20 client.16554 populate_metadata read hostname 'smithi175' 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563638354dd0 0x5636383571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 4 client.16554 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.150+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563638237550 con 0x563638354dd0 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.151+0000 7f135aa00640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563638354dd0 0x5636383571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.151+0000 7f135aa00640 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563638354dd0 0x5636383571f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca8b24b0c0024cc4 server_cookie=23e610aa9b08b8e8 in_seq=0 out_seq=0 2024-07-29T00:42:59.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.151+0000 7f1345600640 10 client.16554 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1345600640 1 -- 172.21.15.175:0/636975729 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563638237550 con 0x563638354dd0 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1345600640 10 client.16554 handle_command_reply: tid=0 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 client.16554 shutdown 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 msgr2=0x5636382ef610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382ef610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5636381eac40 tx=0x7f134c0f0450 comp rx=0 tx=0).stop 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 shutdown_connections 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563638354dd0 0x5636383571f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636382d9f20 0x5636382ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636382cb170 0x5636382ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 --2- 172.21.15.175:0/636975729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636382ca850 0x5636382ef610 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1345600640 0 client.16554 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 >> 172.21.15.175:0/636975729 conn(0x5636382c6ec0 msgr2=0x5636382e56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:59.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 shutdown_connections 2024-07-29T00:42:59.243 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:42:59.243 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:42:59.243 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:42:59.243 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:42:59.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 1 -- 172.21.15.175:0/636975729 wait complete. 2024-07-29T00:42:59.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.152+0000 7f1365253740 20 client.16554 trim_cache size 0 max 16384 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f13400a2d70 msgr2=0x7f13400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f13400a2d70 0x7f13400a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f13540107d0 tx=0x7f1354004a40 comp rx=0 tx=0).stop 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 msgr2=0x7f135c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c198110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f134c009980 tx=0x7f134c0309a0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 shutdown_connections 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c10be60 0x7f135c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f13400a2d70 0x7f13400a5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f135c103350 0x7f135c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 --2- 172.21.15.175:0/2415044456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f135c102a30 0x7f135c198110 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 >> 172.21.15.175:0/2415044456 conn(0x7f135c0fc490 msgr2=0x7f135c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 shutdown_connections 2024-07-29T00:42:59.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:42:59.153+0000 7f1362800640 1 -- 172.21.15.175:0/2415044456 wait complete. 2024-07-29T00:42:59.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 189160 2.71 MB/sec execute 170 sec latency 25.053 ms 2024-07-29T00:43:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:42:59 smithi175 ceph-mon[32138]: pgmap v197: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 870 KiB/s wr, 133 op/s 2024-07-29T00:43:00.180 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:43:00.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:43:00.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:42:59 smithi181 ceph-mon[36706]: pgmap v197: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 870 KiB/s wr, 133 op/s 2024-07-29T00:43:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:42:59 smithi204 ceph-mon[37368]: pgmap v197: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 870 KiB/s wr, 133 op/s 2024-07-29T00:43:00.675 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:00.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 190294 2.72 MB/sec execute 171 sec latency 30.911 ms 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 -- 172.21.15.175:0/2843096614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074280 msgr2=0x7fec48074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 --2- 172.21.15.175:0/2843096614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074280 0x7fec48074660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fec38009ca0 tx=0x7fec3802a950 comp rx=0 tx=0).stop 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 -- 172.21.15.175:0/2843096614 shutdown_connections 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 --2- 172.21.15.175:0/2843096614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec481126d0 0x7fec48114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 --2- 172.21.15.175:0/2843096614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074ba0 0x7fec48112190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 --2- 172.21.15.175:0/2843096614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074280 0x7fec48074660 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 -- 172.21.15.175:0/2843096614 >> 172.21.15.175:0/2843096614 conn(0x7fec480fd400 msgr2=0x7fec480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 -- 172.21.15.175:0/2843096614 shutdown_connections 2024-07-29T00:43:01.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.350+0000 7fec47200640 1 -- 172.21.15.175:0/2843096614 wait complete. 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.351+0000 7fec47200640 1 Processor -- start 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 -- start start 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074280 0x7fec4819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 0x7fec4819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec481126d0 0x7fec481a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec481180f0 con 0x7fec48074280 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fec48117f70 con 0x7fec48074ba0 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.352+0000 7fec47200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec48118270 con 0x7fec481126d0 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 0x7fec4819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 0x7fec4819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54520/0 (socket says 172.21.15.175:54520) 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 -- 172.21.15.175:0/1640335551 learned_addr learned my addr 172.21.15.175:0/1640335551 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:01.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec46800640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec481126d0 0x7fec481a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec45e00640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074280 0x7fec4819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 -- 172.21.15.175:0/1640335551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec481126d0 msgr2=0x7fec481a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec481126d0 0x7fec481a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 -- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074280 msgr2=0x7fec4819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074280 0x7fec4819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 -- 172.21.15.175:0/1640335551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec481a35d0 con 0x7fec48074ba0 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec45e00640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074280 0x7fec4819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3d400640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 0x7fec4819eb60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec3000d950 tx=0x7fec3000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:01.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.353+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30017070 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.354+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30009c30 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.354+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3000f6b0 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.354+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec481a38c0 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.354+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481ab1a0 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.355+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec48074660 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.357+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fec300109d0 con 0x7fec48074ba0 2024-07-29T00:43:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.359+0000 7fec3f400640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fec2c0a2d70 0x7fec2c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:01.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.359+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fec300cf0c0 con 0x7fec48074ba0 2024-07-29T00:43:01.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.359+0000 7fec45e00640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fec2c0a2d70 0x7fec2c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:01.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.360+0000 7fec45e00640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fec2c0a2d70 0x7fec2c0a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec38005e30 tx=0x7fec38005d80 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:01.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.362+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fec30009de0 con 0x7fec48074ba0 2024-07-29T00:43:01.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.618+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec48104b40 con 0x7fec48074ba0 2024-07-29T00:43:01.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.619+0000 7fec3f400640 1 -- 172.21.15.175:0/1640335551 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12001 (secure 0 0 0) 0x7fec3009bc60 con 0x7fec48074ba0 2024-07-29T00:43:01.619 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:43:01.619 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 40 2024-07-29T00:43:01.620 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":40,"btime":"2024-07-29T00:42:50:832623+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:42:50.665693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:43:01.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.621+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fec2c0a2d70 msgr2=0x7fec2c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:01.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.621+0000 7fec47200640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fec2c0a2d70 0x7fec2c0a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec38005e30 tx=0x7fec38005d80 comp rx=0 tx=0).stop 2024-07-29T00:43:01.622 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 msgr2=0x7fec4819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 0x7fec4819eb60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec3000d950 tx=0x7fec3000de10 comp rx=0 tx=0).stop 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 shutdown_connections 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec481126d0 0x7fec481a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48074ba0 0x7fec4819eb60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fec2c0a2d70 0x7fec2c0a5230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 --2- 172.21.15.175:0/1640335551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec48074280 0x7fec4819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 >> 172.21.15.175:0/1640335551 conn(0x7fec480fd400 msgr2=0x7fec480fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 shutdown_connections 2024-07-29T00:43:01.623 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:01.622+0000 7fec47200640 1 -- 172.21.15.175:0/1640335551 wait complete. 2024-07-29T00:43:01.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 191505 2.77 MB/sec execute 172 sec latency 64.734 ms 2024-07-29T00:43:02.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:01 smithi175 ceph-mon[32138]: pgmap v198: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 870 KiB/s wr, 133 op/s 2024-07-29T00:43:02.024 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:01 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1640335551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:02.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:01 smithi181 ceph-mon[36706]: pgmap v198: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 870 KiB/s wr, 133 op/s 2024-07-29T00:43:02.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:01 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1640335551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:02.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:01 smithi204 ceph-mon[37368]: pgmap v198: 97 pgs: 97 active+clean; 121 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 870 KiB/s wr, 133 op/s 2024-07-29T00:43:02.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:01 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1640335551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:02.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:43:02.903 INFO:tasks.workunit.client.0.smithi175.stdout: 1 191827 2.76 MB/sec execute 173 sec latency 254.230 ms 2024-07-29T00:43:03.086 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:03.758 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 -- 172.21.15.175:0/2754642609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e941016c0 msgr2=0x7f7e94101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 --2- 172.21.15.175:0/2754642609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e941016c0 0x7f7e94101b20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009930 tx=0x7f7e8002a420 comp rx=0 tx=0).stop 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 -- 172.21.15.175:0/2754642609 shutdown_connections 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 --2- 172.21.15.175:0/2754642609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e94102060 0x7f7e9410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 --2- 172.21.15.175:0/2754642609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e941016c0 0x7f7e94101b20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 --2- 172.21.15.175:0/2754642609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941076c0 0x7f7e94107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 -- 172.21.15.175:0/2754642609 >> 172.21.15.175:0/2754642609 conn(0x7f7e940fd400 msgr2=0x7f7e940ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 -- 172.21.15.175:0/2754642609 shutdown_connections 2024-07-29T00:43:03.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.758+0000 7f7e9a600640 1 -- 172.21.15.175:0/2754642609 wait complete. 2024-07-29T00:43:03.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.759+0000 7f7e9a600640 1 Processor -- start 2024-07-29T00:43:03.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.759+0000 7f7e9a600640 1 -- start start 2024-07-29T00:43:03.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e9a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941016c0 0x7f7e9419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e9a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e94102060 0x7f7e9419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e9a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e941076c0 0x7f7e9419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e9a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111b10 con 0x7f7e941016c0 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e9a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111990 con 0x7f7e941076c0 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e9a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111c90 con 0x7f7e94102060 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e99200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941016c0 0x7f7e9419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e99200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941016c0 0x7f7e9419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40796/0 (socket says 172.21.15.175:40796) 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e99200640 1 -- 172.21.15.175:0/1694892360 learned_addr learned my addr 172.21.15.175:0/1694892360 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e94102060 0x7f7e9419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.760+0000 7f7e99c00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e941076c0 0x7f7e9419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e93e00640 1 -- 172.21.15.175:0/1694892360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e941076c0 msgr2=0x7f7e9419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e941076c0 0x7f7e9419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e93e00640 1 -- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941016c0 msgr2=0x7f7e9419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941016c0 0x7f7e9419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e93e00640 1 -- 172.21.15.175:0/1694892360 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9419f180 con 0x7f7e94102060 2024-07-29T00:43:03.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e94102060 0x7f7e9419a710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e8002a940 tx=0x7f7e80003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:03.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e91600640 1 -- 172.21.15.175:0/1694892360 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8003f070 con 0x7f7e94102060 2024-07-29T00:43:03.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e91600640 1 -- 172.21.15.175:0/1694892360 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e80031c90 con 0x7f7e94102060 2024-07-29T00:43:03.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e91600640 1 -- 172.21.15.175:0/1694892360 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80035030 con 0x7f7e94102060 2024-07-29T00:43:03.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9419f410 con 0x7f7e94102060 2024-07-29T00:43:03.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.761+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9419f8a0 con 0x7f7e94102060 2024-07-29T00:43:03.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.765+0000 7f7e91600640 1 -- 172.21.15.175:0/1694892360 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7e80042070 con 0x7f7e94102060 2024-07-29T00:43:03.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.765+0000 7f7e9e7c3740 1 Processor -- start 2024-07-29T00:43:03.765 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.765+0000 7f7e9e7c3740 1 -- start start 2024-07-29T00:43:03.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d8a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.766 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d97d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f868cf0ab0 con 0x55f868d95910 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f868c88a80 con 0x55f868d8c0e0 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f868ceca70 con 0x55f868d8ab70 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d8a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d8afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42914/0 (socket says 172.21.15.175:42914) 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 -- 172.21.15.175:0/1869648267 learned_addr learned my addr 172.21.15.175:0/1869648267 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d97d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 -- 172.21.15.175:0/1869648267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 msgr2=0x55f868d8a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d8a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 -- 172.21.15.175:0/1869648267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 msgr2=0x55f868d97d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d97d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99200640 1 -- 172.21.15.175:0/1869648267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f868a623c0 con 0x55f868d8ab70 2024-07-29T00:43:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.766+0000 7f7e99c00640 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d8a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.767+0000 7f7e99200640 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d8afb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009d90 tx=0x7f7e88011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd2e3b5df7fb9ab server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:03.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.767+0000 7f7e86000640 1 -- 172.21.15.175:0/1869648267 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8801e070 con 0x55f868d8ab70 2024-07-29T00:43:03.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.767+0000 7f7e86000640 1 -- 172.21.15.175:0/1869648267 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e88019b10 con 0x55f868d8ab70 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.767+0000 7f7e86000640 1 -- 172.21.15.175:0/1869648267 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800fcb0 con 0x55f868d8ab70 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.767+0000 7f7e91600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7e7c0a2d70 0x7f7e7c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.767+0000 7f7e91600640 1 -- 172.21.15.175:0/1694892360 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f7e800ef9a0 con 0x7f7e94102060 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e99c00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7e7c0a2d70 0x7f7e7c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1869648267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 msgr2=0x55f868d8afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d8afb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009d90 tx=0x7f7e88011810 comp rx=0 tx=0).stop 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1869648267 shutdown_connections 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d97d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d8afb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1869648267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d8a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1869648267 >> 172.21.15.175:0/1869648267 conn(0x55f868d53950 msgr2=0x55f868d87200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1869648267 shutdown_connections 2024-07-29T00:43:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1869648267 wait complete. 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.768+0000 7f7e99c00640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7e7c0a2d70 0x7f7e7c0a5230 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009fd0 tx=0x7f7e8c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 -- start start 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 Processor -- start 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d92890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d92dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d93310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f868d986b0 con 0x55f868d95910 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f868cabc10 con 0x55f868d8c0e0 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f868d98830 con 0x55f868d8ab70 2024-07-29T00:43:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.769+0000 7f7e9e7c3740 10 client.0 _ll_register_callbacks cb 0x55f868c74a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d92dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d92dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54558/0 (socket says 172.21.15.175:54558) 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e93e00640 1 -- 172.21.15.175:0/1287300493 learned_addr learned my addr 172.21.15.175:0/1287300493 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99c00640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d93310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99200640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d92890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99200640 1 -- 172.21.15.175:0/1287300493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 msgr2=0x55f868d92dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99200640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d92dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99200640 1 -- 172.21.15.175:0/1287300493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 msgr2=0x55f868d93310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99200640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d93310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99200640 1 -- 172.21.15.175:0/1287300493 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f868caf1e0 con 0x55f868d8ab70 2024-07-29T00:43:03.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e99c00640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d93310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.770+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d92dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e99200640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d92890 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7e8801c040 tx=0x7f7e88004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ef65f242620cf8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e7aa00640 1 -- 172.21.15.175:0/1287300493 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88027020 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e88019b10 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e7aa00640 1 -- 172.21.15.175:0/1287300493 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f7e8800fbb0 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e7aa00640 1 -- 172.21.15.175:0/1287300493 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7e8802f050 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e9e7c3740 10 client.26668 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f868da4450 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.771+0000 7f7e7aa00640 1 -- 172.21.15.175:0/1287300493 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e88019b10 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e7aa00640 1 -- 172.21.15.175:0/1287300493 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7e880035f0 con 0x55f868d8ab70 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e9e7c3740 10 client.26668 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e9e7c3740 10 client.26668 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e9e7c3740 20 client.26668 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868da5800 0x55f868da7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e9e7c3740 4 client.26668 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f868cfe020 con 0x55f868da5800 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.772+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868da5800 0x55f868da7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.773+0000 7f7e93e00640 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868da5800 0x55f868da7bf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a283aa0f5a178939 server_cookie=c3b801693bec7c99 in_seq=0 out_seq=0 2024-07-29T00:43:03.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.773+0000 7f7e7aa00640 10 client.26668 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.773+0000 7f7e7aa00640 1 -- 172.21.15.175:0/1287300493 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f868cfe020 con 0x55f868da5800 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.773+0000 7f7e7aa00640 10 client.26668 handle_command_reply: tid=0 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.773+0000 7f7e9e7c3740 1 client.26668 shutdown 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 msgr2=0x55f868d92890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d92890 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7e8801c040 tx=0x7f7e88004270 comp rx=0 tx=0).stop 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 shutdown_connections 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d93310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868da5800 0x55f868da7bf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d8c0e0 0x55f868d92dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/1287300493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8ab70 0x55f868d92890 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 >> 172.21.15.175:0/1287300493 conn(0x55f868d53950 msgr2=0x55f868d8cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e7aa00640 0 client.26668 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 shutdown_connections 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/1287300493 wait complete. 2024-07-29T00:43:03.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.774+0000 7f7e9e7c3740 20 client.26668 trim_cache size 0 max 16384 2024-07-29T00:43:03.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 Processor -- start 2024-07-29T00:43:03.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 -- start start 2024-07-29T00:43:03.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d8c0e0 0x55f868d91740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d95910 0x55f868dad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 0x55f868dad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941af8a0 con 0x55f868d8c0e0 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b14a0 con 0x55f868d95910 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941aff70 con 0x55f868d874c0 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e99c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 0x55f868dad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d95910 0x55f868dad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e99200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d8c0e0 0x55f868d91740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e99c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 0x55f868dad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42930/0 (socket says 172.21.15.175:42930) 2024-07-29T00:43:03.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.791+0000 7f7e99c00640 1 -- 172.21.15.175:0/2569712454 learned_addr learned my addr 172.21.15.175:0/2569712454 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e93e00640 1 -- 172.21.15.175:0/2569712454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 msgr2=0x55f868dad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e93e00640 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 0x55f868dad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e93e00640 1 -- 172.21.15.175:0/2569712454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d8c0e0 msgr2=0x55f868d91740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e93e00640 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d8c0e0 0x55f868d91740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e93e00640 1 -- 172.21.15.175:0/2569712454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f868d8d6c0 con 0x55f868d95910 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e99c00640 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 0x55f868dad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e93e00640 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d95910 0x55f868dad330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e800bbb90 tx=0x7f7e80005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b88dfc4002510f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.792+0000 7f7e85600640 1 -- 172.21.15.175:0/2569712454 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800f1e60 con 0x55f868d95910 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.793+0000 7f7e85600640 1 -- 172.21.15.175:0/2569712454 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e800bbe90 con 0x55f868d95910 2024-07-29T00:43:03.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.793+0000 7f7e85600640 1 -- 172.21.15.175:0/2569712454 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80049040 con 0x55f868d95910 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.793+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/2569712454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d95910 msgr2=0x55f868dad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.793+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d95910 0x55f868dad330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e800bbb90 tx=0x7f7e80005d50 comp rx=0 tx=0).stop 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/2569712454 shutdown_connections 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d874c0 0x55f868dad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d95910 0x55f868dad330 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/2569712454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d8c0e0 0x55f868d91740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/2569712454 >> 172.21.15.175:0/2569712454 conn(0x55f868d53950 msgr2=0x55f868d91030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/2569712454 shutdown_connections 2024-07-29T00:43:03.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/2569712454 wait complete. 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.794+0000 7f7e9e7c3740 1 -- start start 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 Processor -- start 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 0x55f868d98ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8c0e0 0x55f868d98ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d99530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b0d10 con 0x55f868d95910 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b0640 con 0x55f868d874c0 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ad8a0 con 0x55f868d8c0e0 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e9e7c3740 10 client.0 _ll_register_callbacks cb 0x7f7e8801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e99200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 0x55f868d98ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e99200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 0x55f868d98ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54576/0 (socket says 172.21.15.175:54576) 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8c0e0 0x55f868d98ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.795+0000 7f7e99200640 1 -- 172.21.15.175:0/3095814482 learned_addr learned my addr 172.21.15.175:0/3095814482 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99c00640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d99530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99200640 1 -- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8c0e0 msgr2=0x55f868d98ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99200640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8c0e0 0x55f868d98ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99200640 1 -- 172.21.15.175:0/3095814482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 msgr2=0x55f868d99530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99200640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d99530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99200640 1 -- 172.21.15.175:0/3095814482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f868e0e540 con 0x55f868d874c0 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99c00640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d99530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e99200640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 0x55f868d98ab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000c00 tx=0x7f7e88009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760a60af6e7b828e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e7aa00640 1 -- 172.21.15.175:0/3095814482 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88027040 con 0x55f868d874c0 2024-07-29T00:43:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e7aa00640 1 -- 172.21.15.175:0/3095814482 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f7e88003c00 con 0x55f868d874c0 2024-07-29T00:43:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.796+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e800bbe90 con 0x55f868d874c0 2024-07-29T00:43:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e7aa00640 1 -- 172.21.15.175:0/3095814482 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7e88040050 con 0x55f868d874c0 2024-07-29T00:43:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 10 client.26720 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f868e0f840 con 0x55f868d874c0 2024-07-29T00:43:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e7aa00640 1 -- 172.21.15.175:0/3095814482 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e800bbe90 con 0x55f868d874c0 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e7aa00640 1 -- 172.21.15.175:0/3095814482 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7e8803b030 con 0x55f868d874c0 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 10 client.26720 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 10 client.26720 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 20 client.26720 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868e0fe90 0x55f868e122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 4 client.26720 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.797+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f868cf8530 con 0x55f868e0fe90 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e93e00640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868e0fe90 0x55f868e122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e93e00640 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868e0fe90 0x55f868e122a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed318c206b0ae855 server_cookie=a173f8a983c7504c in_seq=0 out_seq=0 2024-07-29T00:43:03.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e7aa00640 10 client.26720 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e7aa00640 1 -- 172.21.15.175:0/3095814482 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f868cf8530 con 0x55f868e0fe90 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e7aa00640 10 client.26720 handle_command_reply: tid=0 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e9e7c3740 1 client.26720 shutdown 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 msgr2=0x55f868d98ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.798+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 0x55f868d98ab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000c00 tx=0x7f7e88009860 comp rx=0 tx=0).stop 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 shutdown_connections 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f868d95910 0x55f868d99530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f868d8c0e0 0x55f868d98ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55f868e0fe90 0x55f868e122a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 --2- 172.21.15.175:0/3095814482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f868d874c0 0x55f868d98ab0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 >> 172.21.15.175:0/3095814482 conn(0x55f868d53950 msgr2=0x55f868d93050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e7aa00640 0 client.26720 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 shutdown_connections 2024-07-29T00:43:03.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 1 -- 172.21.15.175:0/3095814482 wait complete. 2024-07-29T00:43:03.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.799+0000 7f7e9e7c3740 20 client.26720 trim_cache size 0 max 16384 2024-07-29T00:43:03.803 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7e7c0a2d70 msgr2=0x7f7e7c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7e7c0a2d70 0x7f7e7c0a5230 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009fd0 tx=0x7f7e8c009500 comp rx=0 tx=0).stop 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e94102060 msgr2=0x7f7e9419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e94102060 0x7f7e9419a710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e8002a940 tx=0x7f7e80003b20 comp rx=0 tx=0).stop 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 shutdown_connections 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e941076c0 0x7f7e9419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e94102060 0x7f7e9419a710 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e8002a940 tx=0x7f7e80003b20 comp rx=0 tx=0).stop 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7e7c0a2d70 0x7f7e7c0a5230 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 --2- 172.21.15.175:0/1694892360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e941016c0 0x7f7e9419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.800+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 >> 172.21.15.175:0/1694892360 conn(0x7f7e940fd400 msgr2=0x7f7e9410ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:03.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.801+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 shutdown_connections 2024-07-29T00:43:03.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:03.801+0000 7f7e9a600640 1 -- 172.21.15.175:0/1694892360 wait complete. 2024-07-29T00:43:03.904 INFO:tasks.workunit.client.0.smithi175.stdout: 1 192605 2.76 MB/sec execute 174 sec latency 23.044 ms 2024-07-29T00:43:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:03 smithi181 ceph-mon[36706]: pgmap v199: 97 pgs: 97 active+clean; 125 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-07-29T00:43:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:03 smithi181 ceph-mon[36706]: osdmap e93: 12 total, 12 up, 12 in 2024-07-29T00:43:04.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:03 smithi175 ceph-mon[32138]: pgmap v199: 97 pgs: 97 active+clean; 125 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-07-29T00:43:04.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:03 smithi175 ceph-mon[32138]: osdmap e93: 12 total, 12 up, 12 in 2024-07-29T00:43:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:03 smithi204 ceph-mon[37368]: pgmap v199: 97 pgs: 97 active+clean; 125 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-07-29T00:43:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:03 smithi204 ceph-mon[37368]: osdmap e93: 12 total, 12 up, 12 in 2024-07-29T00:43:04.341 INFO:tasks.workunit.client.0.smithi175.stdout:[193072] write failed on handle 13807 (Resource temporarily unavailable) 2024-07-29T00:43:04.342 INFO:tasks.workunit.client.0.smithi175.stdout:Child failed with status 1 2024-07-29T00:43:04.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:43:04.345 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-07-29T00:43:04.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-29T00:43:04.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:43:05.248 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:05.594 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi175 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba82afec3628bf830c0830558bf3d3c3aea66701 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-07-29T00:43:05.988 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6f96dc749b3d46c6b8dff7b3d425de51 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi175 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba82afec3628bf830c0830558bf3d3c3aea66701 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-07-29T00:43:05.991 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-07-29T00:43:06.001 INFO:tasks.quiescer:joining Quiescers 2024-07-29T00:43:06.002 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-07-29T00:43:06.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:43:06.007 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:05 smithi175 ceph-mon[32138]: pgmap v201: 97 pgs: 97 active+clean; 125 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1018 KiB/s wr, 201 op/s 2024-07-29T00:43:06.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 -- 172.21.15.175:0/3694865696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 msgr2=0x7f66e80767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 --2- 172.21.15.175:0/3694865696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e80767f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d802a410 comp rx=0 tx=0).stop 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 -- 172.21.15.175:0/3694865696 shutdown_connections 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 --2- 172.21.15.175:0/3694865696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8112720 0x7f66e8114b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 --2- 172.21.15.175:0/3694865696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8076d30 0x7f66e81121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 --2- 172.21.15.175:0/3694865696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e80767f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.960+0000 7f66ede00640 1 -- 172.21.15.175:0/3694865696 >> 172.21.15.175:0/3694865696 conn(0x7f66e80ff780 msgr2=0x7f66e8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.961+0000 7f66ede00640 1 -- 172.21.15.175:0/3694865696 shutdown_connections 2024-07-29T00:43:06.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.961+0000 7f66ede00640 1 -- 172.21.15.175:0/3694865696 wait complete. 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 Processor -- start 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 -- start start 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8076d30 0x7f66e81a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 0x7f66e819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8118300 con 0x7f66e8076410 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8118180 con 0x7f66e8076d30 2024-07-29T00:43:06.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.962+0000 7f66ede00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8118480 con 0x7f66e8112720 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 0x7f66e819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 0x7f66e819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42944/0 (socket says 172.21.15.175:42944) 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 -- 172.21.15.175:0/2165029768 learned_addr learned my addr 172.21.15.175:0/2165029768 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66de000640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8076d30 0x7f66e81a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 -- 172.21.15.175:0/2165029768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8076d30 msgr2=0x7f66e81a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8076d30 0x7f66e81a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 -- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 msgr2=0x7f66e81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e81a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e7400640 1 -- 172.21.15.175:0/2165029768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e819ed00 con 0x7f66e8112720 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66e6a00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.963+0000 7f66de000640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8076d30 0x7f66e81a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.964+0000 7f66e7400640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 0x7f66e819e5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66e000ec50 tx=0x7f66e000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.964+0000 7f66eca00640 1 -- 172.21.15.175:0/2165029768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0003b70 con 0x7f66e8112720 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.964+0000 7f66eca00640 1 -- 172.21.15.175:0/2165029768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e0003d10 con 0x7f66e8112720 2024-07-29T00:43:06.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.964+0000 7f66eca00640 1 -- 172.21.15.175:0/2165029768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00109e0 con 0x7f66e8112720 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.964+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e819eff0 con 0x7f66e8112720 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.965+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e81ab2e0 con 0x7f66e8112720 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.968+0000 7f66f0a70740 1 Processor -- start 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.969+0000 7f66eca00640 1 -- 172.21.15.175:0/2165029768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f66e0003ec0 con 0x7f66e8112720 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.969+0000 7f66f0a70740 1 -- start start 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.969+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb4c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.969+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb4cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb59b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66f0a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55830bab2ab0 con 0x55830bb4def0 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66f0a70740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55830ba4aa80 con 0x55830bb57720 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66f0a70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55830baaea70 con 0x55830bb4c980 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb4cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb4cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42956/0 (socket says 172.21.15.175:42956) 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 -- 172.21.15.175:0/1800722578 learned_addr learned my addr 172.21.15.175:0/1800722578 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66de000640 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb4c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66de000640 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb59b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 -- 172.21.15.175:0/1800722578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 msgr2=0x55830bb59b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb59b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 -- 172.21.15.175:0/1800722578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 msgr2=0x55830bb4c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb4c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.970+0000 7f66e6a00640 1 -- 172.21.15.175:0/1800722578 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830b8243c0 con 0x55830bb4c980 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66e6a00640 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb4cdc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d8005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35c199ae9b9b7261 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66eca00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f66cc0a2d70 0x7f66cc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66d7e00640 1 -- 172.21.15.175:0/1800722578 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8035040 con 0x55830bb4c980 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66d7e00640 1 -- 172.21.15.175:0/1800722578 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d8004760 con 0x55830bb4c980 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66d7e00640 1 -- 172.21.15.175:0/1800722578 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8003910 con 0x55830bb4c980 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66eca00640 1 -- 172.21.15.175:0/2165029768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f66e00cfa60 con 0x7f66e8112720 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.971+0000 7f66de000640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f66cc0a2d70 0x7f66cc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 -- 172.21.15.175:0/1800722578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 msgr2=0x55830bb4cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb4cdc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d8005de0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66de000640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f66cc0a2d70 0x7f66cc0a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009710 tx=0x7f66d0005850 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:06.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 -- 172.21.15.175:0/1800722578 shutdown_connections 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb59b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb4cdc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1800722578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb4c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 -- 172.21.15.175:0/1800722578 >> 172.21.15.175:0/1800722578 conn(0x55830bb15950 msgr2=0x55830bb49010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 -- 172.21.15.175:0/1800722578 shutdown_connections 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.972+0000 7f66f0a70740 1 -- 172.21.15.175:0/1800722578 wait complete. 2024-07-29T00:43:06.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 -- start start 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 Processor -- start 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4def0 0x55830bb54bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb57720 0x55830bb55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55830bb5a460 con 0x55830bb57720 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55830ba6dc10 con 0x55830bb4def0 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.973+0000 7f66f0a70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55830bb5a5e0 con 0x55830bb4c980 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66f0a70740 10 client.0 _ll_register_callbacks cb 0x55830ba36a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:06.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66de000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4def0 0x55830bb54bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42966/0 (socket says 172.21.15.175:42966) 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66e6a00640 1 -- 172.21.15.175:0/333045395 learned_addr learned my addr 172.21.15.175:0/333045395 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66e7400640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb57720 0x55830bb55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66de000640 1 -- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 msgr2=0x55830bb546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66de000640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66de000640 1 -- 172.21.15.175:0/333045395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb57720 msgr2=0x55830bb55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66de000640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb57720 0x55830bb55130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66de000640 1 -- 172.21.15.175:0/333045395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830ba711e0 con 0x55830bb4def0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.974+0000 7f66e6a00640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66de000640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4def0 0x55830bb54bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f66d00058c0 tx=0x7f66d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edbc4e3f147ca134 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66d4c00640 1 -- 172.21.15.175:0/333045395 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00044e0 con 0x55830bb4def0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d8004760 con 0x55830bb4def0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66d4c00640 1 -- 172.21.15.175:0/333045395 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f66d0009040 con 0x55830bb4def0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66d4c00640 1 -- 172.21.15.175:0/333045395 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f66d0003d50 con 0x55830bb4def0 2024-07-29T00:43:06.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66f0a70740 10 client.26732 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.975+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55830bb662a0 con 0x55830bb4def0 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66d4c00640 1 -- 172.21.15.175:0/333045395 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d8004760 con 0x55830bb4def0 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66d4c00640 1 -- 172.21.15.175:0/333045395 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f66d002a020 con 0x55830bb4def0 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66f0a70740 10 client.26732 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66f0a70740 10 client.26732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66f0a70740 20 client.26732 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66f0a70740 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bb676b0 0x55830bb69aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66f0a70740 4 client.26732 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55830bac0020 con 0x55830bb676b0 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.976+0000 7f66e6a00640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bb676b0 0x55830bb69aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.977+0000 7f66e6a00640 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bb676b0 0x55830bb69aa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7401beabed14e0f5 server_cookie=8fc8ee7fe222d008 in_seq=0 out_seq=0 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.977+0000 7f66d4c00640 10 client.26732 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66d4c00640 1 -- 172.21.15.175:0/333045395 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55830bac0020 con 0x55830bb676b0 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66d4c00640 10 client.26732 handle_command_reply: tid=0 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 client.26732 shutdown 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4def0 msgr2=0x55830bb54bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4def0 0x55830bb54bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f66d00058c0 tx=0x7f66d0005f90 comp rx=0 tx=0).stop 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 shutdown_connections 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bb676b0 0x55830bb69aa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb57720 0x55830bb55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4def0 0x55830bb54bf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66d4c00640 0 client.26732 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 --2- 172.21.15.175:0/333045395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4c980 0x55830bb546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 >> 172.21.15.175:0/333045395 conn(0x55830bb15950 msgr2=0x55830bb4ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 shutdown_connections 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.978+0000 7f66f0a70740 1 -- 172.21.15.175:0/333045395 wait complete. 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.979+0000 7f66f0a70740 20 client.26732 trim_cache size 0 max 16384 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 Processor -- start 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 -- start start 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 0x55830bb491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4def0 0x55830bb53f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81b3e60 con 0x55830bb4c980 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81b5a60 con 0x55830bb57720 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66f0a70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81b4530 con 0x55830bb4def0 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.993+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 0x55830bb491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 0x55830bb491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40876/0 (socket says 172.21.15.175:40876) 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 -- 172.21.15.175:0/893509112 learned_addr learned my addr 172.21.15.175:0/893509112 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e7400640 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66de000640 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4def0 0x55830bb53f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 -- 172.21.15.175:0/893509112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4def0 msgr2=0x55830bb53f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4def0 0x55830bb53f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 -- 172.21.15.175:0/893509112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 msgr2=0x55830bb6f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb6f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 -- 172.21.15.175:0/893509112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830bb4f4d0 con 0x55830bb4c980 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.994+0000 7f66e6a00640 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 0x55830bb491c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d80044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94893e712adb733 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66d7400640 1 -- 172.21.15.175:0/893509112 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8035040 con 0x55830bb4c980 2024-07-29T00:43:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66d7400640 1 -- 172.21.15.175:0/893509112 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d8004cb0 con 0x55830bb4c980 2024-07-29T00:43:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66d7400640 1 -- 172.21.15.175:0/893509112 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8003b40 con 0x55830bb4c980 2024-07-29T00:43:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66f0a70740 1 -- 172.21.15.175:0/893509112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 msgr2=0x55830bb491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66f0a70740 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 0x55830bb491c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d80044f0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66f0a70740 1 -- 172.21.15.175:0/893509112 shutdown_connections 2024-07-29T00:43:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66f0a70740 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb57720 0x55830bb6f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.995+0000 7f66f0a70740 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb4def0 0x55830bb53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.996+0000 7f66f0a70740 1 --2- 172.21.15.175:0/893509112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4c980 0x55830bb491c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.996+0000 7f66f0a70740 1 -- 172.21.15.175:0/893509112 >> 172.21.15.175:0/893509112 conn(0x55830bb15950 msgr2=0x55830bb48ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.996+0000 7f66f0a70740 1 -- 172.21.15.175:0/893509112 shutdown_connections 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.996+0000 7f66f0a70740 1 -- 172.21.15.175:0/893509112 wait complete. 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.996+0000 7f66f0a70740 1 -- start start 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.996+0000 7f66f0a70740 1 Processor -- start 2024-07-29T00:43:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 0x55830bb5f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb57720 0x55830bb50a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81b52d0 con 0x55830bb4def0 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81b4c00 con 0x55830bb4c980 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81b1e60 con 0x55830bb57720 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66f0a70740 10 client.0 _ll_register_callbacks cb 0x7f66d0009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66de000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 0x55830bb5f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 0x55830bb5f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54630/0 (socket says 172.21.15.175:54630) 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 -- 172.21.15.175:0/1361695423 learned_addr learned my addr 172.21.15.175:0/1361695423 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e7400640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb57720 0x55830bb50a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 -- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb57720 msgr2=0x55830bb50a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb57720 0x55830bb50a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 -- 172.21.15.175:0/1361695423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 msgr2=0x55830bb5fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb5fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.997+0000 7f66e6a00640 1 -- 172.21.15.175:0/1361695423 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830bb50fc0 con 0x55830bb4c980 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66de000640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66e6a00640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 0x55830bb5f870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009430 tx=0x7f66d8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6722c6969bae1d0a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66d4c00640 1 -- 172.21.15.175:0/1361695423 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d80039d0 con 0x55830bb4c980 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66d4c00640 1 -- 172.21.15.175:0/1361695423 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f66d8003b70 con 0x55830bb4c980 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d8004cb0 con 0x55830bb4c980 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66d4c00640 1 -- 172.21.15.175:0/1361695423 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f66d8003d90 con 0x55830bb4c980 2024-07-29T00:43:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66f0a70740 10 client.26738 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.998+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55830bbd19d0 con 0x55830bb4c980 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66d4c00640 1 -- 172.21.15.175:0/1361695423 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d8004cb0 con 0x55830bb4c980 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66d4c00640 1 -- 172.21.15.175:0/1361695423 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f66d80317b0 con 0x55830bb4c980 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66f0a70740 10 client.26738 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66f0a70740 10 client.26738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66f0a70740 20 client.26738 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bbd1c50 0x55830bbd4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66f0a70740 4 client.26738 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55830baba530 con 0x55830bbd1c50 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:05.999+0000 7f66de000640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bbd1c50 0x55830bbd4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66de000640 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bbd1c50 0x55830bbd4040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa2abf41303da02e server_cookie=5e380dc6eb4237c4 in_seq=0 out_seq=0 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66d4c00640 10 client.26738 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66d4c00640 1 -- 172.21.15.175:0/1361695423 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55830baba530 con 0x55830bbd1c50 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66d4c00640 10 client.26738 handle_command_reply: tid=0 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 client.26738 shutdown 2024-07-29T00:43:06.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 msgr2=0x55830bb5f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 0x55830bb5f870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009430 tx=0x7f66d8005fd0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 shutdown_connections 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55830bbd1c50 0x55830bbd4040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55830bb57720 0x55830bb50a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830bb4def0 0x55830bb5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66d4c00640 0 client.26738 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 --2- 172.21.15.175:0/1361695423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830bb4c980 0x55830bb5f870 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.000+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 >> 172.21.15.175:0/1361695423 conn(0x55830bb15950 msgr2=0x55830bb4ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:06.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 shutdown_connections 2024-07-29T00:43:06.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66f0a70740 1 -- 172.21.15.175:0/1361695423 wait complete. 2024-07-29T00:43:06.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66f0a70740 20 client.26738 trim_cache size 0 max 16384 2024-07-29T00:43:06.053 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f66cc0a2d70 msgr2=0x7f66cc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66ede00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f66cc0a2d70 0x7f66cc0a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009710 tx=0x7f66d0005850 comp rx=0 tx=0).stop 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 msgr2=0x7f66e819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.001+0000 7f66ede00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 0x7f66e819e5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66e000ec50 tx=0x7f66e000c6a0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 shutdown_connections 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e8112720 0x7f66e819e5d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66e8076d30 0x7f66e81a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f66cc0a2d70 0x7f66cc0a5230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 --2- 172.21.15.175:0/2165029768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e8076410 0x7f66e81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:06.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 >> 172.21.15.175:0/2165029768 conn(0x7f66e80ff780 msgr2=0x7f66e8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:06.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 shutdown_connections 2024-07-29T00:43:06.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:06.002+0000 7f66ede00640 1 -- 172.21.15.175:0/2165029768 wait complete. 2024-07-29T00:43:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:05 smithi181 ceph-mon[36706]: pgmap v201: 97 pgs: 97 active+clean; 125 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1018 KiB/s wr, 201 op/s 2024-07-29T00:43:06.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:05 smithi204 ceph-mon[37368]: pgmap v201: 97 pgs: 97 active+clean; 125 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1018 KiB/s wr, 201 op/s 2024-07-29T00:43:06.955 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:43:07.034 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:07.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.729+0000 7f0455000640 1 -- 172.21.15.175:0/1577734945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f045010dee0 msgr2=0x7f0450110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.729+0000 7f0455000640 1 --2- 172.21.15.175:0/1577734945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f045010dee0 0x7f0450110330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0448009ca0 tx=0x7f044802a950 comp rx=0 tx=0).stop 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 -- 172.21.15.175:0/1577734945 shutdown_connections 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 --2- 172.21.15.175:0/1577734945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f045010dee0 0x7f0450110330 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 --2- 172.21.15.175:0/1577734945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0450074c30 0x7f045010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 --2- 172.21.15.175:0/1577734945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0450074310 0x7f04500746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 -- 172.21.15.175:0/1577734945 >> 172.21.15.175:0/1577734945 conn(0x7f04500fd3a0 msgr2=0x7f04500ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 -- 172.21.15.175:0/1577734945 shutdown_connections 2024-07-29T00:43:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.730+0000 7f0455000640 1 -- 172.21.15.175:0/1577734945 wait complete. 2024-07-29T00:43:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 Processor -- start 2024-07-29T00:43:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 -- start start 2024-07-29T00:43:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 0x7f045019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0450074c30 0x7f045019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f045010dee0 0x7f045019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0450113ce0 con 0x7f0450074c30 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0450113b60 con 0x7f0450074310 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.731+0000 7f0455000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0450113e60 con 0x7f045010dee0 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 0x7f045019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0450074c30 0x7f045019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 0x7f045019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:54650/0 (socket says 172.21.15.175:54650) 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 -- 172.21.15.175:0/1602559992 learned_addr learned my addr 172.21.15.175:0/1602559992 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044fe00640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f045010dee0 0x7f045019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 -- 172.21.15.175:0/1602559992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f045010dee0 msgr2=0x7f045019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f045010dee0 0x7f045019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 -- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0450074c30 msgr2=0x7f045019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0450074c30 0x7f045019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 -- 172.21.15.175:0/1602559992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045019f170 con 0x7f0450074310 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.732+0000 7f044f400640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 0x7f045019a1c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.733+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000cd80 con 0x7f0450074310 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.733+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440005500 con 0x7f0450074310 2024-07-29T00:43:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.733+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045019f460 con 0x7f0450074310 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.733+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440010690 con 0x7f0450074310 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.733+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501a6d40 con 0x7f0450074310 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.734+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04500746f0 con 0x7f0450074310 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.736+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f04400046c0 con 0x7f0450074310 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.737+0000 7f0447e00640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04340a2d70 0x7f04340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.738+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f04400cfa90 con 0x7f0450074310 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.738+0000 7f044ea00640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04340a2d70 0x7f04340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.738+0000 7f044ea00640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04340a2d70 0x7f04340a5230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04380095f0 tx=0x7f0438005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:07.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.739+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0440053df0 con 0x7f0450074310 2024-07-29T00:43:07.854 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:07.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.977+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f045019b520 con 0x7f0450074310 2024-07-29T00:43:07.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.979+0000 7f0447e00640 1 -- 172.21.15.175:0/1602559992 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12001 (secure 0 0 0) 0x7f044009c5d0 con 0x7f0450074310 2024-07-29T00:43:07.979 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:43:07.980 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":40,"btime":"2024-07-29T00:42:50:832623+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:42:50.665693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:43:07.980 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 40 2024-07-29T00:43:07.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.981+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04340a2d70 msgr2=0x7f04340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:07.982 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:07 smithi175 ceph-mon[32138]: pgmap v202: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:07.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.981+0000 7f0455000640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04340a2d70 0x7f04340a5230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04380095f0 tx=0x7f0438005f90 comp rx=0 tx=0).stop 2024-07-29T00:43:07.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.981+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 msgr2=0x7f045019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:07.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.981+0000 7f0455000640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 0x7f045019a1c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000edf0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 shutdown_connections 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f045010dee0 0x7f045019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f04340a2d70 0x7f04340a5230 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0450074c30 0x7f045019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 --2- 172.21.15.175:0/1602559992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0450074310 0x7f045019a1c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 >> 172.21.15.175:0/1602559992 conn(0x7f04500fd3a0 msgr2=0x7f04500fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 shutdown_connections 2024-07-29T00:43:07.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:07.982+0000 7f0455000640 1 -- 172.21.15.175:0/1602559992 wait complete. 2024-07-29T00:43:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:07 smithi181 ceph-mon[36706]: pgmap v202: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:07 smithi204 ceph-mon[37368]: pgmap v202: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:09.059 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --cancel --all' 2024-07-29T00:43:09.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- timeout 105 ceph tell mds.24539 quiesce db --cancel --all 2024-07-29T00:43:09.233 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:08 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1602559992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:09.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:08 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1602559992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:08 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1602559992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:09.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 -- 172.21.15.175:0/1651636159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 msgr2=0x7f604c10b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 --2- 172.21.15.175:0/1651636159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c10b890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603802a480 comp rx=0 tx=0).stop 2024-07-29T00:43:09.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 -- 172.21.15.175:0/1651636159 shutdown_connections 2024-07-29T00:43:09.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 --2- 172.21.15.175:0/1651636159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f604c10be60 0x7f604c10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 --2- 172.21.15.175:0/1651636159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c10b890 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 --2- 172.21.15.175:0/1651636159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 0x7f604c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 -- 172.21.15.175:0/1651636159 >> 172.21.15.175:0/1651636159 conn(0x7f604c0fc480 msgr2=0x7f604c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:09.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 -- 172.21.15.175:0/1651636159 shutdown_connections 2024-07-29T00:43:09.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.360+0000 7f6054400640 1 -- 172.21.15.175:0/1651636159 wait complete. 2024-07-29T00:43:09.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.362+0000 7f6054400640 1 Processor -- start 2024-07-29T00:43:09.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.362+0000 7f6054400640 1 -- start start 2024-07-29T00:43:09.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6054400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 0x7f604c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6054400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6054400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f604c10be60 0x7f604c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6054400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f604c111c40 con 0x7f604c1033a0 2024-07-29T00:43:09.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6054400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f604c111ac0 con 0x7f604c102a80 2024-07-29T00:43:09.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6054400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f604c111dc0 con 0x7f604c10be60 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6051000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6051a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 0x7f604c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6051000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44226/0 (socket says 172.21.15.175:44226) 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6051000640 1 -- 172.21.15.175:0/1580724702 learned_addr learned my addr 172.21.15.175:0/1580724702 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.363+0000 7f6052400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f604c10be60 0x7f604c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6051a00640 1 -- 172.21.15.175:0/1580724702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f604c10be60 msgr2=0x7f604c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6051a00640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f604c10be60 0x7f604c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6051a00640 1 -- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 msgr2=0x7f604c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6051a00640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6051a00640 1 -- 172.21.15.175:0/1580724702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c19d110 con 0x7f604c102a80 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6051a00640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 0x7f604c198160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f604000e9f0 tx=0x7f604000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f604a000640 1 -- 172.21.15.175:0/1580724702 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000ccc0 con 0x7f604c102a80 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c19d400 con 0x7f604c102a80 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f604a000640 1 -- 172.21.15.175:0/1580724702 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6040010910 con 0x7f604c102a80 2024-07-29T00:43:09.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.364+0000 7f604a000640 1 -- 172.21.15.175:0/1580724702 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040018960 con 0x7f604c102a80 2024-07-29T00:43:09.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.365+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c100610 con 0x7f604c102a80 2024-07-29T00:43:09.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.368+0000 7f604a000640 1 -- 172.21.15.175:0/1580724702 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6040010ac0 con 0x7f604c102a80 2024-07-29T00:43:09.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.368+0000 7f6056ebc740 1 Processor -- start 2024-07-29T00:43:09.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.368+0000 7f6056ebc740 1 -- start start 2024-07-29T00:43:09.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730d2720 0x5563730d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6056ebc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55637302dab0 con 0x5563730c8ef0 2024-07-29T00:43:09.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6056ebc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556372fc5a80 con 0x5563730d2720 2024-07-29T00:43:09.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6056ebc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556373029a70 con 0x5563730c7980 2024-07-29T00:43:09.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6052400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53528/0 (socket says 172.21.15.175:53528) 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6051a00640 1 -- 172.21.15.175:0/4165171673 learned_addr learned my addr 172.21.15.175:0/4165171673 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.369+0000 7f6051000640 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730d2720 0x5563730d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051a00640 1 -- 172.21.15.175:0/4165171673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730d2720 msgr2=0x5563730d4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051a00640 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730d2720 0x5563730d4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051a00640 1 -- 172.21.15.175:0/4165171673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 msgr2=0x5563730c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051a00640 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051a00640 1 -- 172.21.15.175:0/4165171673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556372d9f3c0 con 0x5563730c7980 2024-07-29T00:43:09.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051000640 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730d2720 0x5563730d4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:09.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f604a000640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f60340a2cf0 0x7f60340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6051a00640 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730c7dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f60400cce70 tx=0x7f60400d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd17fd548fd09ae server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:09.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f6052400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f60340a2cf0 0x7f60340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.370+0000 7f604a000640 1 -- 172.21.15.175:0/1580724702 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f60400cf8c0 con 0x7f604c102a80 2024-07-29T00:43:09.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.371+0000 7f603ea00640 1 -- 172.21.15.175:0/4165171673 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040026030 con 0x5563730c7980 2024-07-29T00:43:09.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.371+0000 7f603ea00640 1 -- 172.21.15.175:0/4165171673 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604009bdb0 con 0x5563730c7980 2024-07-29T00:43:09.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.371+0000 7f603ea00640 1 -- 172.21.15.175:0/4165171673 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604009b8f0 con 0x5563730c7980 2024-07-29T00:43:09.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 -- 172.21.15.175:0/4165171673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 msgr2=0x5563730c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6052400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f60340a2cf0 0x7f60340a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6044005fd0 tx=0x7f60440096a0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730c7dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f60400cce70 tx=0x7f60400d7670 comp rx=0 tx=0).stop 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 -- 172.21.15.175:0/4165171673 shutdown_connections 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730d2720 0x5563730d4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730c7dc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 --2- 172.21.15.175:0/4165171673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 -- 172.21.15.175:0/4165171673 >> 172.21.15.175:0/4165171673 conn(0x556373090950 msgr2=0x5563730c4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:09.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 -- 172.21.15.175:0/4165171673 shutdown_connections 2024-07-29T00:43:09.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 -- 172.21.15.175:0/4165171673 wait complete. 2024-07-29T00:43:09.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.372+0000 7f6056ebc740 1 -- start start 2024-07-29T00:43:09.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 Processor -- start 2024-07-29T00:43:09.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x5563730d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5563730d5460 con 0x5563730c8ef0 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556372fe8c10 con 0x5563730c7980 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5563730d55e0 con 0x5563730d2720 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.373+0000 7f6056ebc740 10 client.0 _ll_register_callbacks cb 0x556372fb1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6052400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x5563730d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6052400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x5563730d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53536/0 (socket says 172.21.15.175:53536) 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6052400640 1 -- 172.21.15.175:0/1472214554 learned_addr learned my addr 172.21.15.175:0/1472214554 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 -- 172.21.15.175:0/1472214554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 msgr2=0x5563730d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x5563730d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 -- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 msgr2=0x5563730cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 -- 172.21.15.175:0/1472214554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556372fec1e0 con 0x5563730c8ef0 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6051000640 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730cfc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6038009230 tx=0x7f6038032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f888005d6fa7282 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:09.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.374+0000 7f6033400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6033400640 1 -- 172.21.15.175:0/1472214554 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038035040 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604009bdb0 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6033400640 1 -- 172.21.15.175:0/1472214554 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f6038031930 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6033400640 1 -- 172.21.15.175:0/1472214554 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6038004040 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6056ebc740 10 client.16611 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5563730e12a0 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.375+0000 7f6033400640 1 -- 172.21.15.175:0/1472214554 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604009bdb0 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6033400640 1 -- 172.21.15.175:0/1472214554 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f60380047d0 con 0x5563730c8ef0 2024-07-29T00:43:09.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6056ebc740 10 client.16611 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6056ebc740 10 client.16611 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6056ebc740 20 client.16611 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5563730e26b0 0x5563730e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6056ebc740 4 client.16611 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637303b020 con 0x5563730e26b0 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.376+0000 7f6051a00640 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5563730e26b0 0x5563730e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.377+0000 7f6051a00640 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5563730e26b0 0x5563730e4ac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5509beff918f522e server_cookie=39311ad428ed3ce4 in_seq=0 out_seq=0 2024-07-29T00:43:09.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.377+0000 7f6033400640 10 client.16611 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:43:09.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6033400640 1 -- 172.21.15.175:0/1472214554 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55637303b020 con 0x5563730e26b0 2024-07-29T00:43:09.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6033400640 10 client.16611 handle_command_reply: tid=0 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 client.16611 shutdown 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 msgr2=0x5563730cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730cfc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6038009230 tx=0x7f6038032f50 comp rx=0 tx=0).stop 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 shutdown_connections 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x5563730d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730cfc50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5563730e26b0 0x5563730e4ac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1472214554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 >> 172.21.15.175:0/1472214554 conn(0x556373090950 msgr2=0x5563730c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:09.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.378+0000 7f6033400640 0 client.16611 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:43:09.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.379+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 shutdown_connections 2024-07-29T00:43:09.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.379+0000 7f6056ebc740 1 -- 172.21.15.175:0/1472214554 wait complete. 2024-07-29T00:43:09.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.379+0000 7f6056ebc740 20 client.16611 trim_cache size 0 max 16384 2024-07-29T00:43:09.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 Processor -- start 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 -- start start 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c8ef0 0x5563730ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730d2720 0x5563730ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1afae0 con 0x5563730d2720 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b16e0 con 0x5563730c8ef0 2024-07-29T00:43:09.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.395+0000 7f6056ebc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b01b0 con 0x5563730c7980 2024-07-29T00:43:09.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c8ef0 0x5563730ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6052400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730d2720 0x5563730ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53540/0 (socket says 172.21.15.175:53540) 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051a00640 1 -- 172.21.15.175:0/1941704252 learned_addr learned my addr 172.21.15.175:0/1941704252 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051000640 1 -- 172.21.15.175:0/1941704252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 msgr2=0x5563730cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051000640 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051000640 1 -- 172.21.15.175:0/1941704252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730d2720 msgr2=0x5563730ea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051000640 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730d2720 0x5563730ea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.396+0000 7f6051000640 1 -- 172.21.15.175:0/1941704252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563730ca4d0 con 0x5563730c8ef0 2024-07-29T00:43:09.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.397+0000 7f6051000640 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c8ef0 0x5563730ea390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6038009a20 tx=0x7f6038004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50df6a5b3402a8db server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:09.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.397+0000 7f603e000640 1 -- 172.21.15.175:0/1941704252 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038035030 con 0x5563730c8ef0 2024-07-29T00:43:09.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.397+0000 7f603e000640 1 -- 172.21.15.175:0/1941704252 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60380043e0 con 0x5563730c8ef0 2024-07-29T00:43:09.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.397+0000 7f603e000640 1 -- 172.21.15.175:0/1941704252 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038031890 con 0x5563730c8ef0 2024-07-29T00:43:09.401 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 -- 172.21.15.175:0/1941704252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c8ef0 msgr2=0x5563730ea390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c8ef0 0x5563730ea390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6038009a20 tx=0x7f6038004990 comp rx=0 tx=0).stop 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 -- 172.21.15.175:0/1941704252 shutdown_connections 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730d2720 0x5563730ea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c8ef0 0x5563730ea390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 --2- 172.21.15.175:0/1941704252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730c7980 0x5563730cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 -- 172.21.15.175:0/1941704252 >> 172.21.15.175:0/1941704252 conn(0x556373090950 msgr2=0x5563730ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:09.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 -- 172.21.15.175:0/1941704252 shutdown_connections 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 -- 172.21.15.175:0/1941704252 wait complete. 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.398+0000 7f6056ebc740 1 -- start start 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 Processor -- start 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x55637314b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b0f50 con 0x5563730c8ef0 2024-07-29T00:43:09.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b0880 con 0x5563730c7980 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1adae0 con 0x5563730d2720 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6056ebc740 10 client.0 _ll_register_callbacks cb 0x7f6038009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.399+0000 7f6051000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730d6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44268/0 (socket says 172.21.15.175:44268) 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051000640 1 -- 172.21.15.175:0/872509857 learned_addr learned my addr 172.21.15.175:0/872509857 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6052400640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x55637314b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 -- 172.21.15.175:0/872509857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 msgr2=0x55637314b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x55637314b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 -- 172.21.15.175:0/872509857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 msgr2=0x5563730d6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730d6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 -- 172.21.15.175:0/872509857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637315c9a0 con 0x5563730c7980 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6051a00640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730d5dd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f604009a060 tx=0x7f604000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cee01de07d8043 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.400+0000 7f6033400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6033400640 1 -- 172.21.15.175:0/872509857 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400b9060 con 0x5563730c7980 2024-07-29T00:43:09.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6033400640 1 -- 172.21.15.175:0/872509857 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8644+0+0 (secure 0 0 0) 0x7f6040098420 con 0x5563730c7980 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60380043e0 con 0x5563730c7980 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6033400640 1 -- 172.21.15.175:0/872509857 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f604009b710 con 0x5563730c7980 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 10 client.26762 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55637315dce0 con 0x5563730c7980 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6033400640 1 -- 172.21.15.175:0/872509857 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60380043e0 con 0x5563730c7980 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6033400640 1 -- 172.21.15.175:0/872509857 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f6040086070 con 0x5563730c7980 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 10 client.26762 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:09.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 10 client.26762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:43:09.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 20 client.26762 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:09.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.401+0000 7f6056ebc740 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55637315dfd0 0x5563731603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:09.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.402+0000 7f6056ebc740 4 client.26762 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:43:09.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.402+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556373035530 con 0x55637315dfd0 2024-07-29T00:43:09.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.402+0000 7f6051000640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55637315dfd0 0x5563731603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:09.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.402+0000 7f6051000640 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55637315dfd0 0x5563731603c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b017e9f323aa65f server_cookie=227c9688390bc1fc in_seq=0 out_seq=0 2024-07-29T00:43:09.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.402+0000 7f6033400640 10 client.26762 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:43:09.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6033400640 1 -- 172.21.15.175:0/872509857 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556373035530 con 0x55637315dfd0 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6033400640 10 client.26762 handle_command_reply: tid=0 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 client.26762 shutdown 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 msgr2=0x5563730d5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730d5dd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f604009a060 tx=0x7f604000cf60 comp rx=0 tx=0).stop 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 shutdown_connections 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563730d2720 0x55637314b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563730c8ef0 0x5563730d6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55637315dfd0 0x5563731603c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 --2- 172.21.15.175:0/872509857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563730c7980 0x5563730d5dd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 >> 172.21.15.175:0/872509857 conn(0x556373090950 msgr2=0x5563730c2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6033400640 0 client.26762 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 shutdown_connections 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 1 -- 172.21.15.175:0/872509857 wait complete. 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.403+0000 7f6056ebc740 20 client.26762 trim_cache size 0 max 16384 2024-07-29T00:43:09.410 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:43:09.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f60340a2cf0 msgr2=0x7f60340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f60340a2cf0 0x7f60340a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6044005fd0 tx=0x7f60440096a0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 msgr2=0x7f604c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 0x7f604c198160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f604000e9f0 tx=0x7f604000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 shutdown_connections 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f604c10be60 0x7f604c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f60340a2cf0 0x7f60340a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c1033a0 0x7f604c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 --2- 172.21.15.175:0/1580724702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c102a80 0x7f604c198160 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.404+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 >> 172.21.15.175:0/1580724702 conn(0x7f604c0fc480 msgr2=0x7f604c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.405+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 shutdown_connections 2024-07-29T00:43:09.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:09.405+0000 7f6054400640 1 -- 172.21.15.175:0/1580724702 wait complete. 2024-07-29T00:43:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:09 smithi181 ceph-mon[36706]: pgmap v203: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:09 smithi175 ceph-mon[32138]: pgmap v203: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:10.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:09 smithi204 ceph-mon[37368]: pgmap v203: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:10.343 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:10.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:43:11.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.033+0000 7f7d20600640 1 -- 172.21.15.175:0/200773338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18074280 msgr2=0x7f7d18074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.033+0000 7f7d20600640 1 --2- 172.21.15.175:0/200773338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18074280 0x7f7d18074660 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009900 tx=0x7f7d0c02a480 comp rx=0 tx=0).stop 2024-07-29T00:43:11.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.033+0000 7f7d20600640 1 -- 172.21.15.175:0/200773338 shutdown_connections 2024-07-29T00:43:11.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.034+0000 7f7d20600640 1 --2- 172.21.15.175:0/200773338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18112870 0x7f7d18114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.034+0000 7f7d20600640 1 --2- 172.21.15.175:0/200773338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074ba0 0x7f7d18112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.034+0000 7f7d20600640 1 --2- 172.21.15.175:0/200773338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18074280 0x7f7d18074660 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.034+0000 7f7d20600640 1 -- 172.21.15.175:0/200773338 >> 172.21.15.175:0/200773338 conn(0x7f7d180fd5a0 msgr2=0x7f7d180ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:11.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.034+0000 7f7d20600640 1 -- 172.21.15.175:0/200773338 shutdown_connections 2024-07-29T00:43:11.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.034+0000 7f7d20600640 1 -- 172.21.15.175:0/200773338 wait complete. 2024-07-29T00:43:11.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 Processor -- start 2024-07-29T00:43:11.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 -- start start 2024-07-29T00:43:11.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 0x7f7d181a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18074ba0 0x7f7d181a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18112870 0x7f7d1819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18118280 con 0x7f7d18112870 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18118100 con 0x7f7d18074280 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.035+0000 7f7d20600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18118400 con 0x7f7d18074ba0 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18112870 0x7f7d1819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 0x7f7d181a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 0x7f7d181a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39252/0 (socket says 172.21.15.175:39252) 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18074ba0 0x7f7d181a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4133461759 learned_addr learned my addr 172.21.15.175:0/4133461759 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4133461759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18074ba0 msgr2=0x7f7d181a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18074ba0 0x7f7d181a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18112870 msgr2=0x7f7d1819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18112870 0x7f7d1819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4133461759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1819ecd0 con 0x7f7d18074280 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1d200640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18074ba0 0x7f7d181a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:11.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.036+0000 7f7d1e600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18112870 0x7f7d1819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:11.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.037+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 0x7f7d181a43e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c004770 tx=0x7f7d0c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:11.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.037+0000 7f7d16000640 1 -- 172.21.15.175:0/4133461759 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c03f070 con 0x7f7d18074280 2024-07-29T00:43:11.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.037+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1819ef60 con 0x7f7d18074280 2024-07-29T00:43:11.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.037+0000 7f7d16000640 1 -- 172.21.15.175:0/4133461759 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c031ca0 con 0x7f7d18074280 2024-07-29T00:43:11.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.037+0000 7f7d16000640 1 -- 172.21.15.175:0/4133461759 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c035030 con 0x7f7d18074280 2024-07-29T00:43:11.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.037+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181ab2b0 con 0x7f7d18074280 2024-07-29T00:43:11.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.041+0000 7f7d16000640 1 -- 172.21.15.175:0/4133461759 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7d0c042070 con 0x7f7d18074280 2024-07-29T00:43:11.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.042+0000 7f7d23046740 1 Processor -- start 2024-07-29T00:43:11.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.042+0000 7f7d16000640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d000a2cf0 0x7f7d000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.042+0000 7f7d23046740 1 -- start start 2024-07-29T00:43:11.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.042+0000 7f7d16000640 1 -- 172.21.15.175:0/4133461759 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f7d0c0efc50 con 0x7f7d18074280 2024-07-29T00:43:11.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d1e600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d000a2cf0 0x7f7d000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d23046740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x5632242fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d23046740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 0x563224308560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d23046740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563224308aa0 0x56322430ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d23046740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563224263a80 con 0x5632242fe1e0 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d23046740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5632241fb960 con 0x5632242fd8c0 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d23046740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56322425fa40 con 0x563224308aa0 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d1e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563224308aa0 0x56322430ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 0x563224308560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 0x563224308560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44298/0 (socket says 172.21.15.175:44298) 2024-07-29T00:43:11.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d1d200640 1 -- 172.21.15.175:0/3113661970 learned_addr learned my addr 172.21.15.175:0/3113661970 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:11.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.043+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x5632242fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1dc00640 1 -- 172.21.15.175:0/3113661970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563224308aa0 msgr2=0x56322430ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563224308aa0 0x56322430ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1dc00640 1 -- 172.21.15.175:0/3113661970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 msgr2=0x563224308560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 0x563224308560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1dc00640 1 -- 172.21.15.175:0/3113661970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563223fd5620 con 0x5632242fd8c0 2024-07-29T00:43:11.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1e600640 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563224308aa0 0x56322430ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1d200640 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 0x563224308560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1e600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d000a2cf0 0x7f7d000a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d180762f0 tx=0x7f7d10010b00 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x5632242fdca0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0ed2f0 tx=0x7f7d0c03b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c544d3913fe2bc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d0aa00640 1 -- 172.21.15.175:0/3113661970 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c049050 con 0x5632242fd8c0 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d0aa00640 1 -- 172.21.15.175:0/3113661970 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c0c0470 con 0x5632242fd8c0 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.044+0000 7f7d0aa00640 1 -- 172.21.15.175:0/3113661970 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0b87e0 con 0x5632242fd8c0 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.045+0000 7f7d23046740 1 -- 172.21.15.175:0/3113661970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 msgr2=0x5632242fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.045+0000 7f7d23046740 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x5632242fdca0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0ed2f0 tx=0x7f7d0c03b990 comp rx=0 tx=0).stop 2024-07-29T00:43:11.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 -- 172.21.15.175:0/3113661970 shutdown_connections 2024-07-29T00:43:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563224308aa0 0x56322430ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632242fe1e0 0x563224308560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 --2- 172.21.15.175:0/3113661970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x5632242fdca0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 -- 172.21.15.175:0/3113661970 >> 172.21.15.175:0/3113661970 conn(0x5632242f9e50 msgr2=0x5632242fa230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 -- 172.21.15.175:0/3113661970 shutdown_connections 2024-07-29T00:43:11.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 -- 172.21.15.175:0/3113661970 wait complete. 2024-07-29T00:43:11.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.046+0000 7f7d23046740 1 -- start start 2024-07-29T00:43:11.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.047+0000 7f7d23046740 1 Processor -- start 2024-07-29T00:43:11.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.047+0000 7f7d23046740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x5632243059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.047+0000 7f7d23046740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x563224305f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d23046740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x563224306450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d23046740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56322430b430 con 0x563224308aa0 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d23046740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56322421e9c0 con 0x5632242fe1e0 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d23046740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56322430b5b0 con 0x5632242fd8c0 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d23046740 10 client.0 _ll_register_callbacks cb 0x56322420ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x563224305f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x563224305f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39260/0 (socket says 172.21.15.175:39260) 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x5632243059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 -- 172.21.15.175:0/3729348757 learned_addr learned my addr 172.21.15.175:0/3729348757 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1e600640 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x563224306450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 -- 172.21.15.175:0/3729348757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 msgr2=0x5632243059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x5632243059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 -- 172.21.15.175:0/3729348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 msgr2=0x563224306450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x563224306450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.048+0000 7f7d1d200640 1 -- 172.21.15.175:0/3729348757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563224201e10 con 0x5632242fe1e0 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7d1d200640 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x563224305f10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7d040117e0 tx=0x7f7d04011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4529f48e7646ea3c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7cff400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:11.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7cff400640 1 -- 172.21.15.175:0/3729348757 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0401f060 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c0c0470 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7cff400640 1 -- 172.21.15.175:0/3729348757 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f7d04010ba0 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7cff400640 1 -- 172.21.15.175:0/3729348757 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7d04010da0 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.049+0000 7f7d23046740 10 client.26774 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5632243171e0 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7cff400640 1 -- 172.21.15.175:0/3729348757 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c0c0470 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7cff400640 1 -- 172.21.15.175:0/3729348757 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7d04022520 con 0x5632242fe1e0 2024-07-29T00:43:11.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 10 client.26774 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:11.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 10 client.26774 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:43:11.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 20 client.26774 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:11.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5632243186b0 0x56322431aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 4 client.26774 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:11.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.050+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563224270ff0 con 0x5632243186b0 2024-07-29T00:43:11.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.051+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5632243186b0 0x56322431aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.051+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5632243186b0 0x56322431aae0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf9943000fc2585 server_cookie=56efeb4366beebc9 in_seq=0 out_seq=0 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.051+0000 7f7cff400640 10 client.26774 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7cff400640 1 -- 172.21.15.175:0/3729348757 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563224270ff0 con 0x5632243186b0 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7cff400640 10 client.26774 handle_command_reply: tid=0 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 client.26774 shutdown 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 msgr2=0x563224305f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x563224305f10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7d040117e0 tx=0x7f7d04011ca0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 shutdown_connections 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5632243186b0 0x56322431aae0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x563224306450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x563224305f10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.052+0000 7f7d23046740 1 --2- 172.21.15.175:0/3729348757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x5632243059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.053+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 >> 172.21.15.175:0/3729348757 conn(0x5632242f9e50 msgr2=0x5632242f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.053+0000 7f7cff400640 0 client.26774 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:11.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.053+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 shutdown_connections 2024-07-29T00:43:11.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.053+0000 7f7d23046740 1 -- 172.21.15.175:0/3729348757 wait complete. 2024-07-29T00:43:11.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.053+0000 7f7d23046740 20 client.26774 trim_cache size 0 max 16384 2024-07-29T00:43:11.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 Processor -- start 2024-07-29T00:43:11.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 -- start start 2024-07-29T00:43:11.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x563224300370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x5632242f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x5632243203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181b3dc0 con 0x563224308aa0 2024-07-29T00:43:11.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181b59c0 con 0x5632242fe1e0 2024-07-29T00:43:11.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d23046740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181b4490 con 0x5632242fd8c0 2024-07-29T00:43:11.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.074+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x5632242f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x563224300370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x5632243203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x5632242f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39274/0 (socket says 172.21.15.175:39274) 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 -- 172.21.15.175:0/93131674 learned_addr learned my addr 172.21.15.175:0/93131674 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 -- 172.21.15.175:0/93131674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 msgr2=0x563224300370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x563224300370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 -- 172.21.15.175:0/93131674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 msgr2=0x5632243203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x5632243203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 -- 172.21.15.175:0/93131674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632242fd6d0 con 0x5632242fe1e0 2024-07-29T00:43:11.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1e600640 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x5632243203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.075+0000 7f7d1d200640 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x5632242f8f00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7d04022910 tx=0x7f7d0400bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58702ea3784c8713 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.076+0000 7f7d0a000640 1 -- 172.21.15.175:0/93131674 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0402f070 con 0x5632242fe1e0 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.076+0000 7f7d0a000640 1 -- 172.21.15.175:0/93131674 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d04022b80 con 0x5632242fe1e0 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.076+0000 7f7d0a000640 1 -- 172.21.15.175:0/93131674 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04000e60 con 0x5632242fe1e0 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.076+0000 7f7d23046740 1 -- 172.21.15.175:0/93131674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 msgr2=0x5632242f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.076+0000 7f7d23046740 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x5632242f8f00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7d04022910 tx=0x7f7d0400bb90 comp rx=0 tx=0).stop 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 -- 172.21.15.175:0/93131674 shutdown_connections 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x5632243203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fe1e0 0x5632242f8f00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 --2- 172.21.15.175:0/93131674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fd8c0 0x563224300370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 -- 172.21.15.175:0/93131674 >> 172.21.15.175:0/93131674 conn(0x5632242f9e50 msgr2=0x5632242f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:11.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 -- 172.21.15.175:0/93131674 shutdown_connections 2024-07-29T00:43:11.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 -- 172.21.15.175:0/93131674 wait complete. 2024-07-29T00:43:11.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 -- start start 2024-07-29T00:43:11.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.077+0000 7f7d23046740 1 Processor -- start 2024-07-29T00:43:11.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x56322430c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fe1e0 0x56322430caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x56322430cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181b5230 con 0x563224308aa0 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181b4b60 con 0x5632242fd8c0 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181b1dc0 con 0x5632242fe1e0 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d23046740 10 client.0 _ll_register_callbacks cb 0x7f7d04011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x56322430c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d1dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x56322430c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39280/0 (socket says 172.21.15.175:39280) 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4032213255 learned_addr learned my addr 172.21.15.175:0/4032213255 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d1d200640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fe1e0 0x56322430caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.078+0000 7f7d1e600640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x56322430cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4032213255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fe1e0 msgr2=0x56322430caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fe1e0 0x56322430caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4032213255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 msgr2=0x56322430cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x56322430cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1dc00640 1 -- 172.21.15.175:0/4032213255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563224382380 con 0x5632242fd8c0 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1d200640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fe1e0 0x56322430caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d1dc00640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x56322430c560 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c000c00 tx=0x7f7d0c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6118d369c587d33 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:11.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7cff400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:11.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7cff400640 1 -- 172.21.15.175:0/4032213255 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c049040 con 0x5632242fd8c0 2024-07-29T00:43:11.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7cff400640 1 -- 172.21.15.175:0/4032213255 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f7d0c0dc0b0 con 0x5632242fd8c0 2024-07-29T00:43:11.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d04022b80 con 0x5632242fd8c0 2024-07-29T00:43:11.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7cff400640 1 -- 172.21.15.175:0/4032213255 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c0c0470 con 0x5632242fd8c0 2024-07-29T00:43:11.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d23046740 10 client.26780 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:11.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.079+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563224383640 con 0x5632242fd8c0 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7cff400640 1 -- 172.21.15.175:0/4032213255 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d04022b80 con 0x5632242fd8c0 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7cff400640 1 -- 172.21.15.175:0/4032213255 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f7d0c0b6ce0 con 0x5632242fd8c0 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d23046740 10 client.26780 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d23046740 10 client.26780 resolve_mds: validated gid 24539 aka daemon mds.b 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d23046740 20 client.26780 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d23046740 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563224383bb0 0x563224385fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d23046740 4 client.26780 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5632243015f0 con 0x563224383bb0 2024-07-29T00:43:11.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.080+0000 7f7d1d200640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563224383bb0 0x563224385fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.081+0000 7f7d1d200640 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563224383bb0 0x563224385fd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4190471348148e57 server_cookie=23d050c4f38ccffb in_seq=0 out_seq=0 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.081+0000 7f7cff400640 10 client.26780 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.081+0000 7f7cff400640 1 -- 172.21.15.175:0/4032213255 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5632243015f0 con 0x563224383bb0 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.081+0000 7f7cff400640 10 client.26780 handle_command_reply: tid=0 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.081+0000 7f7d23046740 1 client.26780 shutdown 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 msgr2=0x56322430c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x56322430c560 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c000c00 tx=0x7f7d0c0045e0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 shutdown_connections 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x563224383bb0 0x563224385fd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563224308aa0 0x56322430cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632242fe1e0 0x56322430caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 --2- 172.21.15.175:0/4032213255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632242fd8c0 0x56322430c560 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7cff400640 0 client.26780 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:11.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 >> 172.21.15.175:0/4032213255 conn(0x5632242f9e50 msgr2=0x5632242f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:11.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 shutdown_connections 2024-07-29T00:43:11.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 1 -- 172.21.15.175:0/4032213255 wait complete. 2024-07-29T00:43:11.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.082+0000 7f7d23046740 20 client.26780 trim_cache size 0 max 16384 2024-07-29T00:43:11.090 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:43:11.090 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 40, 2024-07-29T00:43:11.091 INFO:teuthology.orchestra.run.smithi175.stdout: "leader": 24539, 2024-07-29T00:43:11.091 INFO:teuthology.orchestra.run.smithi175.stdout: "set_version": 20, 2024-07-29T00:43:11.091 INFO:teuthology.orchestra.run.smithi175.stdout: "sets": {} 2024-07-29T00:43:11.091 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d000a2cf0 msgr2=0x7f7d000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d000a2cf0 0x7f7d000a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d180762f0 tx=0x7f7d10010b00 comp rx=0 tx=0).stop 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 msgr2=0x7f7d181a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 0x7f7d181a43e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c004770 tx=0x7f7d0c003c10 comp rx=0 tx=0).stop 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 shutdown_connections 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7d000a2cf0 0x7f7d000a51b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d18112870 0x7f7d1819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18074ba0 0x7f7d181a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 --2- 172.21.15.175:0/4133461759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d18074280 0x7f7d181a43e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:11.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.083+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 >> 172.21.15.175:0/4133461759 conn(0x7f7d180fd5a0 msgr2=0x7f7d180ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:11.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.084+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 shutdown_connections 2024-07-29T00:43:11.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:11.084+0000 7f7d20600640 1 -- 172.21.15.175:0/4133461759 wait complete. 2024-07-29T00:43:11.093 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:11.949 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:11 smithi175 ceph-mon[32138]: pgmap v204: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:11.949 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:11 smithi175 ceph-mon[32138]: osdmap e94: 12 total, 12 up, 12 in 2024-07-29T00:43:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:11 smithi181 ceph-mon[36706]: pgmap v204: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:11 smithi181 ceph-mon[36706]: osdmap e94: 12 total, 12 up, 12 in 2024-07-29T00:43:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:11 smithi204 ceph-mon[37368]: pgmap v204: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 915 KiB/s wr, 201 op/s 2024-07-29T00:43:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:11 smithi204 ceph-mon[37368]: osdmap e94: 12 total, 12 up, 12 in 2024-07-29T00:43:12.552 INFO:tasks.quiescer:done joining Quiescers 2024-07-29T00:43:12.553 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-07-29T00:43:12.562 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-07-29T00:43:12.562 INFO:teuthology.task.exec:Executing custom commands... 2024-07-29T00:43:12.562 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:43:12.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-07-29T00:43:12.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-07-29T00:43:12.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 -- 172.21.15.175:0/1122287261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074c30 msgr2=0x7f26e810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 --2- 172.21.15.175:0/1122287261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074c30 0x7f26e810d9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009900 tx=0x7f26d402a490 comp rx=0 tx=0).stop 2024-07-29T00:43:12.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 -- 172.21.15.175:0/1122287261 shutdown_connections 2024-07-29T00:43:12.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 --2- 172.21.15.175:0/1122287261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e810def0 0x7f26e8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 --2- 172.21.15.175:0/1122287261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074c30 0x7f26e810d9b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 --2- 172.21.15.175:0/1122287261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8074310 0x7f26e80746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.088+0000 7f26e7200640 1 -- 172.21.15.175:0/1122287261 >> 172.21.15.175:0/1122287261 conn(0x7f26e80fd3d0 msgr2=0x7f26e80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.089+0000 7f26e7200640 1 -- 172.21.15.175:0/1122287261 shutdown_connections 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.089+0000 7f26e7200640 1 -- 172.21.15.175:0/1122287261 wait complete. 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.089+0000 7f26e7200640 1 Processor -- start 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 -- start start 2024-07-29T00:43:12.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074310 0x7f26e819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 0x7f26e819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e810def0 0x7f26e819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8113cf0 con 0x7f26e8074310 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8113b70 con 0x7f26e810def0 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8113e70 con 0x7f26e8074c30 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074310 0x7f26e819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.090+0000 7f26e5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 0x7f26e819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 0x7f26e819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53624/0 (socket says 172.21.15.175:53624) 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 -- 172.21.15.175:0/1476783764 learned_addr learned my addr 172.21.15.175:0/1476783764 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e6800640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e810def0 0x7f26e819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 -- 172.21.15.175:0/1476783764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e810def0 msgr2=0x7f26e819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e810def0 0x7f26e819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 -- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074310 msgr2=0x7f26e819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074310 0x7f26e819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 -- 172.21.15.175:0/1476783764 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e819f120 con 0x7f26e8074c30 2024-07-29T00:43:12.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.091+0000 7f26e5400640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 0x7f26e819a6b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004770 tx=0x7f26d4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.092+0000 7f26dea00640 1 -- 172.21.15.175:0/1476783764 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d403f070 con 0x7f26e8074c30 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.092+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e819f3b0 con 0x7f26e8074c30 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.092+0000 7f26dea00640 1 -- 172.21.15.175:0/1476783764 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4031ca0 con 0x7f26e8074c30 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.092+0000 7f26dea00640 1 -- 172.21.15.175:0/1476783764 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4035030 con 0x7f26e8074c30 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.092+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81a6cf0 con 0x7f26e8074c30 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 Processor -- start 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 -- start start 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26dea00640 1 -- 172.21.15.175:0/1476783764 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f26d4042070 con 0x7f26e8074c30 2024-07-29T00:43:12.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d546b0e0 0x5596d5469630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d5469fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5476d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5596d53cfab0 con 0x5596d5469b70 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5596d5367a80 con 0x5596d546b0e0 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.095+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596d53cba70 con 0x5596d5474910 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d5469fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d546b0e0 0x5596d5469630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d5469fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44342/0 (socket says 172.21.15.175:44342) 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e5e00640 1 -- 172.21.15.175:0/1941300130 learned_addr learned my addr 172.21.15.175:0/1941300130 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e5400640 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5476d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e6800640 1 -- 172.21.15.175:0/1941300130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 msgr2=0x5596d5476d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e6800640 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5476d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e6800640 1 -- 172.21.15.175:0/1941300130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 msgr2=0x5596d5469fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e6800640 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d5469fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e6800640 1 -- 172.21.15.175:0/1941300130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d51413c0 con 0x5596d546b0e0 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e5400640 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5476d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:12.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.096+0000 7f26e5e00640 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d5469fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26e6800640 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d546b0e0 0x5596d5469630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009f00 tx=0x7f26e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1db316dd3e5f6b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26d3000640 1 -- 172.21.15.175:0/1941300130 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0004d20 con 0x5596d546b0e0 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26d3000640 1 -- 172.21.15.175:0/1941300130 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e0004050 con 0x5596d546b0e0 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26d3000640 1 -- 172.21.15.175:0/1941300130 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0005280 con 0x5596d546b0e0 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26dea00640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26cc0a2d70 0x7f26cc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26ef5ff740 1 -- 172.21.15.175:0/1941300130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d546b0e0 msgr2=0x5596d5469630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.097+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d546b0e0 0x5596d5469630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009f00 tx=0x7f26e0011810 comp rx=0 tx=0).stop 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26dea00640 1 -- 172.21.15.175:0/1476783764 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f26d40efa10 con 0x7f26e8074c30 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 -- 172.21.15.175:0/1941300130 shutdown_connections 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5476d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d5469fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/1941300130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d546b0e0 0x5596d5469630 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 -- 172.21.15.175:0/1941300130 >> 172.21.15.175:0/1941300130 conn(0x5596d5432950 msgr2=0x5596d5466200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26e5e00640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26cc0a2d70 0x7f26cc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 -- 172.21.15.175:0/1941300130 shutdown_connections 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26ef5ff740 1 -- 172.21.15.175:0/1941300130 wait complete. 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.098+0000 7f26e5e00640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26cc0a2d70 0x7f26cc0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5596d5467060 tx=0x7f26d8005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 -- start start 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 Processor -- start 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5469b70 0x5596d5471880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d5471dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5472300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5596d54776b0 con 0x5596d546b0e0 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5596d538ac10 con 0x5596d5469b70 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596d5477830 con 0x5596d5474910 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.099+0000 7f26ef5ff740 10 client.0 _ll_register_callbacks cb 0x5596d5353a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5472300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5469b70 0x5596d5471880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5472300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53654/0 (socket says 172.21.15.175:53654) 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 -- 172.21.15.175:0/918872346 learned_addr learned my addr 172.21.15.175:0/918872346 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:12.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e5400640 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d5471dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 -- 172.21.15.175:0/918872346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5469b70 msgr2=0x5596d5471880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5469b70 0x5596d5471880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 -- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 msgr2=0x5596d5471dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d5471dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 -- 172.21.15.175:0/918872346 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d538e1e0 con 0x5596d5474910 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26e6800640 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5472300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26e0000f80 tx=0x7f26e0004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d140743a7aa04b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:12.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.100+0000 7f26c7400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26c7400640 1 -- 172.21.15.175:0/918872346 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0029020 con 0x5596d5474910 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26c7400640 1 -- 172.21.15.175:0/918872346 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f26e0025470 con 0x5596d5474910 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e0004050 con 0x5596d5474910 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26c7400640 1 -- 172.21.15.175:0/918872346 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f26e0034050 con 0x5596d5474910 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26ef5ff740 10 client.26746 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596d5483420 con 0x5596d5474910 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26c7400640 1 -- 172.21.15.175:0/918872346 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e0004050 con 0x5596d5474910 2024-07-29T00:43:12.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.101+0000 7f26c7400640 1 -- 172.21.15.175:0/918872346 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f26e0019730 con 0x5596d5474910 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26ef5ff740 10 client.26746 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26ef5ff740 10 client.26746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26ef5ff740 20 client.26746 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d5484880 0x5596d5486c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26ef5ff740 4 client.26746 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d53dd020 con 0x5596d5484880 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.102+0000 7f26e5e00640 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d5484880 0x5596d5486c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.103+0000 7f26e5e00640 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d5484880 0x5596d5486c70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c438a3f6b449e284 server_cookie=b7bf812b9c7762be in_seq=0 out_seq=0 2024-07-29T00:43:12.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.103+0000 7f26c7400640 10 client.26746 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.103+0000 7f26c7400640 1 -- 172.21.15.175:0/918872346 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5596d53dd020 con 0x5596d5484880 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.103+0000 7f26c7400640 10 client.26746 handle_command_reply: tid=0 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.103+0000 7f26ef5ff740 1 client.26746 shutdown 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 msgr2=0x5596d5472300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5472300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26e0000f80 tx=0x7f26e0004430 comp rx=0 tx=0).stop 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 shutdown_connections 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d5484880 0x5596d5486c70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5474910 0x5596d5472300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d5471dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/918872346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5469b70 0x5596d5471880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26c7400640 0 client.26746 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 >> 172.21.15.175:0/918872346 conn(0x5596d5432950 msgr2=0x5596d546bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 shutdown_connections 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 1 -- 172.21.15.175:0/918872346 wait complete. 2024-07-29T00:43:12.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.104+0000 7f26ef5ff740 20 client.26746 trim_cache size 0 max 16384 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.118+0000 7f26ef5ff740 1 Processor -- start 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.118+0000 7f26ef5ff740 1 -- start start 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5469b70 0x5596d54737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d548c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d548c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81af770 con 0x5596d546b0e0 2024-07-29T00:43:12.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81b1370 con 0x5596d5474910 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81afe40 con 0x5596d5469b70 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26e5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d548c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26e5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d548c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44362/0 (socket says 172.21.15.175:44362) 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26e5400640 1 -- 172.21.15.175:0/3707717831 learned_addr learned my addr 172.21.15.175:0/3707717831 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26e5e00640 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5469b70 0x5596d54737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.119+0000 7f26e6800640 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d548c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5e00640 1 -- 172.21.15.175:0/3707717831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 msgr2=0x5596d548c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5e00640 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d548c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5e00640 1 -- 172.21.15.175:0/3707717831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 msgr2=0x5596d548c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5e00640 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d548c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5e00640 1 -- 172.21.15.175:0/3707717831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d5472850 con 0x5596d5469b70 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5400640 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d548c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26e5e00640 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5469b70 0x5596d54737f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5596d54872f0 tx=0x7f26d801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a02535d8d5520b8a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:12.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26d2600640 1 -- 172.21.15.175:0/3707717831 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d801de20 con 0x5596d5469b70 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26d2600640 1 -- 172.21.15.175:0/3707717831 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d802b910 con 0x5596d5469b70 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.120+0000 7f26d2600640 1 -- 172.21.15.175:0/3707717831 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8029be0 con 0x5596d5469b70 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 -- 172.21.15.175:0/3707717831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5469b70 msgr2=0x5596d54737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5469b70 0x5596d54737f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5596d54872f0 tx=0x7f26d801b000 comp rx=0 tx=0).stop 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 -- 172.21.15.175:0/3707717831 shutdown_connections 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d548c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d546b0e0 0x5596d548c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/3707717831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d5469b70 0x5596d54737f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 -- 172.21.15.175:0/3707717831 >> 172.21.15.175:0/3707717831 conn(0x5596d5432950 msgr2=0x5596d54730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 -- 172.21.15.175:0/3707717831 shutdown_connections 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 -- 172.21.15.175:0/3707717831 wait complete. 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.121+0000 7f26ef5ff740 1 -- start start 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 Processor -- start 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d547d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 0x5596d546e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d546eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81b0be0 con 0x5596d5469b70 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81b0510 con 0x5596d5474910 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81ad770 con 0x5596d546b0e0 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26ef5ff740 10 client.0 _ll_register_callbacks cb 0x7f26d80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d547d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.122+0000 7f26e5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 0x5596d546e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 0x5596d546e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53662/0 (socket says 172.21.15.175:53662) 2024-07-29T00:43:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 -- 172.21.15.175:0/2745780040 learned_addr learned my addr 172.21.15.175:0/2745780040 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e6800640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d546eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 -- 172.21.15.175:0/2745780040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 msgr2=0x5596d546eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d546eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 -- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 msgr2=0x5596d547d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d547d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 -- 172.21.15.175:0/2745780040 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d546f2d0 con 0x5596d546b0e0 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e6800640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d546eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:12.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5e00640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d547d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26e5400640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 0x5596d546e580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26d40b8450 tx=0x7f26d403b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d37756eb320d33e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26c7400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.123+0000 7f26c7400640 1 -- 172.21.15.175:0/2745780040 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4049070 con 0x5596d546b0e0 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d802b910 con 0x5596d546b0e0 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26c7400640 1 -- 172.21.15.175:0/2745780040 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f26d40b8660 con 0x5596d546b0e0 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26c7400640 1 -- 172.21.15.175:0/2745780040 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f26d40bbff0 con 0x5596d546b0e0 2024-07-29T00:43:12.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 10 client.26752 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596d54ef400 con 0x5596d546b0e0 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26c7400640 1 -- 172.21.15.175:0/2745780040 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d802b910 con 0x5596d546b0e0 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26c7400640 1 -- 172.21.15.175:0/2745780040 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f26d40ed3c0 con 0x5596d546b0e0 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 10 client.26752 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 10 client.26752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 20 client.26752 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d54efad0 0x5596d54f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:12.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 4 client.26752 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.124+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596d53d7530 con 0x5596d54efad0 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.125+0000 7f26e5e00640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d54efad0 0x5596d54f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.125+0000 7f26e5e00640 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d54efad0 0x5596d54f1ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=912b907fc18ba574 server_cookie=dfec380c9afa50f0 in_seq=0 out_seq=0 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.125+0000 7f26c7400640 10 client.26752 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26c7400640 1 -- 172.21.15.175:0/2745780040 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596d53d7530 con 0x5596d54efad0 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26c7400640 10 client.26752 handle_command_reply: tid=0 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 client.26752 shutdown 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 msgr2=0x5596d546e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 0x5596d546e580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26d40b8450 tx=0x7f26d403b890 comp rx=0 tx=0).stop 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 shutdown_connections 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x5596d54efad0 0x5596d54f1ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596d5474910 0x5596d546eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596d546b0e0 0x5596d546e580 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 --2- 172.21.15.175:0/2745780040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5596d5469b70 0x5596d547d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26c7400640 0 client.26752 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 >> 172.21.15.175:0/2745780040 conn(0x5596d5432950 msgr2=0x5596d5472040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:12.719 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 shutdown_connections 2024-07-29T00:43:12.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 1 -- 172.21.15.175:0/2745780040 wait complete. 2024-07-29T00:43:12.720 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:43:12.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.126+0000 7f26ef5ff740 20 client.26752 trim_cache size 0 max 16384 2024-07-29T00:43:12.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26cc0a2d70 msgr2=0x7f26cc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26cc0a2d70 0x7f26cc0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5596d5467060 tx=0x7f26d8005f90 comp rx=0 tx=0).stop 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 msgr2=0x7f26e819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 0x7f26e819a6b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004770 tx=0x7f26d4003c10 comp rx=0 tx=0).stop 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 shutdown_connections 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e810def0 0x7f26e819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26e8074c30 0x7f26e819a6b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.721 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26cc0a2d70 0x7f26cc0a5230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 --2- 172.21.15.175:0/1476783764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e8074310 0x7f26e819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:12.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 >> 172.21.15.175:0/1476783764 conn(0x7f26e80fd3d0 msgr2=0x7f26e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:12.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 shutdown_connections 2024-07-29T00:43:12.722 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:12.127+0000 7f26e7200640 1 -- 172.21.15.175:0/1476783764 wait complete. 2024-07-29T00:43:13.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:43:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:13 smithi175 ceph-mon[32138]: pgmap v206: 97 pgs: 97 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 396 KiB/s wr, 92 op/s 2024-07-29T00:43:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:13 smithi181 ceph-mon[36706]: pgmap v206: 97 pgs: 97 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 396 KiB/s wr, 92 op/s 2024-07-29T00:43:14.036 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:13 smithi204 ceph-mon[37368]: pgmap v206: 97 pgs: 97 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 396 KiB/s wr, 92 op/s 2024-07-29T00:43:15.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:15 smithi181 ceph-mon[36706]: pgmap v207: 97 pgs: 97 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 367 KiB/s wr, 85 op/s 2024-07-29T00:43:16.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:15 smithi175 ceph-mon[32138]: pgmap v207: 97 pgs: 97 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 367 KiB/s wr, 85 op/s 2024-07-29T00:43:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:15 smithi204 ceph-mon[37368]: pgmap v207: 97 pgs: 97 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 367 KiB/s wr, 85 op/s 2024-07-29T00:43:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.648+0000 7fcd19e00640 1 -- 172.21.15.175:0/72466472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1410df50 msgr2=0x7fcd14110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.648+0000 7fcd19e00640 1 --2- 172.21.15.175:0/72466472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1410df50 0x7fcd14110340 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009ca0 tx=0x7fcd0c02a940 comp rx=0 tx=0).stop 2024-07-29T00:43:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 -- 172.21.15.175:0/72466472 shutdown_connections 2024-07-29T00:43:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 --2- 172.21.15.175:0/72466472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1410df50 0x7fcd14110340 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 --2- 172.21.15.175:0/72466472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074b20 0x7fcd1410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 --2- 172.21.15.175:0/72466472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd14074200 0x7fcd140745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 -- 172.21.15.175:0/72466472 >> 172.21.15.175:0/72466472 conn(0x7fcd140fd370 msgr2=0x7fcd140ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 -- 172.21.15.175:0/72466472 shutdown_connections 2024-07-29T00:43:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.649+0000 7fcd19e00640 1 -- 172.21.15.175:0/72466472 wait complete. 2024-07-29T00:43:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 Processor -- start 2024-07-29T00:43:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 -- start start 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 0x7fcd1419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd14074b20 0x7fcd1419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1410df50 0x7fcd1419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14113c80 con 0x7fcd14074b20 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14113b00 con 0x7fcd1410df50 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd19e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14113e00 con 0x7fcd14074200 2024-07-29T00:43:16.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 0x7fcd1419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 0x7fcd1419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53680/0 (socket says 172.21.15.175:53680) 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd18a00640 1 -- 172.21.15.175:0/2508888048 learned_addr learned my addr 172.21.15.175:0/2508888048 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.650+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd14074b20 0x7fcd1419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd19400640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1410df50 0x7fcd1419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd18a00640 1 -- 172.21.15.175:0/2508888048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1410df50 msgr2=0x7fcd1419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1410df50 0x7fcd1419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd18a00640 1 -- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd14074b20 msgr2=0x7fcd1419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.654 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd14074b20 0x7fcd1419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd18a00640 1 -- 172.21.15.175:0/2508888048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1419f110 con 0x7fcd14074200 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd19400640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1410df50 0x7fcd1419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 0x7fcd1419a160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800d950 tx=0x7fcd0800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd11600640 1 -- 172.21.15.175:0/2508888048 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08017070 con 0x7fcd14074200 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd11600640 1 -- 172.21.15.175:0/2508888048 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd08009c30 con 0x7fcd14074200 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1419f400 con 0x7fcd14074200 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.651+0000 7fcd11600640 1 -- 172.21.15.175:0/2508888048 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800f6b0 con 0x7fcd14074200 2024-07-29T00:43:16.655 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.652+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd141a6ce0 con 0x7fcd14074200 2024-07-29T00:43:16.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.654+0000 7fcd1de28740 1 Processor -- start 2024-07-29T00:43:16.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 -- start start 2024-07-29T00:43:16.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd11600640 1 -- 172.21.15.175:0/2508888048 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcd080109d0 con 0x7fcd14074200 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c1622f8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1622f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c162303910 0x55c162305d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c16225eab0 con 0x55c1622f8b70 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c1621f6a80 con 0x55c162303910 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd1de28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c16225aa70 con 0x55c1622fa0e0 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.655+0000 7fcd19400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c1622f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1622f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c162303910 0x55c162305d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd19400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c1622f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53694/0 (socket says 172.21.15.175:53694) 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd19400640 1 -- 172.21.15.175:0/3212292863 learned_addr learned my addr 172.21.15.175:0/3212292863 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 -- 172.21.15.175:0/3212292863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 msgr2=0x55c1622f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c1622f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 -- 172.21.15.175:0/3212292863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c162303910 msgr2=0x55c162305d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c162303910 0x55c162305d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 -- 172.21.15.175:0/3212292863 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c161fd03c0 con 0x55c1622f8b70 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.656+0000 7fcd18a00640 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1622f8fb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd080cc810 tx=0x7fcd080d7910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05922e896a59e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.657+0000 7fcd11600640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fccfc0a2cf0 0x7fccfc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.657+0000 7fcd05c00640 1 -- 172.21.15.175:0/3212292863 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080e2070 con 0x55c1622f8b70 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.657+0000 7fcd05c00640 1 -- 172.21.15.175:0/3212292863 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0809bce0 con 0x55c1622f8b70 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.657+0000 7fcd05c00640 1 -- 172.21.15.175:0/3212292863 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0809bf70 con 0x55c1622f8b70 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.657+0000 7fcd11600640 1 -- 172.21.15.175:0/2508888048 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fcd080cf170 con 0x7fcd14074200 2024-07-29T00:43:16.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.657+0000 7fcd19400640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fccfc0a2cf0 0x7fccfc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 -- 172.21.15.175:0/3212292863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 msgr2=0x55c1622f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1622f8fb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd080cc810 tx=0x7fcd080d7910 comp rx=0 tx=0).stop 2024-07-29T00:43:16.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd19400640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fccfc0a2cf0 0x7fccfc0a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004af0 tx=0x7fcd0c02ffa0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:16.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 -- 172.21.15.175:0/3212292863 shutdown_connections 2024-07-29T00:43:16.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c162303910 0x55c162305d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1622f8fb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3212292863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c1622f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 -- 172.21.15.175:0/3212292863 >> 172.21.15.175:0/3212292863 conn(0x55c1622c1950 msgr2=0x55c1622f5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 -- 172.21.15.175:0/3212292863 shutdown_connections 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 -- 172.21.15.175:0/3212292863 wait complete. 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.658+0000 7fcd1de28740 1 -- start start 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 Processor -- start 2024-07-29T00:43:16.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1623008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c162300e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162301370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1623066b0 con 0x55c1622f8b70 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c162219c10 con 0x55c1622fa0e0 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c162306830 con 0x55c162303910 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd1de28740 10 client.0 _ll_register_callbacks cb 0x55c1621e2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1623008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1623008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44406/0 (socket says 172.21.15.175:44406) 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd18a00640 1 -- 172.21.15.175:0/2525156250 learned_addr learned my addr 172.21.15.175:0/2525156250 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd19400640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162301370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.659+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c162300e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd18a00640 1 -- 172.21.15.175:0/2525156250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 msgr2=0x55c162301370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162301370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd18a00640 1 -- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 msgr2=0x55c162300e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c162300e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd18a00640 1 -- 172.21.15.175:0/2525156250 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16221d1e0 con 0x55c1622f8b70 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1623008f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcd0809b2e0 tx=0x7fcd08010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a09c8790a26e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fccfa000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fccfa000640 1 -- 172.21.15.175:0/2525156250 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080c7380 con 0x55c1622f8b70 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fccfa000640 1 -- 172.21.15.175:0/2525156250 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fcd080998e0 con 0x55c1622f8b70 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.660+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0809bce0 con 0x55c1622f8b70 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fccfa000640 1 -- 172.21.15.175:0/2525156250 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcd080c54c0 con 0x55c1622f8b70 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 10 client.16665 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c1623066b0 con 0x55c1622f8b70 2024-07-29T00:43:16.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fccfa000640 1 -- 172.21.15.175:0/2525156250 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0809bce0 con 0x55c1622f8b70 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fccfa000640 1 -- 172.21.15.175:0/2525156250 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fcd08097c50 con 0x55c1622f8b70 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 10 client.16665 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 20 client.16665 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c162313600 0x55c1623159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 4 client.16665 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.661+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c16226c020 con 0x55c162313600 2024-07-29T00:43:16.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.662+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c162313600 0x55c1623159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.662+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c162313600 0x55c1623159f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2e36702938b640 server_cookie=6a732097aef6a7ba in_seq=0 out_seq=0 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.662+0000 7fccfa000640 10 client.16665 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fccfa000640 1 -- 172.21.15.175:0/2525156250 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c16226c020 con 0x55c162313600 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fccfa000640 10 client.16665 handle_command_reply: tid=0 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 client.16665 shutdown 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 msgr2=0x55c1623008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1623008f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcd0809b2e0 tx=0x7fcd08010cc0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 shutdown_connections 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c162313600 0x55c1623159f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162301370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c162300e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2525156250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c1623008f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fccfa000640 0 client.16665 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.663+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 >> 172.21.15.175:0/2525156250 conn(0x55c1622c1950 msgr2=0x55c1622faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.664+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 shutdown_connections 2024-07-29T00:43:16.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.664+0000 7fcd1de28740 1 -- 172.21.15.175:0/2525156250 wait complete. 2024-07-29T00:43:16.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.664+0000 7fcd1de28740 20 client.16665 trim_cache size 0 max 16384 2024-07-29T00:43:16.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.678+0000 7fcd1de28740 1 Processor -- start 2024-07-29T00:43:16.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 -- start start 2024-07-29T00:43:16.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622f8b70 0x55c1622f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c16231aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c162303910 0x55c16231b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141af830 con 0x55c162303910 2024-07-29T00:43:16.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141b1430 con 0x55c1622f8b70 2024-07-29T00:43:16.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd1de28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141aff00 con 0x55c1622fa0e0 2024-07-29T00:43:16.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c16231aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd19400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c162303910 0x55c16231b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c16231aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53714/0 (socket says 172.21.15.175:53714) 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd13e00640 1 -- 172.21.15.175:0/2878529903 learned_addr learned my addr 172.21.15.175:0/2878529903 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.679+0000 7fcd18a00640 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622f8b70 0x55c1622f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd13e00640 1 -- 172.21.15.175:0/2878529903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622f8b70 msgr2=0x55c1622f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622f8b70 0x55c1622f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd13e00640 1 -- 172.21.15.175:0/2878529903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c162303910 msgr2=0x55c16231b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c162303910 0x55c16231b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd13e00640 1 -- 172.21.15.175:0/2878529903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1622f5f00 con 0x55c1622fa0e0 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd13e00640 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c16231aff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55c1622f9610 tx=0x7fcd0000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7fe3f3089d78390 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd05200640 1 -- 172.21.15.175:0/2878529903 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00003f70 con 0x55c1622fa0e0 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd05200640 1 -- 172.21.15.175:0/2878529903 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd00005500 con 0x55c1622fa0e0 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.680+0000 7fcd05200640 1 -- 172.21.15.175:0/2878529903 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000116a0 con 0x55c1622fa0e0 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 -- 172.21.15.175:0/2878529903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 msgr2=0x55c16231aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c16231aff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55c1622f9610 tx=0x7fcd0000b390 comp rx=0 tx=0).stop 2024-07-29T00:43:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 -- 172.21.15.175:0/2878529903 shutdown_connections 2024-07-29T00:43:16.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c162303910 0x55c16231b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1622fa0e0 0x55c16231aff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 --2- 172.21.15.175:0/2878529903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622f8b70 0x55c1622f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 -- 172.21.15.175:0/2878529903 >> 172.21.15.175:0/2878529903 conn(0x55c1622c1950 msgr2=0x55c1622f4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:16.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 -- 172.21.15.175:0/2878529903 shutdown_connections 2024-07-29T00:43:16.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.681+0000 7fcd1de28740 1 -- 172.21.15.175:0/2878529903 wait complete. 2024-07-29T00:43:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 -- start start 2024-07-29T00:43:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 Processor -- start 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c16230be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c16230c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162307250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141b0ca0 con 0x55c1622f8b70 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141b05d0 con 0x55c1622fa0e0 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.682+0000 7fcd1de28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141ad830 con 0x55c162303910 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd1de28740 10 client.0 _ll_register_callbacks cb 0x7fcd080e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c16230be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c16230be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44416/0 (socket says 172.21.15.175:44416) 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 -- 172.21.15.175:0/3301835098 learned_addr learned my addr 172.21.15.175:0/3301835098 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd13e00640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c16230c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd19400640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162307250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 -- 172.21.15.175:0/3301835098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 msgr2=0x55c162307250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162307250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 -- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 msgr2=0x55c16230c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c16230c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 -- 172.21.15.175:0/3301835098 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c162307a80 con 0x55c1622f8b70 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd19400640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162307250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fcd18a00640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c16230be00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcd08097fb0 tx=0x7fcd08010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980a4b4ab7838c2e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.683+0000 7fccfa000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fccfa000640 1 -- 172.21.15.175:0/3301835098 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08020040 con 0x55c1622f8b70 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fccfa000640 1 -- 172.21.15.175:0/3301835098 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fcd080d29f0 con 0x55c1622f8b70 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd00005500 con 0x55c1622f8b70 2024-07-29T00:43:16.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fccfa000640 1 -- 172.21.15.175:0/3301835098 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcd080c5690 con 0x55c1622f8b70 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 10 client.16671 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c16237dbc0 con 0x55c1622f8b70 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fccfa000640 1 -- 172.21.15.175:0/3301835098 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd00005500 con 0x55c1622f8b70 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fccfa000640 1 -- 172.21.15.175:0/3301835098 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fcd08097c50 con 0x55c1622f8b70 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 10 client.16671 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 20 client.16671 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c16237e230 0x55c162380650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:16.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 4 client.16671 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.684+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c162266530 con 0x55c16237e230 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.685+0000 7fcd13e00640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c16237e230 0x55c162380650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.685+0000 7fcd13e00640 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c16237e230 0x55c162380650 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f868c6ad6c75b0 server_cookie=f13cf2ba6bd762ff in_seq=0 out_seq=0 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.685+0000 7fccfa000640 10 client.16671 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fccfa000640 1 -- 172.21.15.175:0/3301835098 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c162266530 con 0x55c16237e230 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fccfa000640 10 client.16671 handle_command_reply: tid=0 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 client.16671 shutdown 2024-07-29T00:43:16.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 msgr2=0x55c16230be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c16230be00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcd08097fb0 tx=0x7fcd08010d70 comp rx=0 tx=0).stop 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 shutdown_connections 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55c16237e230 0x55c162380650 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c162303910 0x55c162307250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c1622fa0e0 0x55c16230c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 --2- 172.21.15.175:0/3301835098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1622f8b70 0x55c16230be00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 >> 172.21.15.175:0/3301835098 conn(0x55c1622c1950 msgr2=0x55c162300840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:16.693 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fccfa000640 0 client.16671 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:16.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 shutdown_connections 2024-07-29T00:43:16.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 1 -- 172.21.15.175:0/3301835098 wait complete. 2024-07-29T00:43:16.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.686+0000 7fcd1de28740 20 client.16671 trim_cache size 0 max 16384 2024-07-29T00:43:16.694 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:43:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fccfc0a2cf0 msgr2=0x7fccfc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fccfc0a2cf0 0x7fccfc0a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004af0 tx=0x7fcd0c02ffa0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 msgr2=0x7fcd1419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:16.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 0x7fcd1419a160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800d950 tx=0x7fcd0800de10 comp rx=0 tx=0).stop 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 shutdown_connections 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1410df50 0x7fcd1419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fccfc0a2cf0 0x7fccfc0a51b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd14074b20 0x7fcd1419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 --2- 172.21.15.175:0/2508888048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd14074200 0x7fcd1419a160 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 >> 172.21.15.175:0/2508888048 conn(0x7fcd140fd370 msgr2=0x7fcd140feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 shutdown_connections 2024-07-29T00:43:16.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:16.687+0000 7fcd19e00640 1 -- 172.21.15.175:0/2508888048 wait complete. 2024-07-29T00:43:17.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:17 smithi175 ceph-mon[32138]: pgmap v208: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:17.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:17 smithi181 ceph-mon[36706]: pgmap v208: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:17 smithi204 ceph-mon[37368]: pgmap v208: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:18.720 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:43:18.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:43:19.189 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:19.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.846+0000 7fc87de00640 1 -- 172.21.15.175:0/3158297761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 msgr2=0x7fc878108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.846+0000 7fc87de00640 1 --2- 172.21.15.175:0/3158297761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc878108130 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc868009930 tx=0x7fc86802a420 comp rx=0 tx=0).stop 2024-07-29T00:43:19.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 -- 172.21.15.175:0/3158297761 shutdown_connections 2024-07-29T00:43:19.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 --2- 172.21.15.175:0/3158297761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878108670 0x7fc878114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 --2- 172.21.15.175:0/3158297761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc878108130 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 --2- 172.21.15.175:0/3158297761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87810dd50 0x7fc87810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 -- 172.21.15.175:0/3158297761 >> 172.21.15.175:0/3158297761 conn(0x7fc878077670 msgr2=0x7fc878079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 -- 172.21.15.175:0/3158297761 shutdown_connections 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.847+0000 7fc87de00640 1 -- 172.21.15.175:0/3158297761 wait complete. 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 Processor -- start 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 -- start start 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc87819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878108670 0x7fc87819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87810dd50 0x7fc8781a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc878118070 con 0x7fc878107d10 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc878117ef0 con 0x7fc878108670 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781181f0 con 0x7fc87810dd50 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.848+0000 7fc87ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc87819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc87819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47920/0 (socket says 172.21.15.175:47920) 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 -- 172.21.15.175:0/3123961166 learned_addr learned my addr 172.21.15.175:0/3123961166 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc86fe00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878108670 0x7fc87819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87d400640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87810dd50 0x7fc8781a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 -- 172.21.15.175:0/3123961166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87810dd50 msgr2=0x7fc8781a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87810dd50 0x7fc8781a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 -- 172.21.15.175:0/3123961166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878108670 msgr2=0x7fc87819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878108670 0x7fc87819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 -- 172.21.15.175:0/3123961166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8781a3580 con 0x7fc878107d10 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.849+0000 7fc87ca00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc87819e5d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e9f0 tx=0x7fc86000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.850+0000 7fc876000640 1 -- 172.21.15.175:0/3123961166 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000cd80 con 0x7fc878107d10 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.850+0000 7fc876000640 1 -- 172.21.15.175:0/3123961166 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc860005500 con 0x7fc878107d10 2024-07-29T00:43:19.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.850+0000 7fc876000640 1 -- 172.21.15.175:0/3123961166 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860010690 con 0x7fc878107d10 2024-07-29T00:43:19.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.850+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8781a3870 con 0x7fc878107d10 2024-07-29T00:43:19.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.850+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8781ab150 con 0x7fc878107d10 2024-07-29T00:43:19.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.853+0000 7fc88206c740 1 Processor -- start 2024-07-29T00:43:19.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 -- start start 2024-07-29T00:43:19.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b78050100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7805a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7805af00 0x558b7805d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b77fb58a0 con 0x558b78050640 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b77f4d710 con 0x558b7804fd20 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc88206c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b77fb1860 con 0x558b7805af00 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc876000640 1 -- 172.21.15.175:0/3123961166 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fc8600046c0 con 0x7fc878107d10 2024-07-29T00:43:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc87d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b78050100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.854+0000 7fc87d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b78050100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41476/0 (socket says 172.21.15.175:41476) 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc87d400640 1 -- 172.21.15.175:0/2298905311 learned_addr learned my addr 172.21.15.175:0/2298905311 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7805af00 0x558b7805d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc86fe00640 1 -- 172.21.15.175:0/2298905311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 msgr2=0x558b78050100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b78050100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc86fe00640 1 -- 172.21.15.175:0/2298905311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 msgr2=0x558b7805a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc87ca00640 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7805a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7805a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc86fe00640 1 -- 172.21.15.175:0/2298905311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b77d276a0 con 0x558b7805af00 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.855+0000 7fc87ca00640 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7805a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7805af00 0x558b7805d2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc868005940 tx=0x7fc868005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b10bda963e800c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc86dc00640 1 -- 172.21.15.175:0/2298905311 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868040070 con 0x558b7805af00 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc86dc00640 1 -- 172.21.15.175:0/2298905311 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8680318f0 con 0x558b7805af00 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc86dc00640 1 -- 172.21.15.175:0/2298905311 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868035040 con 0x558b7805af00 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc876000640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc8580a2d70 0x7fc8580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc876000640 1 -- 172.21.15.175:0/3123961166 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fc8600cf7f0 con 0x7fc878107d10 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.856+0000 7fc87d400640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc8580a2d70 0x7fc8580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 -- 172.21.15.175:0/2298905311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7805af00 msgr2=0x558b7805d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7805af00 0x558b7805d2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc868005940 tx=0x7fc868005c00 comp rx=0 tx=0).stop 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc87d400640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc8580a2d70 0x7fc8580a5230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc870005f40 tx=0x7fc8700095f0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 -- 172.21.15.175:0/2298905311 shutdown_connections 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7805af00 0x558b7805d2f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7805a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 --2- 172.21.15.175:0/2298905311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b78050100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 -- 172.21.15.175:0/2298905311 >> 172.21.15.175:0/2298905311 conn(0x558b7804c2b0 msgr2=0x558b7804c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 -- 172.21.15.175:0/2298905311 shutdown_connections 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.857+0000 7fc88206c740 1 -- 172.21.15.175:0/2298905311 wait complete. 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.858+0000 7fc88206c740 1 -- start start 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.858+0000 7fc88206c740 1 Processor -- start 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 0x558b78057de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b78058320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78058860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b7805dad0 con 0x558b7804fd20 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b77f708c0 con 0x558b7805af00 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b7805dc50 con 0x558b78050640 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc88206c740 10 client.0 _ll_register_callbacks cb 0x558b77f5ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc87ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 0x558b78057de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc87ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 0x558b78057de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47942/0 (socket says 172.21.15.175:47942) 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc87ca00640 1 -- 172.21.15.175:0/1889674438 learned_addr learned my addr 172.21.15.175:0/1889674438 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:19.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc87d400640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78058860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc86fe00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b78058320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.859+0000 7fc87ca00640 1 -- 172.21.15.175:0/1889674438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 msgr2=0x558b78058320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc87ca00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b78058320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc87ca00640 1 -- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 msgr2=0x558b78058860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc87ca00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78058860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc87ca00640 1 -- 172.21.15.175:0/1889674438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b77f53bc0 con 0x558b7804fd20 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc87d400640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78058860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:19.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc86fe00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b78058320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc87ca00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 0x558b78057de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8600ccf50 tx=0x7fc86009b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3929c348e5018c09 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc866000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc866000640 1 -- 172.21.15.175:0/1889674438 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8600a46a0 con 0x558b7804fd20 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc866000640 1 -- 172.21.15.175:0/1889674438 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fc86009b110 con 0x558b7804fd20 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8680318f0 con 0x558b7804fd20 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.860+0000 7fc866000640 1 -- 172.21.15.175:0/1889674438 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc8600a0070 con 0x558b7804fd20 2024-07-29T00:43:19.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 10 client.16680 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558b78069a80 con 0x558b7804fd20 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc866000640 1 -- 172.21.15.175:0/1889674438 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8680318f0 con 0x558b7804fd20 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc866000640 1 -- 172.21.15.175:0/1889674438 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc860097ba0 con 0x558b7804fd20 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 10 client.16680 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 20 client.16680 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b7806afe0 0x558b7806d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 4 client.16680 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:19.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.861+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b77f75c90 con 0x558b7806afe0 2024-07-29T00:43:19.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.862+0000 7fc86fe00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b7806afe0 0x558b7806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.862+0000 7fc86fe00640 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b7806afe0 0x558b7806d410 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ea92dc36f768cf server_cookie=232ca6f80459d311 in_seq=0 out_seq=0 2024-07-29T00:43:19.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.862+0000 7fc866000640 10 client.16680 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:19.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc866000640 1 -- 172.21.15.175:0/1889674438 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b77f75c90 con 0x558b7806afe0 2024-07-29T00:43:19.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc866000640 10 client.16680 handle_command_reply: tid=0 2024-07-29T00:43:19.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 client.16680 shutdown 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 msgr2=0x558b78057de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 0x558b78057de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8600ccf50 tx=0x7fc86009b880 comp rx=0 tx=0).stop 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 shutdown_connections 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b7806afe0 0x558b7806d410 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78058860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b78058320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 --2- 172.21.15.175:0/1889674438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7804fd20 0x558b78057de0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 >> 172.21.15.175:0/1889674438 conn(0x558b7804c2b0 msgr2=0x558b78051fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:19.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.863+0000 7fc866000640 0 client.16680 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:19.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.864+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 shutdown_connections 2024-07-29T00:43:19.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.864+0000 7fc88206c740 1 -- 172.21.15.175:0/1889674438 wait complete. 2024-07-29T00:43:19.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.864+0000 7fc88206c740 20 client.16680 trim_cache size 0 max 16384 2024-07-29T00:43:19.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 Processor -- start 2024-07-29T00:43:19.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 -- start start 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7804fd20 0x558b78052780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7804b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78072d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b3b40 con 0x558b78050640 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b5740 con 0x558b7805af00 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.883+0000 7fc88206c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b4210 con 0x558b7804fd20 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78072d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41486/0 (socket says 172.21.15.175:41486) 2024-07-29T00:43:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7804fd20 0x558b78052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7804b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 -- 172.21.15.175:0/2270959014 learned_addr learned my addr 172.21.15.175:0/2270959014 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 -- 172.21.15.175:0/2270959014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7804fd20 msgr2=0x558b78052780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7804fd20 0x558b78052780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 -- 172.21.15.175:0/2270959014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 msgr2=0x558b7804b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7804b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 -- 172.21.15.175:0/2270959014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b78047c70 con 0x558b7805af00 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7804b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.884+0000 7fc87d400640 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78072d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc87001e550 tx=0x7fc870019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79754c05a1d0d091 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:19.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.885+0000 7fc86d200640 1 -- 172.21.15.175:0/2270959014 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8700115c0 con 0x558b7805af00 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.885+0000 7fc86d200640 1 -- 172.21.15.175:0/2270959014 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc870011c40 con 0x558b7805af00 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.885+0000 7fc86d200640 1 -- 172.21.15.175:0/2270959014 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870029d60 con 0x558b7805af00 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.885+0000 7fc88206c740 1 -- 172.21.15.175:0/2270959014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 msgr2=0x558b78072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.885+0000 7fc88206c740 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78072d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc87001e550 tx=0x7fc870019000 comp rx=0 tx=0).stop 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 -- 172.21.15.175:0/2270959014 shutdown_connections 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7805af00 0x558b78072d20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b78050640 0x558b7804b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 --2- 172.21.15.175:0/2270959014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b7804fd20 0x558b78052780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 -- 172.21.15.175:0/2270959014 >> 172.21.15.175:0/2270959014 conn(0x558b7804c2b0 msgr2=0x558b7804ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 -- 172.21.15.175:0/2270959014 shutdown_connections 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 -- 172.21.15.175:0/2270959014 wait complete. 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 -- start start 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.886+0000 7fc88206c740 1 Processor -- start 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b7805dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b7805e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 0x558b780d3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b4fb0 con 0x558b7805af00 2024-07-29T00:43:19.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b48e0 con 0x558b7804fd20 2024-07-29T00:43:19.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b1b40 con 0x558b78050640 2024-07-29T00:43:19.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc88206c740 10 client.0 _ll_register_callbacks cb 0x7fc86009ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:19.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc87d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 0x558b780d3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc87d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 0x558b780d3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47958/0 (socket says 172.21.15.175:47958) 2024-07-29T00:43:19.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc87d400640 1 -- 172.21.15.175:0/2000494931 learned_addr learned my addr 172.21.15.175:0/2000494931 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:19.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc87ca00640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b7805dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.887+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b7805e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc87d400640 1 -- 172.21.15.175:0/2000494931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 msgr2=0x558b7805e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc87d400640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b7805e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc87d400640 1 -- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 msgr2=0x558b7805dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc87d400640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b7805dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc87d400640 1 -- 172.21.15.175:0/2000494931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b780d4220 con 0x558b7805af00 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc86fe00640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b7805e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:19.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc87d400640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 0x558b780d3ce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc870029860 tx=0x7fc870029480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572dcd5c23551243 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc866000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc866000640 1 -- 172.21.15.175:0/2000494931 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870029a60 con 0x558b7805af00 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc866000640 1 -- 172.21.15.175:0/2000494931 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fc870029c00 con 0x558b7805af00 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.888+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc870011c40 con 0x558b7805af00 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc866000640 1 -- 172.21.15.175:0/2000494931 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc870029e00 con 0x558b7805af00 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 10 client.16686 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558b780d5410 con 0x558b7805af00 2024-07-29T00:43:19.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc866000640 1 -- 172.21.15.175:0/2000494931 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc870011c40 con 0x558b7805af00 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc866000640 1 -- 172.21.15.175:0/2000494931 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc870030070 con 0x558b7805af00 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 10 client.16686 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 20 client.16686 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b780d5ab0 0x558b780d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 4 client.16686 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.889+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b77fc2e10 con 0x558b780d5ab0 2024-07-29T00:43:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.890+0000 7fc87ca00640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b780d5ab0 0x558b780d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.890+0000 7fc87ca00640 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b780d5ab0 0x558b780d7ed0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82db5bf3937880a8 server_cookie=a79bcb9d2c3ceb7f in_seq=0 out_seq=0 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.890+0000 7fc866000640 10 client.16686 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.891+0000 7fc866000640 1 -- 172.21.15.175:0/2000494931 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b77fc2e10 con 0x558b780d5ab0 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.891+0000 7fc866000640 10 client.16686 handle_command_reply: tid=0 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.891+0000 7fc88206c740 1 client.16686 shutdown 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 msgr2=0x558b780d3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 0x558b780d3ce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc870029860 tx=0x7fc870029480 comp rx=0 tx=0).stop 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 shutdown_connections 2024-07-29T00:43:19.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558b780d5ab0 0x558b780d7ed0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b7805af00 0x558b780d3ce0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b78050640 0x558b7805e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc866000640 0 client.16686 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 --2- 172.21.15.175:0/2000494931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b7804fd20 0x558b7805dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 >> 172.21.15.175:0/2000494931 conn(0x558b7804c2b0 msgr2=0x558b7804b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 shutdown_connections 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 1 -- 172.21.15.175:0/2000494931 wait complete. 2024-07-29T00:43:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.892+0000 7fc88206c740 20 client.16686 trim_cache size 0 max 16384 2024-07-29T00:43:19.898 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:43:19.898 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:43:19.898 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "a365168a-db76-472c-8f73-3e2c70400c85", 2024-07-29T00:43:19.898 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:43:19.898 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:43:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc8580a2d70 msgr2=0x7fc8580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc8580a2d70 0x7fc8580a5230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc870005f40 tx=0x7fc8700095f0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 msgr2=0x7fc87819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc87819e5d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e9f0 tx=0x7fc86000edf0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 shutdown_connections 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87810dd50 0x7fc8781a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc878108670 0x7fc87819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fc8580a2d70 0x7fc8580a5230 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 --2- 172.21.15.175:0/3123961166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc878107d10 0x7fc87819e5d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.893+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 >> 172.21.15.175:0/3123961166 conn(0x7fc878077670 msgr2=0x7fc878078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.894+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 shutdown_connections 2024-07-29T00:43:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:19.894+0000 7fc87de00640 1 -- 172.21.15.175:0/3123961166 wait complete. 2024-07-29T00:43:19.961 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:19 smithi175 ceph-mon[32138]: pgmap v209: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:19 smithi181 ceph-mon[36706]: pgmap v209: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:19 smithi204 ceph-mon[37368]: pgmap v209: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:20 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:43:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:20 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:43:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:43:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:21.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:20 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.514 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:21.913 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:21.913 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.913 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:21.913 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.914 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:43:21.914 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:21.914 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:21 smithi175 ceph-mon[32138]: pgmap v210: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:22.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:22.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:22.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:43:22.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:22.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:21 smithi181 ceph-mon[36706]: pgmap v210: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:22.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:22.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:22.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:43:22.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:43:22.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:21 smithi204 ceph-mon[37368]: pgmap v210: 97 pgs: 97 active+clean; 136 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-07-29T00:43:22.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.156+0000 7f9668a00640 1 -- 172.21.15.175:0/2047378125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664100830 msgr2=0x7f9664100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.156+0000 7f9668a00640 1 --2- 172.21.15.175:0/2047378125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664100830 0x7f9664100c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9654009900 tx=0x7f965402a470 comp rx=0 tx=0).stop 2024-07-29T00:43:22.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.156+0000 7f9668a00640 1 -- 172.21.15.175:0/2047378125 shutdown_connections 2024-07-29T00:43:22.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.157+0000 7f9668a00640 1 --2- 172.21.15.175:0/2047378125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9664105c40 0x7f96641060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.157+0000 7f9668a00640 1 --2- 172.21.15.175:0/2047378125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664101150 0x7f9664105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.157+0000 7f9668a00640 1 --2- 172.21.15.175:0/2047378125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664100830 0x7f9664100c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.157+0000 7f9668a00640 1 -- 172.21.15.175:0/2047378125 >> 172.21.15.175:0/2047378125 conn(0x7f96640fc4c0 msgr2=0x7f96640fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:22.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.157+0000 7f9668a00640 1 -- 172.21.15.175:0/2047378125 shutdown_connections 2024-07-29T00:43:22.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.157+0000 7f9668a00640 1 -- 172.21.15.175:0/2047378125 wait complete. 2024-07-29T00:43:22.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 Processor -- start 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 -- start start 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 0x7f966419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664101150 0x7f966419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9664105c40 0x7f96641a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9664118270 con 0x7f9664101150 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96641180f0 con 0x7f9664100830 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.158+0000 7f9668a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96641183f0 con 0x7f9664105c40 2024-07-29T00:43:22.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 0x7f966419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 0x7f966419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:41512/0 (socket says 172.21.15.175:41512) 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 -- 172.21.15.175:0/2662095448 learned_addr learned my addr 172.21.15.175:0/2662095448 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9659400640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664101150 0x7f966419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9662800640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9664105c40 0x7f96641a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 -- 172.21.15.175:0/2662095448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9664105c40 msgr2=0x7f96641a2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9664105c40 0x7f96641a2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 -- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664101150 msgr2=0x7f966419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664101150 0x7f966419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.159+0000 7f9661e00640 1 -- 172.21.15.175:0/2662095448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96641a36c0 con 0x7f9664100830 2024-07-29T00:43:22.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.160+0000 7f9661e00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 0x7f966419e710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9654006010 tx=0x7f9654004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:22.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.160+0000 7f965b400640 1 -- 172.21.15.175:0/2662095448 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965403f030 con 0x7f9664100830 2024-07-29T00:43:22.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.160+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96641a3950 con 0x7f9664100830 2024-07-29T00:43:22.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.160+0000 7f965b400640 1 -- 172.21.15.175:0/2662095448 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9654004440 con 0x7f9664100830 2024-07-29T00:43:22.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.160+0000 7f965b400640 1 -- 172.21.15.175:0/2662095448 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654035070 con 0x7f9664100830 2024-07-29T00:43:22.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.160+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96641ab290 con 0x7f9664100830 2024-07-29T00:43:22.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 Processor -- start 2024-07-29T00:43:22.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f965b400640 1 -- 172.21.15.175:0/2662095448 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9654042030 con 0x7f9664100830 2024-07-29T00:43:22.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 -- start start 2024-07-29T00:43:22.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b4930 0x5636ea3b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5636ea319ad0 con 0x5636ea3bfb10 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5636ea2b1af0 con 0x5636ea3b4930 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f966b628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636ea315a90 con 0x5636ea3b5250 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.164+0000 7f9661e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45242/0 (socket says 172.21.15.175:45242) 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 -- 172.21.15.175:0/1833119505 learned_addr learned my addr 172.21.15.175:0/1833119505 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9662800640 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b4930 0x5636ea3b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9659400640 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 -- 172.21.15.175:0/1833119505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b4930 msgr2=0x5636ea3b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b4930 0x5636ea3b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 -- 172.21.15.175:0/1833119505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 msgr2=0x5636ea3c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 -- 172.21.15.175:0/1833119505 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ea08b3c0 con 0x5636ea3b5250 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9659400640 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:22.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.165+0000 7f9661e00640 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bf5d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96540ecaa0 tx=0x7f96540f56c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaef78dc7415885c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:22.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f965b400640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f96480a2cf0 0x7f96480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f9653000640 1 -- 172.21.15.175:0/1833119505 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540efcc0 con 0x5636ea3b5250 2024-07-29T00:43:22.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f9653000640 1 -- 172.21.15.175:0/1833119505 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96540fc030 con 0x5636ea3b5250 2024-07-29T00:43:22.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f9653000640 1 -- 172.21.15.175:0/1833119505 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540bb340 con 0x5636ea3b5250 2024-07-29T00:43:22.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f9662800640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f96480a2cf0 0x7f96480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f965b400640 1 -- 172.21.15.175:0/2662095448 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f96540ef4c0 con 0x7f9664100830 2024-07-29T00:43:22.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f966b628740 1 -- 172.21.15.175:0/1833119505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 msgr2=0x5636ea3bf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.166+0000 7f966b628740 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bf5d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96540ecaa0 tx=0x7f96540f56c0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 -- 172.21.15.175:0/1833119505 shutdown_connections 2024-07-29T00:43:22.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bf5d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 --2- 172.21.15.175:0/1833119505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b4930 0x5636ea3b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f9662800640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f96480a2cf0 0x7f96480a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f965c005fd0 tx=0x7f965c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 -- 172.21.15.175:0/1833119505 >> 172.21.15.175:0/1833119505 conn(0x5636ea3b0ec0 msgr2=0x5636ea3b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 -- 172.21.15.175:0/1833119505 shutdown_connections 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 -- 172.21.15.175:0/1833119505 wait complete. 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.167+0000 7f966b628740 1 -- start start 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 Processor -- start 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3bfb10 0x5636ea3bd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5636ea3c24a0 con 0x5636ea3b4930 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5636ea2d4bc0 con 0x5636ea3b5250 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636ea3c2620 con 0x5636ea3bfb10 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f966b628740 10 client.0 _ll_register_callbacks cb 0x5636ea2c0d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f9661e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f9661e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3bc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48010/0 (socket says 172.21.15.175:48010) 2024-07-29T00:43:22.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f9661e00640 1 -- 172.21.15.175:0/1425699127 learned_addr learned my addr 172.21.15.175:0/1425699127 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f9662800640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3bfb10 0x5636ea3bd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.168+0000 7f9659400640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9661e00640 1 -- 172.21.15.175:0/1425699127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3bfb10 msgr2=0x5636ea3bd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9661e00640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3bfb10 0x5636ea3bd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9661e00640 1 -- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 msgr2=0x5636ea3bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9661e00640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9661e00640 1 -- 172.21.15.175:0/1425699127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ea3cb880 con 0x5636ea3b4930 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9662800640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3bfb10 0x5636ea3bd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:22.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9661e00640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3bc910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f96540c4070 tx=0x7f9654005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32812a3931f21694 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9647400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.169+0000 7f9647400640 1 -- 172.21.15.175:0/1425699127 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540c0a30 con 0x5636ea3b4930 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f9647400640 1 -- 172.21.15.175:0/1425699127 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f96540b9ce0 con 0x5636ea3b4930 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96540fc030 con 0x5636ea3b4930 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f9647400640 1 -- 172.21.15.175:0/1425699127 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f96540c0c10 con 0x5636ea3b4930 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f966b628740 10 client.16698 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5636ea3c24a0 con 0x5636ea3b4930 2024-07-29T00:43:22.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f9647400640 1 -- 172.21.15.175:0/1425699127 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96540cfe50 con 0x5636ea3b4930 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f9647400640 1 -- 172.21.15.175:0/1425699127 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f96540b8050 con 0x5636ea3b4930 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f966b628740 10 client.16698 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f966b628740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.170+0000 7f966b628740 20 client.16698 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.171+0000 7f966b628740 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea3cf3a0 0x5636ea3d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.171+0000 7f966b628740 4 client.16698 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.171+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636ea327040 con 0x5636ea3cf3a0 2024-07-29T00:43:22.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.171+0000 7f9659400640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea3cf3a0 0x5636ea3d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.171+0000 7f9659400640 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea3cf3a0 0x5636ea3d17d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ab0f7af85f22da server_cookie=52ca5cdc7c0508c2 in_seq=0 out_seq=0 2024-07-29T00:43:22.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.171+0000 7f9647400640 10 client.16698 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:22.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.172+0000 7f9647400640 1 -- 172.21.15.175:0/1425699127 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636ea327040 con 0x5636ea3cf3a0 2024-07-29T00:43:22.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.172+0000 7f9647400640 10 client.16698 handle_command_reply: tid=0 2024-07-29T00:43:22.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.172+0000 7f966b628740 1 client.16698 shutdown 2024-07-29T00:43:22.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 msgr2=0x5636ea3bc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3bc910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f96540c4070 tx=0x7f9654005890 comp rx=0 tx=0).stop 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 shutdown_connections 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea3cf3a0 0x5636ea3d17d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3bfb10 0x5636ea3bd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 --2- 172.21.15.175:0/1425699127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3bc910 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 >> 172.21.15.175:0/1425699127 conn(0x5636ea3b0ec0 msgr2=0x5636ea3b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:22.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f9647400640 0 client.16698 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:22.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 shutdown_connections 2024-07-29T00:43:22.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 1 -- 172.21.15.175:0/1425699127 wait complete. 2024-07-29T00:43:22.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.173+0000 7f966b628740 20 client.16698 trim_cache size 0 max 16384 2024-07-29T00:43:22.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.192+0000 7f966b628740 1 Processor -- start 2024-07-29T00:43:22.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.192+0000 7f966b628740 1 -- start start 2024-07-29T00:43:22.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.192+0000 7f966b628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f966b628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f966b628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3bfb10 0x5636ea3afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f966b628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96641b3d10 con 0x5636ea3b4930 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f966b628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96641b5910 con 0x5636ea3bfb10 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f966b628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96641b43e0 con 0x5636ea3b5250 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9661e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9661e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48026/0 (socket says 172.21.15.175:48026) 2024-07-29T00:43:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9659400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9661e00640 1 -- 172.21.15.175:0/2025499340 learned_addr learned my addr 172.21.15.175:0/2025499340 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:22.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9662800640 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3bfb10 0x5636ea3afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9659400640 1 -- 172.21.15.175:0/2025499340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3bfb10 msgr2=0x5636ea3afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9659400640 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3bfb10 0x5636ea3afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9659400640 1 -- 172.21.15.175:0/2025499340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 msgr2=0x5636ea3b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9659400640 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.193+0000 7f9659400640 1 -- 172.21.15.175:0/2025499340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ea3b4760 con 0x5636ea3b5250 2024-07-29T00:43:22.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.194+0000 7f9662800640 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3bfb10 0x5636ea3afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:22.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.194+0000 7f9659400640 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bd1e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5636ea3d20d0 tx=0x7f964c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10be097e15cb504 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:22.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.194+0000 7f9652600640 1 -- 172.21.15.175:0/2025499340 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c022070 con 0x5636ea3b5250 2024-07-29T00:43:22.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.194+0000 7f9652600640 1 -- 172.21.15.175:0/2025499340 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f964c01fb10 con 0x5636ea3b5250 2024-07-29T00:43:22.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.194+0000 7f9652600640 1 -- 172.21.15.175:0/2025499340 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c026c70 con 0x5636ea3b5250 2024-07-29T00:43:22.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 -- 172.21.15.175:0/2025499340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 msgr2=0x5636ea3bd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bd1e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5636ea3d20d0 tx=0x7f964c005f90 comp rx=0 tx=0).stop 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 -- 172.21.15.175:0/2025499340 shutdown_connections 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3bfb10 0x5636ea3afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b5250 0x5636ea3bd1e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 --2- 172.21.15.175:0/2025499340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3b4930 0x5636ea3b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 -- 172.21.15.175:0/2025499340 >> 172.21.15.175:0/2025499340 conn(0x5636ea3b0ec0 msgr2=0x5636ea3af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 -- 172.21.15.175:0/2025499340 shutdown_connections 2024-07-29T00:43:22.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 -- 172.21.15.175:0/2025499340 wait complete. 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.195+0000 7f966b628740 1 -- start start 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 Processor -- start 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 0x5636ea3c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96641b5180 con 0x5636ea3bfb10 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96641b4ab0 con 0x5636ea3b5250 2024-07-29T00:43:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96641b1d10 con 0x5636ea3b4930 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f966b628740 10 client.0 _ll_register_callbacks cb 0x7f96540f5480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f9661e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 0x5636ea3c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f9661e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 0x5636ea3c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45278/0 (socket says 172.21.15.175:45278) 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f9661e00640 1 -- 172.21.15.175:0/4191589796 learned_addr learned my addr 172.21.15.175:0/4191589796 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.196+0000 7f9659400640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9662800640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9661e00640 1 -- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 msgr2=0x5636ea3c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9661e00640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9661e00640 1 -- 172.21.15.175:0/4191589796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 msgr2=0x5636ea3c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9661e00640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9661e00640 1 -- 172.21.15.175:0/4191589796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ea4380f0 con 0x5636ea3b4930 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9659400640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9662800640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9661e00640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 0x5636ea3c80c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f96540b7e40 tx=0x7f96540f5cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21eb6f6848111227 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:22.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9647400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:43:22.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9647400640 1 -- 172.21.15.175:0/4191589796 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654049070 con 0x5636ea3b4930 2024-07-29T00:43:22.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f9647400640 1 -- 172.21.15.175:0/4191589796 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f96540b8050 con 0x5636ea3b4930 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.197+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f964c01fb10 con 0x5636ea3b4930 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f9647400640 1 -- 172.21.15.175:0/4191589796 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f96540c0e70 con 0x5636ea3b4930 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 10 client.26806 fetch_fsmap learned FSMap version 40 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5636ea4380f0 con 0x5636ea3b4930 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f9647400640 1 -- 172.21.15.175:0/4191589796 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f964c01fb10 con 0x5636ea3b4930 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f9647400640 1 -- 172.21.15.175:0/4191589796 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f96540ecde0 con 0x5636ea3b4930 2024-07-29T00:43:22.205 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 10 client.26806 fetch_fsmap finished waiting for FSMap version 40 2024-07-29T00:43:22.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 10 client.26806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:22.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 20 client.26806 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:22.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea439660 0x5636ea43ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:22.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 4 client.26806 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:43:22.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.198+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636ea321550 con 0x5636ea439660 2024-07-29T00:43:22.206 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.199+0000 7f9659400640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea439660 0x5636ea43ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.199+0000 7f9659400640 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea439660 0x5636ea43ba80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92538e350dee9bf5 server_cookie=b58a49593d84c7c7 in_seq=0 out_seq=0 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.199+0000 7f9647400640 10 client.26806 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.199+0000 7f9647400640 1 -- 172.21.15.175:0/4191589796 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5636ea321550 con 0x5636ea439660 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.199+0000 7f9647400640 10 client.26806 handle_command_reply: tid=0 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.199+0000 7f966b628740 1 client.26806 shutdown 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 msgr2=0x5636ea3c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 0x5636ea3c80c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f96540b7e40 tx=0x7f96540f5cf0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 shutdown_connections 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x5636ea439660 0x5636ea43ba80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636ea3bfb10 0x5636ea3c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636ea3b5250 0x5636ea3c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f9647400640 0 client.26806 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 --2- 172.21.15.175:0/4191589796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5636ea3b4930 0x5636ea3c80c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.208 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 >> 172.21.15.175:0/4191589796 conn(0x5636ea3b0ec0 msgr2=0x5636ea3cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:22.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 shutdown_connections 2024-07-29T00:43:22.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 1 -- 172.21.15.175:0/4191589796 wait complete. 2024-07-29T00:43:22.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.200+0000 7f966b628740 20 client.26806 trim_cache size 0 max 16384 2024-07-29T00:43:22.209 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:43:22.209 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:43:22.209 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout: "a365168a-db76-472c-8f73-3e2c70400c85": { 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "a365168a-db76-472c-8f73-3e2c70400c85", 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:43:22.210 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:43:22.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f96480a2cf0 msgr2=0x7f96480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f96480a2cf0 0x7f96480a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f965c005fd0 tx=0x7f965c009500 comp rx=0 tx=0).stop 2024-07-29T00:43:22.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 msgr2=0x7f966419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:22.211 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 0x7f966419e710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9654006010 tx=0x7f9654004040 comp rx=0 tx=0).stop 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 shutdown_connections 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9664105c40 0x7f96641a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f96480a2cf0 0x7f96480a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9664101150 0x7f966419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 --2- 172.21.15.175:0/2662095448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9664100830 0x7f966419e710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 >> 172.21.15.175:0/2662095448 conn(0x7f96640fc4c0 msgr2=0x7f96640fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:22.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 shutdown_connections 2024-07-29T00:43:22.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:22.201+0000 7f9668a00640 1 -- 172.21.15.175:0/2662095448 wait complete. 2024-07-29T00:43:23.291 INFO:tasks.cephfs.filesystem:scrub status for tag:a365168a-db76-472c-8f73-3e2c70400c85 - {'path': '/', 'tag': 'a365168a-db76-472c-8f73-3e2c70400c85', 'options': 'recursive,force'} 2024-07-29T00:43:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:23 smithi181 ceph-mon[36706]: pgmap v211: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.2 MiB/s wr, 65 op/s 2024-07-29T00:43:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:23 smithi175 ceph-mon[32138]: pgmap v211: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.2 MiB/s wr, 65 op/s 2024-07-29T00:43:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:23 smithi204 ceph-mon[37368]: pgmap v211: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.2 MiB/s wr, 65 op/s 2024-07-29T00:43:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:25 smithi181 ceph-mon[36706]: pgmap v212: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-07-29T00:43:26.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:25 smithi175 ceph-mon[32138]: pgmap v212: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-07-29T00:43:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:25 smithi204 ceph-mon[37368]: pgmap v212: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-07-29T00:43:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:27 smithi181 ceph-mon[36706]: pgmap v213: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-07-29T00:43:28.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:27 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:43:28.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:27 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:43:28.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:27 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:43:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:27 smithi175 ceph-mon[32138]: pgmap v213: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-07-29T00:43:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:27 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:43:28.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:27 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:43:28.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:27 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:43:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:27 smithi204 ceph-mon[37368]: pgmap v213: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-07-29T00:43:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:27 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:43:28.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:27 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:43:28.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:27 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:43:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:28 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:43:28.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:28 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:43:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:28 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:43:29.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:28 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:43:29.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:28 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:43:29.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:28 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:43:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:29 smithi181 ceph-mon[36706]: pgmap v214: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-07-29T00:43:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:29 smithi175 ceph-mon[32138]: pgmap v214: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-07-29T00:43:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:29 smithi204 ceph-mon[37368]: pgmap v214: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-07-29T00:43:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:31 smithi181 ceph-mon[36706]: pgmap v215: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-07-29T00:43:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:31 smithi175 ceph-mon[32138]: pgmap v215: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-07-29T00:43:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:31 smithi204 ceph-mon[37368]: pgmap v215: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-07-29T00:43:33.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:33 smithi181 ceph-mon[36706]: pgmap v216: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 64 KiB/s wr, 28 op/s 2024-07-29T00:43:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:33 smithi175 ceph-mon[32138]: pgmap v216: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 64 KiB/s wr, 28 op/s 2024-07-29T00:43:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:33 smithi204 ceph-mon[37368]: pgmap v216: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 64 KiB/s wr, 28 op/s 2024-07-29T00:43:35.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:35 smithi181 ceph-mon[36706]: pgmap v217: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-07-29T00:43:36.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:35 smithi175 ceph-mon[32138]: pgmap v217: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-07-29T00:43:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:35 smithi204 ceph-mon[37368]: pgmap v217: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-07-29T00:43:37.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:37 smithi181 ceph-mon[36706]: pgmap v218: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-07-29T00:43:38.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:37 smithi175 ceph-mon[32138]: pgmap v218: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-07-29T00:43:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:37 smithi204 ceph-mon[37368]: pgmap v218: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-07-29T00:43:39.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:39 smithi181 ceph-mon[36706]: pgmap v219: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-29T00:43:40.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:39 smithi175 ceph-mon[32138]: pgmap v219: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-29T00:43:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:39 smithi204 ceph-mon[37368]: pgmap v219: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-29T00:43:41.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:41 smithi181 ceph-mon[36706]: pgmap v220: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-29T00:43:42.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:41 smithi175 ceph-mon[32138]: pgmap v220: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-29T00:43:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:41 smithi204 ceph-mon[37368]: pgmap v220: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-29T00:43:44.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:43 smithi175 ceph-mon[32138]: pgmap v221: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-29T00:43:44.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:43 smithi204 ceph-mon[37368]: pgmap v221: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-29T00:43:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:43 smithi181 ceph-mon[36706]: pgmap v221: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-29T00:43:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:45 smithi175 ceph-mon[32138]: pgmap v222: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-07-29T00:43:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:45 smithi204 ceph-mon[37368]: pgmap v222: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-07-29T00:43:46.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:45 smithi181 ceph-mon[36706]: pgmap v222: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-07-29T00:43:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:47 smithi175 ceph-mon[32138]: pgmap v223: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-29T00:43:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:47 smithi204 ceph-mon[37368]: pgmap v223: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-29T00:43:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:47 smithi181 ceph-mon[36706]: pgmap v223: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-29T00:43:49.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:48 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:49.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:48 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:49.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:48 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:49 smithi204 ceph-mon[37368]: pgmap v224: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-29T00:43:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:49 smithi181 ceph-mon[36706]: pgmap v224: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-29T00:43:50.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:49 smithi175 ceph-mon[32138]: pgmap v224: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-29T00:43:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:51 smithi204 ceph-mon[37368]: pgmap v225: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-29T00:43:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:51 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:43:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:51 smithi181 ceph-mon[36706]: pgmap v225: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-29T00:43:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:51 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:43:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:51 smithi175 ceph-mon[32138]: pgmap v225: 97 pgs: 97 active+clean; 127 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-29T00:43:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:51 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:43:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:52 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:43:53.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:52 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:43:53.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:52 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:43:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:52 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:43:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:52 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:43:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:52 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:43:53.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:52 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:43:53.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:52 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:43:53.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:52 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:43:53.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:43:53.768 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:54.115 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:53 smithi175 ceph-mon[32138]: pgmap v226: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.0 KiB/s wr, 14 op/s 2024-07-29T00:43:54.115 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:53 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:53 smithi181 ceph-mon[36706]: pgmap v226: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.0 KiB/s wr, 14 op/s 2024-07-29T00:43:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:53 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:54.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:53 smithi204 ceph-mon[37368]: pgmap v226: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.0 KiB/s wr, 14 op/s 2024-07-29T00:43:54.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:53 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:54.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 -- 172.21.15.175:0/3714736055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 msgr2=0x7fa60c110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 --2- 172.21.15.175:0/3714736055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c110790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa604009c10 tx=0x7fa60402a6e0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 -- 172.21.15.175:0/3714736055 shutdown_connections 2024-07-29T00:43:54.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 --2- 172.21.15.175:0/3714736055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c110790 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 --2- 172.21.15.175:0/3714736055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 --2- 172.21.15.175:0/3714736055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 0x7fa60c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 -- 172.21.15.175:0/3714736055 >> 172.21.15.175:0/3714736055 conn(0x7fa60c0ff5c0 msgr2=0x7fa60c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.459+0000 7fa611a00640 1 -- 172.21.15.175:0/3714736055 shutdown_connections 2024-07-29T00:43:54.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.460+0000 7fa611a00640 1 -- 172.21.15.175:0/3714736055 wait complete. 2024-07-29T00:43:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.460+0000 7fa611a00640 1 Processor -- start 2024-07-29T00:43:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 -- start start 2024-07-29T00:43:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 0x7fa60c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c113cc0 con 0x7fa60c1038c0 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c113b40 con 0x7fa60c1098c0 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa611a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c113e40 con 0x7fa60c104260 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa60be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa60be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c19a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45854/0 (socket says 172.21.15.175:45854) 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa60b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.461+0000 7fa60be00640 1 -- 172.21.15.175:0/802518595 learned_addr learned my addr 172.21.15.175:0/802518595 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa611000640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 0x7fa60c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60be00640 1 -- 172.21.15.175:0/802518595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 msgr2=0x7fa60c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60be00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60be00640 1 -- 172.21.15.175:0/802518595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 msgr2=0x7fa60c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60be00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 0x7fa60c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60be00640 1 -- 172.21.15.175:0/802518595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c19f140 con 0x7fa60c1038c0 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa611000640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 0x7fa60c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60b400640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:54.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.462+0000 7fa60be00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c19a190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e9f0 tx=0x7fa5fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.463+0000 7fa608c00640 1 -- 172.21.15.175:0/802518595 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00ccc0 con 0x7fa60c1038c0 2024-07-29T00:43:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.463+0000 7fa608c00640 1 -- 172.21.15.175:0/802518595 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5fc00ce60 con 0x7fa60c1038c0 2024-07-29T00:43:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.463+0000 7fa608c00640 1 -- 172.21.15.175:0/802518595 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc010690 con 0x7fa60c1038c0 2024-07-29T00:43:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.463+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c19f430 con 0x7fa60c1038c0 2024-07-29T00:43:54.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.463+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c1a6d10 con 0x7fa60c1038c0 2024-07-29T00:43:54.467 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.466+0000 7fa608c00640 1 -- 172.21.15.175:0/802518595 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa5fc0046c0 con 0x7fa60c1038c0 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.466+0000 7fa615bbc740 1 Processor -- start 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.466+0000 7fa615bbc740 1 -- start start 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 0x558277a60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a6b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a6bcf0 0x558277a6e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa615bbc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5582779c5ad0 con 0x558277a60b10 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa615bbc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55827795daf0 con 0x558277a61430 2024-07-29T00:43:54.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa615bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5582779c1a90 con 0x558277a6bcf0 2024-07-29T00:43:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa611000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 0x558277a60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa611000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 0x558277a60ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45864/0 (socket says 172.21.15.175:45864) 2024-07-29T00:43:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa611000640 1 -- 172.21.15.175:0/3345778613 learned_addr learned my addr 172.21.15.175:0/3345778613 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa60be00640 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a6b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.467+0000 7fa60b400640 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a6bcf0 0x558277a6e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa611000640 1 -- 172.21.15.175:0/3345778613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a6bcf0 msgr2=0x558277a6e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa611000640 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a6bcf0 0x558277a6e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa611000640 1 -- 172.21.15.175:0/3345778613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 msgr2=0x558277a6b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa611000640 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a6b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa611000640 1 -- 172.21.15.175:0/3345778613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582777373c0 con 0x558277a60b10 2024-07-29T00:43:54.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa60b400640 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a6bcf0 0x558277a6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa608c00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5f00a2cf0 0x7fa5f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa611000640 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 0x558277a60ef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa604009be0 tx=0x7fa604030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a264e78a93bfd0c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa608c00640 1 -- 172.21.15.175:0/802518595 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fa5fc0cf800 con 0x7fa60c1038c0 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.468+0000 7fa60be00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5f00a2cf0 0x7fa5f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.469+0000 7fa601600640 1 -- 172.21.15.175:0/3345778613 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604041030 con 0x558277a60b10 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.469+0000 7fa601600640 1 -- 172.21.15.175:0/3345778613 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604003970 con 0x558277a60b10 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.469+0000 7fa601600640 1 -- 172.21.15.175:0/3345778613 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604036040 con 0x558277a60b10 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.469+0000 7fa615bbc740 1 -- 172.21.15.175:0/3345778613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 msgr2=0x558277a60ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.469+0000 7fa60be00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5f00a2cf0 0x7fa5f00a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0ccfb0 tx=0x7fa5fc0d5550 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:54.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.469+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 0x558277a60ef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa604009be0 tx=0x7fa604030de0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 -- 172.21.15.175:0/3345778613 shutdown_connections 2024-07-29T00:43:54.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a6bcf0 0x558277a6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a6b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3345778613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a60b10 0x558277a60ef0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 -- 172.21.15.175:0/3345778613 >> 172.21.15.175:0/3345778613 conn(0x558277a5d0a0 msgr2=0x558277a5d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 -- 172.21.15.175:0/3345778613 shutdown_connections 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 -- 172.21.15.175:0/3345778613 wait complete. 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.470+0000 7fa615bbc740 1 -- start start 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 Processor -- start 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a60b10 0x558277a68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a69600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558277a6e910 con 0x558277a6bcf0 2024-07-29T00:43:54.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558277980bc0 con 0x558277a61430 2024-07-29T00:43:54.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558277a6ea90 con 0x558277a60b10 2024-07-29T00:43:54.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa615bbc740 10 client.0 _ll_register_callbacks cb 0x55827796cd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:54.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.471+0000 7fa611000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a60b10 0x558277a68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa611000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a60b10 0x558277a68b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:56108/0 (socket says 172.21.15.175:56108) 2024-07-29T00:43:54.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa611000640 1 -- 172.21.15.175:0/3821299473 learned_addr learned my addr 172.21.15.175:0/3821299473 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:54.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa611000640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 -- 172.21.15.175:0/3821299473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a60b10 msgr2=0x558277a68b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a60b10 0x558277a68b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 -- 172.21.15.175:0/3821299473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 msgr2=0x558277a69600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a69600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 -- 172.21.15.175:0/3821299473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558277a77db0 con 0x558277a61430 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa611000640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a69600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa60b400640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a690c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa60c101bb0 tx=0x7fa5f400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9800264d03a6e0b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.472+0000 7fa5fa000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa5fa000640 1 -- 172.21.15.175:0/3821299473 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f401e060 con 0x558277a61430 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa604003970 con 0x558277a61430 2024-07-29T00:43:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa5fa000640 1 -- 172.21.15.175:0/3821299473 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fa5f4004020 con 0x558277a61430 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa5fa000640 1 -- 172.21.15.175:0/3821299473 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa5f4004260 con 0x558277a61430 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa615bbc740 10 client.26864 fetch_fsmap learned FSMap version 42 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558277a7a780 con 0x558277a61430 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.473+0000 7fa5fa000640 1 -- 172.21.15.175:0/3821299473 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604003970 con 0x558277a61430 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa5fa000640 1 -- 172.21.15.175:0/3821299473 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7fa5f4019070 con 0x558277a61430 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa615bbc740 10 client.26864 fetch_fsmap finished waiting for FSMap version 42 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa615bbc740 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:54.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa615bbc740 20 client.26864 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277a7bce0 0x558277a7e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa615bbc740 4 client.26864 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582779d3040 con 0x558277a7bce0 2024-07-29T00:43:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.474+0000 7fa611000640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277a7bce0 0x558277a7e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.475+0000 7fa611000640 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277a7bce0 0x558277a7e0d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6d21eb43e8c871 server_cookie=8c4892252fc58922 in_seq=0 out_seq=0 2024-07-29T00:43:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.475+0000 7fa5fa000640 10 client.26864 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.475+0000 7fa5fa000640 1 -- 172.21.15.175:0/3821299473 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582779d3040 con 0x558277a7bce0 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.475+0000 7fa5fa000640 10 client.26864 handle_command_reply: tid=0 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.475+0000 7fa615bbc740 1 client.26864 shutdown 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 msgr2=0x558277a690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a690c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa60c101bb0 tx=0x7fa5f400f790 comp rx=0 tx=0).stop 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 shutdown_connections 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277a7bce0 0x558277a7e0d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.483 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a69600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a61430 0x558277a690c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa5fa000640 0 client.26864 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 --2- 172.21.15.175:0/3821299473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a60b10 0x558277a68b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 >> 172.21.15.175:0/3821299473 conn(0x558277a5d0a0 msgr2=0x558277a62dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 shutdown_connections 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 1 -- 172.21.15.175:0/3821299473 wait complete. 2024-07-29T00:43:54.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.476+0000 7fa615bbc740 20 client.26864 trim_cache size 0 max 16384 2024-07-29T00:43:54.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 Processor -- start 2024-07-29T00:43:54.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 -- start start 2024-07-29T00:43:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a83f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1af770 con 0x558277a6bcf0 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b12e0 con 0x558277a60b10 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.495+0000 7fa615bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1afe40 con 0x558277a61430 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a83f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a69450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39912/0 (socket says 172.21.15.175:39912) 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 -- 172.21.15.175:0/2054038948 learned_addr learned my addr 172.21.15.175:0/2054038948 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:54.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa60b400640 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 -- 172.21.15.175:0/2054038948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 msgr2=0x558277a839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 -- 172.21.15.175:0/2054038948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 msgr2=0x558277a83f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a83f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 -- 172.21.15.175:0/2054038948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558277a7e440 con 0x558277a60b10 2024-07-29T00:43:54.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa60b400640 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.496+0000 7fa611000640 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a69450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x558277a67380 tx=0x7fa604033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783e1a658977556c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.497+0000 7fa600c00640 1 -- 172.21.15.175:0/2054038948 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604036040 con 0x558277a60b10 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.497+0000 7fa600c00640 1 -- 172.21.15.175:0/2054038948 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604003970 con 0x558277a60b10 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.497+0000 7fa600c00640 1 -- 172.21.15.175:0/2054038948 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604041040 con 0x558277a60b10 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 -- 172.21.15.175:0/2054038948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 msgr2=0x558277a69450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a69450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x558277a67380 tx=0x7fa604033040 comp rx=0 tx=0).stop 2024-07-29T00:43:54.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 -- 172.21.15.175:0/2054038948 shutdown_connections 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a83f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 --2- 172.21.15.175:0/2054038948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a69450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 -- 172.21.15.175:0/2054038948 >> 172.21.15.175:0/2054038948 conn(0x558277a5d0a0 msgr2=0x558277a67c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 -- 172.21.15.175:0/2054038948 shutdown_connections 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 -- 172.21.15.175:0/2054038948 wait complete. 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.498+0000 7fa615bbc740 1 -- start start 2024-07-29T00:43:54.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 Processor -- start 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a64f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b0be0 con 0x558277a6bcf0 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b0510 con 0x558277a60b10 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1ad770 con 0x558277a61430 2024-07-29T00:43:54.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa615bbc740 10 client.0 _ll_register_callbacks cb 0x7fa5f4011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa611000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a64f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa60b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa611000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a64f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45904/0 (socket says 172.21.15.175:45904) 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.499+0000 7fa611000640 1 -- 172.21.15.175:0/1871424084 learned_addr learned my addr 172.21.15.175:0/1871424084 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 -- 172.21.15.175:0/1871424084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 msgr2=0x558277a74220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a74220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 -- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 msgr2=0x558277a73ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a73ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 -- 172.21.15.175:0/1871424084 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558277a654d0 con 0x558277a6bcf0 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa60b400640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa611000640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a64f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa60402ac00 tx=0x7fa6040041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1802d3ce6360de5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa5fa000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa5fa000640 1 -- 172.21.15.175:0/1871424084 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604036070 con 0x558277a6bcf0 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa5fa000640 1 -- 172.21.15.175:0/1871424084 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7fa6040303a0 con 0x558277a6bcf0 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.500+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa604003970 con 0x558277a6bcf0 2024-07-29T00:43:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa5fa000640 1 -- 172.21.15.175:0/1871424084 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa604041040 con 0x558277a6bcf0 2024-07-29T00:43:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 10 client.16722 fetch_fsmap learned FSMap version 42 2024-07-29T00:43:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558277ae5ff0 con 0x558277a6bcf0 2024-07-29T00:43:54.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa5fa000640 1 -- 172.21.15.175:0/1871424084 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604003970 con 0x558277a6bcf0 2024-07-29T00:43:54.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa5fa000640 1 -- 172.21.15.175:0/1871424084 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7fa60403d740 con 0x558277a6bcf0 2024-07-29T00:43:54.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 10 client.16722 fetch_fsmap finished waiting for FSMap version 42 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 20 client.16722 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277ae6270 0x558277ae8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 4 client.16722 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.501+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582779cd550 con 0x558277ae6270 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.502+0000 7fa60b400640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277ae6270 0x558277ae8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.502+0000 7fa60b400640 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277ae6270 0x558277ae8660 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbd30b7fc9e9f65 server_cookie=aa5523dace0918 in_seq=0 out_seq=0 2024-07-29T00:43:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.502+0000 7fa5fa000640 10 client.16722 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.502+0000 7fa5fa000640 1 -- 172.21.15.175:0/1871424084 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582779cd550 con 0x558277ae6270 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.502+0000 7fa5fa000640 10 client.16722 handle_command_reply: tid=0 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 client.16722 shutdown 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 msgr2=0x558277a64f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a64f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa60402ac00 tx=0x7fa6040041d0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 shutdown_connections 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558277ae6270 0x558277ae8660 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558277a6bcf0 0x558277a64f90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558277a61430 0x558277a74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 --2- 172.21.15.175:0/1871424084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558277a60b10 0x558277a73ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 >> 172.21.15.175:0/1871424084 conn(0x558277a5d0a0 msgr2=0x558277a5b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa5fa000640 0 client.16722 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 shutdown_connections 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 1 -- 172.21.15.175:0/1871424084 wait complete. 2024-07-29T00:43:54.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.503+0000 7fa615bbc740 20 client.16722 trim_cache size 0 max 16384 2024-07-29T00:43:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:43:54.512 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:43:54.512 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:43:54.512 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5f00a2cf0 msgr2=0x7fa5f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5f00a2cf0 0x7fa5f00a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0ccfb0 tx=0x7fa5fc0d5550 comp rx=0 tx=0).stop 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 msgr2=0x7fa60c19a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c19a190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e9f0 tx=0x7fa5fc00edf0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 shutdown_connections 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60c1098c0 0x7fa60c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa60c104260 0x7fa60c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa5f00a2cf0 0x7fa5f00a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 --2- 172.21.15.175:0/802518595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa60c1038c0 0x7fa60c19a190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:54.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 >> 172.21.15.175:0/802518595 conn(0x7fa60c0ff5c0 msgr2=0x7fa60c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:54.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 shutdown_connections 2024-07-29T00:43:54.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:54.504+0000 7fa611a00640 1 -- 172.21.15.175:0/802518595 wait complete. 2024-07-29T00:43:55.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:43:55.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:43:55.886 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:55 smithi181 ceph-mon[36706]: pgmap v227: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-07-29T00:43:56.278 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:55 smithi175 ceph-mon[32138]: pgmap v227: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-07-29T00:43:56.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:55 smithi204 ceph-mon[37368]: pgmap v227: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-07-29T00:43:56.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.589+0000 7ff092a00640 1 -- 172.21.15.175:0/3682252860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c076d30 msgr2=0x7ff08c1121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:56.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.589+0000 7ff092a00640 1 --2- 172.21.15.175:0/3682252860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c076d30 0x7ff08c1121e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff074009900 tx=0x7ff07402a400 comp rx=0 tx=0).stop 2024-07-29T00:43:56.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 -- 172.21.15.175:0/3682252860 shutdown_connections 2024-07-29T00:43:56.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 --2- 172.21.15.175:0/3682252860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c112720 0x7ff08c114b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 --2- 172.21.15.175:0/3682252860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c076d30 0x7ff08c1121e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 --2- 172.21.15.175:0/3682252860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076410 0x7ff08c0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 -- 172.21.15.175:0/3682252860 >> 172.21.15.175:0/3682252860 conn(0x7ff08c0ff760 msgr2=0x7ff08c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:56.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 -- 172.21.15.175:0/3682252860 shutdown_connections 2024-07-29T00:43:56.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.590+0000 7ff092a00640 1 -- 172.21.15.175:0/3682252860 wait complete. 2024-07-29T00:43:56.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.591+0000 7ff092a00640 1 Processor -- start 2024-07-29T00:43:56.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.591+0000 7ff092a00640 1 -- start start 2024-07-29T00:43:56.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff092a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 0x7ff08c1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:56.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff092a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076d30 0x7ff08c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:56.593 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff092a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c112720 0x7ff08c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:56.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff092a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c118290 con 0x7ff08c076d30 2024-07-29T00:43:56.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff092a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c118110 con 0x7ff08c112720 2024-07-29T00:43:56.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff092a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c118410 con 0x7ff08c076410 2024-07-29T00:43:56.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff08b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076d30 0x7ff08c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:56.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff08be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 0x7ff08c1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:56.594 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff090a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c112720 0x7ff08c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff08be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 0x7ff08c1a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:56146/0 (socket says 172.21.15.175:56146) 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff08b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076d30 0x7ff08c1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45930/0 (socket says 172.21.15.175:45930) 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.592+0000 7ff08be00640 1 -- 172.21.15.175:0/3567960952 learned_addr learned my addr 172.21.15.175:0/3567960952 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff08be00640 1 -- 172.21.15.175:0/3567960952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c112720 msgr2=0x7ff08c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff08be00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c112720 0x7ff08c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff08be00640 1 -- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076d30 msgr2=0x7ff08c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff08be00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076d30 0x7ff08c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff08be00640 1 -- 172.21.15.175:0/3567960952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c19ecf0 con 0x7ff08c076410 2024-07-29T00:43:56.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff090a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c112720 0x7ff08c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:56.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff08be00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 0x7ff08c1a4400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d950 tx=0x7ff07c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:56.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.593+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c017070 con 0x7ff08c076410 2024-07-29T00:43:56.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.594+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff07c009c30 con 0x7ff08c076410 2024-07-29T00:43:56.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.594+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00f6b0 con 0x7ff08c076410 2024-07-29T00:43:56.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.594+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c19efe0 con 0x7ff08c076410 2024-07-29T00:43:56.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.594+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1ab2d0 con 0x7ff08c076410 2024-07-29T00:43:56.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.595+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c0767f0 con 0x7ff08c076410 2024-07-29T00:43:56.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.597+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff07c0109d0 con 0x7ff08c076410 2024-07-29T00:43:56.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.598+0000 7ff088c00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff0700a2cf0 0x7ff0700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:56.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.599+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7ff07c0cf090 con 0x7ff08c076410 2024-07-29T00:43:56.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.599+0000 7ff08b400640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff0700a2cf0 0x7ff0700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:56.599 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.599+0000 7ff08b400640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff0700a2cf0 0x7ff0700a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff074009230 tx=0x7ff0740058d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:56.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.600+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff07c009de0 con 0x7ff08c076410 2024-07-29T00:43:56.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.834+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08c1ab500 con 0x7ff08c076410 2024-07-29T00:43:56.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.836+0000 7ff088c00640 1 -- 172.21.15.175:0/3567960952 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11995 (secure 0 0 0) 0x7ff07c09b5b0 con 0x7ff08c076410 2024-07-29T00:43:56.837 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:43:56.837 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":42,"btime":"2024-07-29T00:43:52:852359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:52.442654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24484,24499,24503,24515,24539]},"id":1}]} 2024-07-29T00:43:56.837 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 42 2024-07-29T00:43:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff0700a2cf0 msgr2=0x7ff0700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff0700a2cf0 0x7ff0700a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff074009230 tx=0x7ff0740058d0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 msgr2=0x7ff08c1a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:56.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 0x7ff08c1a4400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d950 tx=0x7ff07c00de10 comp rx=0 tx=0).stop 2024-07-29T00:43:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 shutdown_connections 2024-07-29T00:43:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff08c112720 0x7ff08c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff0700a2cf0 0x7ff0700a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff08c076d30 0x7ff08c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 --2- 172.21.15.175:0/3567960952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c076410 0x7ff08c1a4400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.839+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 >> 172.21.15.175:0/3567960952 conn(0x7ff08c0ff760 msgr2=0x7ff08c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.840+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 shutdown_connections 2024-07-29T00:43:56.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:56.840+0000 7ff092a00640 1 -- 172.21.15.175:0/3567960952 wait complete. 2024-07-29T00:43:57.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:56 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3567960952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:57.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:56 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3567960952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:57.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:56 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3567960952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:43:57.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:43:58.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:57 smithi175 ceph-mon[32138]: pgmap v228: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-07-29T00:43:58.150 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:57 smithi181 ceph-mon[36706]: pgmap v228: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-07-29T00:43:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:43:58.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:57 smithi204 ceph-mon[37368]: pgmap v228: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-07-29T00:43:58.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 -- 172.21.15.175:0/2835648289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 msgr2=0x7f32f4114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2835648289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f4114b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009cd0 tx=0x7f32ec02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 -- 172.21.15.175:0/2835648289 shutdown_connections 2024-07-29T00:43:58.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2835648289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f4114b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2835648289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f4107d10 0x7f32f4108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2835648289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f410dd50 0x7f32f410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 -- 172.21.15.175:0/2835648289 >> 172.21.15.175:0/2835648289 conn(0x7f32f4077670 msgr2=0x7f32f4079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 -- 172.21.15.175:0/2835648289 shutdown_connections 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.927+0000 7f32f9a00640 1 -- 172.21.15.175:0/2835648289 wait complete. 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.928+0000 7f32f9a00640 1 Processor -- start 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.928+0000 7f32f9a00640 1 -- start start 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.928+0000 7f32f9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4107d10 0x7f32f419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.928+0000 7f32f9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f410dd50 0x7f32f41a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4118060 con 0x7f32f4108670 2024-07-29T00:43:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f9a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4117ee0 con 0x7f32f410dd50 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41181e0 con 0x7f32f4107d10 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37568/0 (socket says 172.21.15.175:37568) 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 -- 172.21.15.175:0/2140605511 learned_addr learned my addr 172.21.15.175:0/2140605511 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4107d10 0x7f32f419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f9000640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f410dd50 0x7f32f41a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 -- 172.21.15.175:0/2140605511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4107d10 msgr2=0x7f32f419e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4107d10 0x7f32f419e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 -- 172.21.15.175:0/2140605511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f410dd50 msgr2=0x7f32f41a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f410dd50 0x7f32f41a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.929+0000 7f32f3400640 1 -- 172.21.15.175:0/2140605511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f41a34d0 con 0x7f32f4108670 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.930+0000 7f32f3400640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f419ea60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.930+0000 7f32f0c00640 1 -- 172.21.15.175:0/2140605511 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32f4108670 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.930+0000 7f32f0c00640 1 -- 172.21.15.175:0/2140605511 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e0010910 con 0x7f32f4108670 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.930+0000 7f32f0c00640 1 -- 172.21.15.175:0/2140605511 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0018990 con 0x7f32f4108670 2024-07-29T00:43:58.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.930+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f41a37c0 con 0x7f32f4108670 2024-07-29T00:43:58.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.930+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41ab0a0 con 0x7f32f4108670 2024-07-29T00:43:58.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 Processor -- start 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 -- start start 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32f0c00640 1 -- 172.21.15.175:0/2140605511 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f32e0010ac0 con 0x7f32f4108670 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8df0e0 0x56176e8dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56176e843ab0 con 0x56176e8e8910 2024-07-29T00:43:58.937 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56176e7dba80 con 0x56176e8df0e0 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.934+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56176e83fa70 con 0x56176e8ddb70 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8df0e0 0x56176e8dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37584/0 (socket says 172.21.15.175:37584) 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f3400640 1 -- 172.21.15.175:0/2505385534 learned_addr learned my addr 172.21.15.175:0/2505385534 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f9000640 1 -- 172.21.15.175:0/2505385534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 msgr2=0x56176e8ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f9000640 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f9000640 1 -- 172.21.15.175:0/2505385534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 msgr2=0x56176e8ead00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f9000640 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8ead00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.935+0000 7f32f9000640 1 -- 172.21.15.175:0/2505385534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176e5b53c0 con 0x56176e8df0e0 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32f9000640 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8df0e0 0x56176e8dd630 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eead0bf18b25cf44 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32e9600640 1 -- 172.21.15.175:0/2505385534 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec048070 con 0x56176e8df0e0 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32e9600640 1 -- 172.21.15.175:0/2505385534 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec036070 con 0x56176e8df0e0 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32e9600640 1 -- 172.21.15.175:0/2505385534 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004440 con 0x56176e8df0e0 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32f0c00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f32d80a2d70 0x7f32d80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32f0c00640 1 -- 172.21.15.175:0/2140605511 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f32e00cf7b0 con 0x7f32f4108670 2024-07-29T00:43:58.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.936+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f32d80a2d70 0x7f32d80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2505385534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8df0e0 msgr2=0x56176e8dd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8df0e0 0x56176e8dd630 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec009a20 comp rx=0 tx=0).stop 2024-07-29T00:43:58.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2505385534 shutdown_connections 2024-07-29T00:43:58.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8ead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2505385534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8df0e0 0x56176e8dd630 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2505385534 >> 172.21.15.175:0/2505385534 conn(0x56176e8a6950 msgr2=0x56176e8da200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2505385534 shutdown_connections 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2505385534 wait complete. 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.937+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f32d80a2d70 0x7f32d80a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x56176e8db060 tx=0x7f32e400b390 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 -- start start 2024-07-29T00:43:58.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 Processor -- start 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 0x56176e8e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56176e8eb650 con 0x56176e8e8910 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56176e7fec10 con 0x56176e8ddb70 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56176e8eb7d0 con 0x56176e8df0e0 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.938+0000 7f32fdaa9740 10 client.0 _ll_register_callbacks cb 0x56176e7c7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 0x56176e8e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 0x56176e8e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55088/0 (socket says 172.21.15.175:55088) 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f3e00640 1 -- 172.21.15.175:0/2377722772 learned_addr learned my addr 172.21.15.175:0/2377722772 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f3400640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 -- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 msgr2=0x56176e8e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 -- 172.21.15.175:0/2377722772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 msgr2=0x56176e8e58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 0x56176e8e58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 -- 172.21.15.175:0/2377722772 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176e8021e0 con 0x56176e8e8910 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 0x56176e8e58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32f9000640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8e6350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02ae00 tx=0x7f32ec031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f517e64f13c6cd60 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.939+0000 7f32de000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32de000640 1 -- 172.21.15.175:0/2377722772 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec048040 con 0x56176e8e8910 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32de000640 1 -- 172.21.15.175:0/2377722772 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f32ec031530 con 0x56176e8e8910 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec036070 con 0x56176e8e8910 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32de000640 1 -- 172.21.15.175:0/2377722772 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f32ec003c50 con 0x56176e8e8910 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32fdaa9740 10 client.16740 fetch_fsmap learned FSMap version 43 2024-07-29T00:43:58.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56176e8eb650 con 0x56176e8e8910 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.940+0000 7f32de000640 1 -- 172.21.15.175:0/2377722772 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec036070 con 0x56176e8e8910 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32de000640 1 -- 172.21.15.175:0/2377722772 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f32ec04b030 con 0x56176e8e8910 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32fdaa9740 10 client.16740 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32fdaa9740 10 client.16740 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32fdaa9740 20 client.16740 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e8f8570 0x56176e8fa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32fdaa9740 4 client.16740 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176e851020 con 0x56176e8f8570 2024-07-29T00:43:58.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.941+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e8f8570 0x56176e8fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.942+0000 7f32f3e00640 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e8f8570 0x56176e8fa960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81069d5eae7328b5 server_cookie=54678564df309b5f in_seq=0 out_seq=0 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.942+0000 7f32de000640 10 client.16740 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32de000640 1 -- 172.21.15.175:0/2377722772 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56176e851020 con 0x56176e8f8570 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32de000640 10 client.16740 handle_command_reply: tid=0 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 client.16740 shutdown 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 msgr2=0x56176e8e6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8e6350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02ae00 tx=0x7f32ec031e40 comp rx=0 tx=0).stop 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 shutdown_connections 2024-07-29T00:43:58.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8e8910 0x56176e8e6350 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e8f8570 0x56176e8fa960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/2377722772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8ddb70 0x56176e8e58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32de000640 0 client.16740 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 >> 172.21.15.175:0/2377722772 conn(0x56176e8a6950 msgr2=0x56176e8dff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 shutdown_connections 2024-07-29T00:43:58.950 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.943+0000 7f32fdaa9740 1 -- 172.21.15.175:0/2377722772 wait complete. 2024-07-29T00:43:58.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.944+0000 7f32fdaa9740 20 client.16740 trim_cache size 0 max 16384 2024-07-29T00:43:58.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 Processor -- start 2024-07-29T00:43:58.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 -- start start 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8d9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8df0e0 0x56176e9000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e900610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b3b40 con 0x56176e8df0e0 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b5680 con 0x56176e8e8910 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.958+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b4210 con 0x56176e8ddb70 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8d9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8d9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57824/0 (socket says 172.21.15.175:57824) 2024-07-29T00:43:58.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e900610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 -- 172.21.15.175:0/1393830759 learned_addr learned my addr 172.21.15.175:0/1393830759 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3400640 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8df0e0 0x56176e9000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 -- 172.21.15.175:0/1393830759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 msgr2=0x56176e900610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e900610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 -- 172.21.15.175:0/1393830759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8df0e0 msgr2=0x56176e9000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8df0e0 0x56176e9000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 -- 172.21.15.175:0/1393830759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176e8e4e30 con 0x56176e8ddb70 2024-07-29T00:43:58.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.959+0000 7f32f3e00640 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8d9250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56176e8fb070 tx=0x7f32e4019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc59eadc152fd9b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:58.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.960+0000 7f32e8c00640 1 -- 172.21.15.175:0/1393830759 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e401e9f0 con 0x56176e8ddb70 2024-07-29T00:43:58.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.960+0000 7f32e8c00640 1 -- 172.21.15.175:0/1393830759 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e4005500 con 0x56176e8ddb70 2024-07-29T00:43:58.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.960+0000 7f32e8c00640 1 -- 172.21.15.175:0/1393830759 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4004e00 con 0x56176e8ddb70 2024-07-29T00:43:58.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 -- 172.21.15.175:0/1393830759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 msgr2=0x56176e8d9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8d9250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56176e8fb070 tx=0x7f32e4019000 comp rx=0 tx=0).stop 2024-07-29T00:43:58.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 -- 172.21.15.175:0/1393830759 shutdown_connections 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e900610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8df0e0 0x56176e9000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/1393830759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8ddb70 0x56176e8d9250 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 -- 172.21.15.175:0/1393830759 >> 172.21.15.175:0/1393830759 conn(0x56176e8a6950 msgr2=0x56176e8d8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 -- 172.21.15.175:0/1393830759 shutdown_connections 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.961+0000 7f32fdaa9740 1 -- 172.21.15.175:0/1393830759 wait complete. 2024-07-29T00:43:58.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 -- start start 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 Processor -- start 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8ddb70 0x56176e8f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e8ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b4fb0 con 0x56176e8ddb70 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b48e0 con 0x56176e8e8910 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.962+0000 7f32fdaa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b1b40 con 0x56176e8df0e0 2024-07-29T00:43:58.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32fdaa9740 10 client.0 _ll_register_callbacks cb 0x7f32ec046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8ddb70 0x56176e8f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8ddb70 0x56176e8f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37616/0 (socket says 172.21.15.175:37616) 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e8ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f3e00640 1 -- 172.21.15.175:0/3018208550 learned_addr learned my addr 172.21.15.175:0/3018208550 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f3400640 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f9000640 1 -- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 msgr2=0x56176e8ec2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f9000640 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8ec2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f9000640 1 -- 172.21.15.175:0/3018208550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8ddb70 msgr2=0x56176e8f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:43:58.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f9000640 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8ddb70 0x56176e8f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.963+0000 7f32f9000640 1 -- 172.21.15.175:0/3018208550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176e8ecda0 con 0x56176e8e8910 2024-07-29T00:43:58.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32f9000640 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e8ec7e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f32ec031eb0 tx=0x7f32ec030e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743e003a540bbfe4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:43:58.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32de000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:43:58.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32de000640 1 -- 172.21.15.175:0/3018208550 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec048040 con 0x56176e8e8910 2024-07-29T00:43:58.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32de000640 1 -- 172.21.15.175:0/3018208550 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f32ec031430 con 0x56176e8e8910 2024-07-29T00:43:58.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e4005500 con 0x56176e8e8910 2024-07-29T00:43:58.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32de000640 1 -- 172.21.15.175:0/3018208550 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f32ec031650 con 0x56176e8e8910 2024-07-29T00:43:58.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32fdaa9740 10 client.26894 fetch_fsmap learned FSMap version 43 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.964+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56176e962e70 con 0x56176e8e8910 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32de000640 1 -- 172.21.15.175:0/3018208550 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e4005500 con 0x56176e8e8910 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32de000640 1 -- 172.21.15.175:0/3018208550 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f32ec04b040 con 0x56176e8e8910 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32fdaa9740 10 client.26894 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32fdaa9740 10 client.26894 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32fdaa9740 20 client.26894 populate_metadata read hostname 'smithi175' 2024-07-29T00:43:58.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e963580 0x56176e965970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32fdaa9740 4 client.26894 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56176e84b530 con 0x56176e963580 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.965+0000 7f32f3e00640 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e963580 0x56176e965970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32f3e00640 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e963580 0x56176e965970 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c247b6a2db334a3e server_cookie=d5c90a50ac67da32 in_seq=0 out_seq=0 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32de000640 10 client.26894 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32de000640 1 -- 172.21.15.175:0/3018208550 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56176e84b530 con 0x56176e963580 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32de000640 10 client.26894 handle_command_reply: tid=0 2024-07-29T00:43:58.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 client.26894 shutdown 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 msgr2=0x56176e8ec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e8ec7e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f32ec031eb0 tx=0x7f32ec030e70 comp rx=0 tx=0).stop 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 shutdown_connections 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x56176e963580 0x56176e965970 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56176e8e8910 0x56176e8ec7e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56176e8df0e0 0x56176e8ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 --2- 172.21.15.175:0/3018208550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56176e8ddb70 0x56176e8f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 >> 172.21.15.175:0/3018208550 conn(0x56176e8a6950 msgr2=0x56176e8e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.966+0000 7f32de000640 0 client.26894 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:43:58.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.967+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 shutdown_connections 2024-07-29T00:43:58.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.967+0000 7f32fdaa9740 1 -- 172.21.15.175:0/3018208550 wait complete. 2024-07-29T00:43:58.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.967+0000 7f32fdaa9740 20 client.26894 trim_cache size 0 max 16384 2024-07-29T00:43:58.974 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f32d80a2d70 msgr2=0x7f32d80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f32d80a2d70 0x7f32d80a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x56176e8db060 tx=0x7f32e400b390 comp rx=0 tx=0).stop 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 msgr2=0x7f32f419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f419ea60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 shutdown_connections 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32f410dd50 0x7f32f41a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f32d80a2d70 0x7f32d80a5230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f4108670 0x7f32f419ea60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 --2- 172.21.15.175:0/2140605511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4107d10 0x7f32f419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:43:58.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 >> 172.21.15.175:0/2140605511 conn(0x7f32f4077670 msgr2=0x7f32f4078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:43:58.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 shutdown_connections 2024-07-29T00:43:58.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:43:58.968+0000 7f32f9a00640 1 -- 172.21.15.175:0/2140605511 wait complete. 2024-07-29T00:43:59.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:58 smithi181 ceph-mon[36706]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:59.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:58 smithi175 ceph-mon[32138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:59.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:58 smithi204 ceph-mon[37368]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-29T00:43:59.880 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:44:00.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:43:59 smithi181 ceph-mon[36706]: pgmap v229: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-29T00:44:00.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:43:59 smithi175 ceph-mon[32138]: pgmap v229: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-29T00:44:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:43:59 smithi204 ceph-mon[37368]: pgmap v229: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-29T00:44:00.341 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:01.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 -- 172.21.15.175:0/1275225650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc104420 msgr2=0x7f7ddc110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1275225650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc104420 0x7f7ddc110950 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009be0 tx=0x7f7dd402a6c0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 -- 172.21.15.175:0/1275225650 shutdown_connections 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1275225650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc104420 0x7f7ddc110950 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1275225650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc103a80 0x7f7ddc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1275225650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc109a80 0x7f7ddc109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 -- 172.21.15.175:0/1275225650 >> 172.21.15.175:0/1275225650 conn(0x7f7ddc0ff780 msgr2=0x7f7ddc101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 -- 172.21.15.175:0/1275225650 shutdown_connections 2024-07-29T00:44:01.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.023+0000 7f7de3c00640 1 -- 172.21.15.175:0/1275225650 wait complete. 2024-07-29T00:44:01.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.024+0000 7f7de3c00640 1 Processor -- start 2024-07-29T00:44:01.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.024+0000 7f7de3c00640 1 -- start start 2024-07-29T00:44:01.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.024+0000 7f7de3c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc103a80 0x7f7ddc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de3c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 0x7f7ddc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de3c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc109a80 0x7f7ddc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de3c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc113e30 con 0x7f7ddc104420 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de3c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc113cb0 con 0x7f7ddc109a80 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de3c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc113fb0 con 0x7f7ddc103a80 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 0x7f7ddc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc103a80 0x7f7ddc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc109a80 0x7f7ddc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 0x7f7ddc19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37636/0 (socket says 172.21.15.175:37636) 2024-07-29T00:44:01.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1253201562 learned_addr learned my addr 172.21.15.175:0/1253201562 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7de1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc103a80 0x7f7ddc19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57860/0 (socket says 172.21.15.175:57860) 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1253201562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc103a80 msgr2=0x7f7ddc19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc103a80 0x7f7ddc19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1253201562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc109a80 msgr2=0x7f7ddc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc109a80 0x7f7ddc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.025+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1253201562 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc19f260 con 0x7f7ddc104420 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.026+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 0x7f7ddc19a7f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e9c0 tx=0x7f7dcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.026+0000 7f7dd9600640 1 -- 172.21.15.175:0/1253201562 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00cde0 con 0x7f7ddc104420 2024-07-29T00:44:01.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.026+0000 7f7dd9600640 1 -- 172.21.15.175:0/1253201562 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dcc0035f0 con 0x7f7ddc104420 2024-07-29T00:44:01.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.026+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc19f550 con 0x7f7ddc104420 2024-07-29T00:44:01.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.026+0000 7f7dd9600640 1 -- 172.21.15.175:0/1253201562 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc010690 con 0x7f7ddc104420 2024-07-29T00:44:01.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.027+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc1a6e30 con 0x7f7ddc104420 2024-07-29T00:44:01.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.029+0000 7f7dd9600640 1 -- 172.21.15.175:0/1253201562 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7dcc0037a0 con 0x7f7ddc104420 2024-07-29T00:44:01.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.030+0000 7f7de65e4740 1 Processor -- start 2024-07-29T00:44:01.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.030+0000 7f7de65e4740 1 -- start start 2024-07-29T00:44:01.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x56235941f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x56235941fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x56235942cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de65e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562359385ab0 con 0x56235942a720 2024-07-29T00:44:01.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de65e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56235931da80 con 0x56235941f980 2024-07-29T00:44:01.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de65e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562359381a70 con 0x562359420ef0 2024-07-29T00:44:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7dd9600640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7dc40a2d70 0x7f7dc40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x56235941fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x56235941fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55128/0 (socket says 172.21.15.175:55128) 2024-07-29T00:44:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de1200640 1 -- 172.21.15.175:0/4242900101 learned_addr learned my addr 172.21.15.175:0/4242900101 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x56235942cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7de1c00640 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x56235941f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.031+0000 7f7dd9600640 1 -- 172.21.15.175:0/1253201562 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f7dcc0cf6e0 con 0x7f7ddc104420 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1200640 1 -- 172.21.15.175:0/4242900101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 msgr2=0x56235941f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1200640 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x56235941f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1200640 1 -- 172.21.15.175:0/4242900101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 msgr2=0x56235942cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1200640 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x56235942cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1200640 1 -- 172.21.15.175:0/4242900101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623590f73c0 con 0x56235941f980 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1c00640 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x56235941f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7dc40a2d70 0x7f7dc40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7de1200640 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x56235941fdc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009ca0 tx=0x7f7dd0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47099bcd7382815e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7dca000640 1 -- 172.21.15.175:0/4242900101 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd001e070 con 0x56235941f980 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7dca000640 1 -- 172.21.15.175:0/4242900101 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd0019b10 con 0x56235941f980 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.032+0000 7f7dca000640 1 -- 172.21.15.175:0/4242900101 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000eae0 con 0x56235941f980 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 -- 172.21.15.175:0/4242900101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 msgr2=0x56235941fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de1c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7dc40a2d70 0x7f7dc40a5230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009370 tx=0x7f7dd40098e0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x56235941fdc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009ca0 tx=0x7f7dd0011810 comp rx=0 tx=0).stop 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 -- 172.21.15.175:0/4242900101 shutdown_connections 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x56235942cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x56235941fdc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4242900101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x56235941f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 -- 172.21.15.175:0/4242900101 >> 172.21.15.175:0/4242900101 conn(0x5623593e8950 msgr2=0x56235941c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 -- 172.21.15.175:0/4242900101 shutdown_connections 2024-07-29T00:44:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.033+0000 7f7de65e4740 1 -- 172.21.15.175:0/4242900101 wait complete. 2024-07-29T00:44:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.034+0000 7f7de65e4740 1 -- start start 2024-07-29T00:44:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.034+0000 7f7de65e4740 1 Processor -- start 2024-07-29T00:44:01.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x562359427730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359427c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235942a720 0x5623594281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56235942d4c0 con 0x562359420ef0 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562359340c10 con 0x56235941f980 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56235942d640 con 0x56235942a720 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de65e4740 10 client.0 _ll_register_callbacks cb 0x562359309a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7ddbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359427c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7ddbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359427c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37654/0 (socket says 172.21.15.175:37654) 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1793197670 learned_addr learned my addr 172.21.15.175:0/1793197670 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de1c00640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235942a720 0x5623594281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.035+0000 7f7de1200640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x562359427730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235942a720 msgr2=0x5623594281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235942a720 0x5623594281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1793197670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 msgr2=0x562359427730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x562359427730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7ddbe00640 1 -- 172.21.15.175:0/1793197670 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623593441e0 con 0x562359420ef0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7de1200640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x562359427730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359427c70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc09bb40 tx=0x7f7dcc009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b199ae90fc6986 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7dbea00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7dbea00640 1 -- 172.21.15.175:0/1793197670 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc09ff60 con 0x562359420ef0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7dbea00640 1 -- 172.21.15.175:0/1793197670 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f7dcc0cfb90 con 0x562359420ef0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.036+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd0019b10 con 0x562359420ef0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7dbea00640 1 -- 172.21.15.175:0/1793197670 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7dcc0a4820 con 0x562359420ef0 2024-07-29T00:44:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 10 client.16758 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5623594392d0 con 0x562359420ef0 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7dbea00640 1 -- 172.21.15.175:0/1793197670 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd0019b10 con 0x562359420ef0 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7dbea00640 1 -- 172.21.15.175:0/1793197670 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f7dcc097900 con 0x562359420ef0 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 10 client.16758 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 10 client.16758 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 20 client.16758 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56235943a790 0x56235943cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 4 client.16758 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.037+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562359393020 con 0x56235943a790 2024-07-29T00:44:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.038+0000 7f7de1200640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56235943a790 0x56235943cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.038+0000 7f7de1200640 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56235943a790 0x56235943cc00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=169b550f856dddd2 server_cookie=8a6f91671bace88 in_seq=0 out_seq=0 2024-07-29T00:44:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.038+0000 7f7dbea00640 10 client.16758 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7dbea00640 1 -- 172.21.15.175:0/1793197670 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562359393020 con 0x56235943a790 2024-07-29T00:44:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7dbea00640 10 client.16758 handle_command_reply: tid=0 2024-07-29T00:44:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 client.16758 shutdown 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 msgr2=0x562359427c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359427c70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc09bb40 tx=0x7f7dcc009600 comp rx=0 tx=0).stop 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 shutdown_connections 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56235943a790 0x56235943cc00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235942a720 0x5623594281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359427c70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 --2- 172.21.15.175:0/1793197670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x562359427730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 >> 172.21.15.175:0/1793197670 conn(0x5623593e8950 msgr2=0x562359421d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.039+0000 7f7dbea00640 0 client.16758 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.040+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 shutdown_connections 2024-07-29T00:44:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.040+0000 7f7de65e4740 1 -- 172.21.15.175:0/1793197670 wait complete. 2024-07-29T00:44:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.040+0000 7f7de65e4740 20 client.16758 trim_cache size 0 max 16384 2024-07-29T00:44:01.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 Processor -- start 2024-07-29T00:44:01.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 -- start start 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x5623594296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x5623594424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x562359442a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1af7d0 con 0x56235942a720 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1b1310 con 0x56235941f980 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.054+0000 7f7de65e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1afea0 con 0x562359420ef0 2024-07-29T00:44:01.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x5623594296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x5623594296a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55142/0 (socket says 172.21.15.175:55142) 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 -- 172.21.15.175:0/4102758103 learned_addr learned my addr 172.21.15.175:0/4102758103 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1c00640 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x562359442a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x5623594424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 -- 172.21.15.175:0/4102758103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 msgr2=0x5623594424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x5623594424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 -- 172.21.15.175:0/4102758103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 msgr2=0x562359442a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x562359442a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 -- 172.21.15.175:0/4102758103 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623594224d0 con 0x56235941f980 2024-07-29T00:44:01.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.055+0000 7f7de1200640 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x5623594296a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00058c0 tx=0x7f7dd00093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731d4503c0c8bd7b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:01.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.056+0000 7f7dc9600640 1 -- 172.21.15.175:0/4102758103 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd001e040 con 0x56235941f980 2024-07-29T00:44:01.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.056+0000 7f7dc9600640 1 -- 172.21.15.175:0/4102758103 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd001f040 con 0x56235941f980 2024-07-29T00:44:01.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.056+0000 7f7dc9600640 1 -- 172.21.15.175:0/4102758103 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0004880 con 0x56235941f980 2024-07-29T00:44:01.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 -- 172.21.15.175:0/4102758103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 msgr2=0x5623594296a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x5623594296a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00058c0 tx=0x7f7dd00093b0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 -- 172.21.15.175:0/4102758103 shutdown_connections 2024-07-29T00:44:01.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56235942a720 0x562359442a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562359420ef0 0x5623594424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 --2- 172.21.15.175:0/4102758103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235941f980 0x5623594296a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 -- 172.21.15.175:0/4102758103 >> 172.21.15.175:0/4102758103 conn(0x5623593e8950 msgr2=0x562359428f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 -- 172.21.15.175:0/4102758103 shutdown_connections 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 -- 172.21.15.175:0/4102758103 wait complete. 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 -- start start 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.057+0000 7f7de65e4740 1 Processor -- start 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235941f980 0x5623594328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359432e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235942a720 0x562359423d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1b0c40 con 0x562359420ef0 2024-07-29T00:44:01.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1b0570 con 0x56235942a720 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1ad7d0 con 0x56235941f980 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de65e4740 10 client.0 _ll_register_callbacks cb 0x7f7dcc003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7ddbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359432e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7ddbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359432e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37678/0 (socket says 172.21.15.175:37678) 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7ddbe00640 1 -- 172.21.15.175:0/463669990 learned_addr learned my addr 172.21.15.175:0/463669990 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de1200640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235941f980 0x5623594328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.058+0000 7f7de1c00640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235942a720 0x562359423d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7ddbe00640 1 -- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235941f980 msgr2=0x5623594328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235941f980 0x5623594328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7ddbe00640 1 -- 172.21.15.175:0/463669990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235942a720 msgr2=0x562359423d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235942a720 0x562359423d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7ddbe00640 1 -- 172.21.15.175:0/463669990 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623594242c0 con 0x562359420ef0 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7ddbe00640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359432e10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc098010 tx=0x7f7dcc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7826a62284c99f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7dbea00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7dbea00640 1 -- 172.21.15.175:0/463669990 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc09b510 con 0x562359420ef0 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7dbea00640 1 -- 172.21.15.175:0/463669990 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f7dcc0ba4b0 con 0x562359420ef0 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd001f040 con 0x562359420ef0 2024-07-29T00:44:01.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7dbea00640 1 -- 172.21.15.175:0/463669990 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7dcc09ff60 con 0x562359420ef0 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.059+0000 7f7de65e4740 10 client.16764 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5623594a4c00 con 0x562359420ef0 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7dbea00640 1 -- 172.21.15.175:0/463669990 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd001f040 con 0x562359420ef0 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7dbea00640 1 -- 172.21.15.175:0/463669990 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f7dcc09ab00 con 0x562359420ef0 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 10 client.16764 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 10 client.16764 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:44:01.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 20 client.16764 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:01.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5623594a52c0 0x5623594a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:01.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 4 client.16764 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:01.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56235938d530 con 0x5623594a52c0 2024-07-29T00:44:01.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.060+0000 7f7de1200640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5623594a52c0 0x5623594a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:01.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7de1200640 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5623594a52c0 0x5623594a76b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4901074c21095173 server_cookie=badf3db1c11cb854 in_seq=0 out_seq=0 2024-07-29T00:44:01.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7dbea00640 10 client.16764 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:01.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7dbea00640 1 -- 172.21.15.175:0/463669990 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56235938d530 con 0x5623594a52c0 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7dbea00640 10 client.16764 handle_command_reply: tid=0 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7de65e4740 1 client.16764 shutdown 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 msgr2=0x562359432e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.061+0000 7f7de65e4740 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359432e10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc098010 tx=0x7f7dcc005be0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 shutdown_connections 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x5623594a52c0 0x5623594a76b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56235942a720 0x562359423d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562359420ef0 0x562359432e10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 --2- 172.21.15.175:0/463669990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235941f980 0x5623594328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 >> 172.21.15.175:0/463669990 conn(0x5623593e8950 msgr2=0x562359421cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7dbea00640 0 client.16764 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 shutdown_connections 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 1 -- 172.21.15.175:0/463669990 wait complete. 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.062+0000 7f7de65e4740 20 client.16764 trim_cache size 0 max 16384 2024-07-29T00:44:01.070 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:01.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7dc40a2d70 msgr2=0x7f7dc40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7dc40a2d70 0x7f7dc40a5230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009370 tx=0x7f7dd40098e0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 msgr2=0x7f7ddc19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 0x7f7ddc19a7f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e9c0 tx=0x7f7dcc00ee80 comp rx=0 tx=0).stop 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 shutdown_connections 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ddc109a80 0x7f7ddc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f7dc40a2d70 0x7f7dc40a5230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ddc104420 0x7f7ddc19a7f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 --2- 172.21.15.175:0/1253201562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ddc103a80 0x7f7ddc19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:01.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 >> 172.21.15.175:0/1253201562 conn(0x7f7ddc0ff780 msgr2=0x7f7ddc0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:01.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 shutdown_connections 2024-07-29T00:44:01.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:01.063+0000 7f7de3c00640 1 -- 172.21.15.175:0/1253201562 wait complete. 2024-07-29T00:44:02.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:44:02.179 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:01 smithi175 ceph-mon[32138]: pgmap v230: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-29T00:44:02.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:01 smithi181 ceph-mon[36706]: pgmap v230: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-29T00:44:02.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:01 smithi204 ceph-mon[37368]: pgmap v230: 97 pgs: 97 active+clean; 127 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-29T00:44:02.580 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:03.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 -- 172.21.15.175:0/4190570044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301076a0 msgr2=0x7f6830107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 --2- 172.21.15.175:0/4190570044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301076a0 0x7f6830107a80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6820009860 tx=0x7f682002a340 comp rx=0 tx=0).stop 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 -- 172.21.15.175:0/4190570044 shutdown_connections 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 --2- 172.21.15.175:0/4190570044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 0x7f683010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 --2- 172.21.15.175:0/4190570044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68301016a0 0x7f6830101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 --2- 172.21.15.175:0/4190570044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301076a0 0x7f6830107a80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 -- 172.21.15.175:0/4190570044 >> 172.21.15.175:0/4190570044 conn(0x7f68300fd3a0 msgr2=0x7f68300ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 -- 172.21.15.175:0/4190570044 shutdown_connections 2024-07-29T00:44:03.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.232+0000 7f682f200640 1 -- 172.21.15.175:0/4190570044 wait complete. 2024-07-29T00:44:03.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.233+0000 7f682f200640 1 Processor -- start 2024-07-29T00:44:03.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 -- start start 2024-07-29T00:44:03.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301016a0 0x7f683019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 0x7f683019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68301076a0 0x7f683019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111aa0 con 0x7f68301076a0 2024-07-29T00:44:03.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111920 con 0x7f68301016a0 2024-07-29T00:44:03.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111c20 con 0x7f6830102040 2024-07-29T00:44:03.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 0x7f683019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68301076a0 0x7f683019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301016a0 0x7f683019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 0x7f683019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57930/0 (socket says 172.21.15.175:57930) 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.234+0000 7f682d400640 1 -- 172.21.15.175:0/499573408 learned_addr learned my addr 172.21.15.175:0/499573408 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f682de00640 1 -- 172.21.15.175:0/499573408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 msgr2=0x7f683019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f682de00640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 0x7f683019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f682de00640 1 -- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68301076a0 msgr2=0x7f683019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f682de00640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68301076a0 0x7f683019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f682de00640 1 -- 172.21.15.175:0/499573408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683019f110 con 0x7f68301016a0 2024-07-29T00:44:03.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f682de00640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301016a0 0x7f683019a160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f682002a860 tx=0x7f6820033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:03.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.235+0000 7f6826a00640 1 -- 172.21.15.175:0/499573408 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820037040 con 0x7f68301016a0 2024-07-29T00:44:03.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.236+0000 7f6826a00640 1 -- 172.21.15.175:0/499573408 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6820033d50 con 0x7f68301016a0 2024-07-29T00:44:03.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.236+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683019f3a0 con 0x7f68301016a0 2024-07-29T00:44:03.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.236+0000 7f6826a00640 1 -- 172.21.15.175:0/499573408 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200039e0 con 0x7f68301016a0 2024-07-29T00:44:03.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.236+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a6ce0 con 0x7f68301016a0 2024-07-29T00:44:03.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 Processor -- start 2024-07-29T00:44:03.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 -- start start 2024-07-29T00:44:03.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6826a00640 1 -- 172.21.15.175:0/499573408 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6820042020 con 0x7f68301016a0 2024-07-29T00:44:03.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c501630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c501fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc8c467ab0 con 0x55dc8c501b70 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dc8c3ffa80 con 0x55dc8c5030e0 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.239+0000 7f6837a63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc8c463a70 con 0x55dc8c50c910 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c501630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c501fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c501630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55172/0 (socket says 172.21.15.175:55172) 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 -- 172.21.15.175:0/815828494 learned_addr learned my addr 172.21.15.175:0/815828494 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682d400640 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 -- 172.21.15.175:0/815828494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 msgr2=0x55dc8c50ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 -- 172.21.15.175:0/815828494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 msgr2=0x55dc8c501fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c501fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682e800640 1 -- 172.21.15.175:0/815828494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8c1d93c0 con 0x55dc8c5030e0 2024-07-29T00:44:03.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.240+0000 7f682d400640 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:44:03.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f682e800640 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c501630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6828009f50 tx=0x7f68280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7c30219289bcea server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:03.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f681f400640 1 -- 172.21.15.175:0/815828494 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828004d20 con 0x55dc8c5030e0 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f681f400640 1 -- 172.21.15.175:0/815828494 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828004050 con 0x55dc8c5030e0 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f681f400640 1 -- 172.21.15.175:0/815828494 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828005280 con 0x55dc8c5030e0 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f6826a00640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68140a2d70 0x7f68140a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f6826a00640 1 -- 172.21.15.175:0/499573408 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7f68200ee920 con 0x7f68301016a0 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.241+0000 7f682de00640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68140a2d70 0x7f68140a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 -- 172.21.15.175:0/815828494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 msgr2=0x55dc8c501630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c501630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6828009f50 tx=0x7f68280118d0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 -- 172.21.15.175:0/815828494 shutdown_connections 2024-07-29T00:44:03.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c501fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 --2- 172.21.15.175:0/815828494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c501630 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 -- 172.21.15.175:0/815828494 >> 172.21.15.175:0/815828494 conn(0x55dc8c4ca950 msgr2=0x55dc8c4fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:03.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 -- 172.21.15.175:0/815828494 shutdown_connections 2024-07-29T00:44:03.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f6837a63740 1 -- 172.21.15.175:0/815828494 wait complete. 2024-07-29T00:44:03.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.242+0000 7f682de00640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68140a2d70 0x7f68140a5230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55dc8c4ff060 tx=0x7f68200f4530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:03.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 -- start start 2024-07-29T00:44:03.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 Processor -- start 2024-07-29T00:44:03.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c509860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c509da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc8c50f6b0 con 0x55dc8c501b70 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dc8c422c10 con 0x55dc8c5030e0 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.243+0000 7f6837a63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc8c50f830 con 0x55dc8c50c910 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f6837a63740 10 client.0 _ll_register_callbacks cb 0x55dc8c3eba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c509860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c509860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37702/0 (socket says 172.21.15.175:37702) 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682e800640 1 -- 172.21.15.175:0/2481643576 learned_addr learned my addr 172.21.15.175:0/2481643576 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c509da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682e800640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682d400640 1 -- 172.21.15.175:0/2481643576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 msgr2=0x55dc8c50a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682d400640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682d400640 1 -- 172.21.15.175:0/2481643576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 msgr2=0x55dc8c509860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682d400640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c509860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682d400640 1 -- 172.21.15.175:0/2481643576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8c4261e0 con 0x55dc8c5030e0 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.244+0000 7f682e800640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c509860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f682d400640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c509da0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f68180058c0 tx=0x7f6818012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6792dfb41a8d9f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:03.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6813e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6813e00640 1 -- 172.21.15.175:0/2481643576 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818010d40 con 0x55dc8c5030e0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6828004050 con 0x55dc8c5030e0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6813e00640 1 -- 172.21.15.175:0/2481643576 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7f6818005020 con 0x55dc8c5030e0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6813e00640 1 -- 172.21.15.175:0/2481643576 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6818005220 con 0x55dc8c5030e0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6837a63740 10 client.26924 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.245+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dc8c51b400 con 0x55dc8c5030e0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6813e00640 1 -- 172.21.15.175:0/2481643576 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828004050 con 0x55dc8c5030e0 2024-07-29T00:44:03.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6813e00640 1 -- 172.21.15.175:0/2481643576 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f681801c5c0 con 0x55dc8c5030e0 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6837a63740 10 client.26924 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6837a63740 10 client.26924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6837a63740 20 client.26924 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6837a63740 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c51c8c0 0x55dc8c51ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6837a63740 4 client.26924 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8c475020 con 0x55dc8c51c8c0 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.246+0000 7f682e800640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c51c8c0 0x55dc8c51ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.247+0000 7f682e800640 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c51c8c0 0x55dc8c51ecf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=276225d31bba93b0 server_cookie=86bd93b47564656e in_seq=0 out_seq=0 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.247+0000 7f6813e00640 10 client.26924 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6813e00640 1 -- 172.21.15.175:0/2481643576 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc8c475020 con 0x55dc8c51c8c0 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6813e00640 10 client.26924 handle_command_reply: tid=0 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 client.26924 shutdown 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 msgr2=0x55dc8c509da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c509da0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f68180058c0 tx=0x7f6818012b60 comp rx=0 tx=0).stop 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 shutdown_connections 2024-07-29T00:44:03.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c50a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c51c8c0 0x55dc8c51ecf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c5030e0 0x55dc8c509da0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 --2- 172.21.15.175:0/2481643576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c501b70 0x55dc8c509860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6813e00640 0 client.26924 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.248+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 >> 172.21.15.175:0/2481643576 conn(0x55dc8c4ca950 msgr2=0x55dc8c503ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.249+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 shutdown_connections 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.249+0000 7f6837a63740 1 -- 172.21.15.175:0/2481643576 wait complete. 2024-07-29T00:44:03.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.249+0000 7f6837a63740 20 client.26924 trim_cache size 0 max 16384 2024-07-29T00:44:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.263+0000 7f6837a63740 1 Processor -- start 2024-07-29T00:44:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 -- start start 2024-07-29T00:44:03.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c524600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c5030e0 0x55dc8c524b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c50c910 0x55dc8c4fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68301af800 con 0x55dc8c50c910 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b1340 con 0x55dc8c501b70 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f6837a63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68301afed0 con 0x55dc8c5030e0 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f682e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c524600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f682e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c524600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55180/0 (socket says 172.21.15.175:55180) 2024-07-29T00:44:03.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f682d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c5030e0 0x55dc8c524b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f682e800640 1 -- 172.21.15.175:0/506093287 learned_addr learned my addr 172.21.15.175:0/506093287 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:03.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.264+0000 7f682e800640 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c50c910 0x55dc8c4fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.265+0000 7f682d400640 1 -- 172.21.15.175:0/506093287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 msgr2=0x55dc8c524600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.265+0000 7f682d400640 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c524600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.265+0000 7f682d400640 1 -- 172.21.15.175:0/506093287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c50c910 msgr2=0x55dc8c4fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.265+0000 7f682d400640 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c50c910 0x55dc8c4fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.265+0000 7f682d400640 1 -- 172.21.15.175:0/506093287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8c5028a0 con 0x55dc8c5030e0 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.265+0000 7f682d400640 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c5030e0 0x55dc8c524b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f681801ca60 tx=0x7f6818009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b89ce6fda9e78f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f681ea00640 1 -- 172.21.15.175:0/506093287 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180053c0 con 0x55dc8c5030e0 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f681ea00640 1 -- 172.21.15.175:0/506093287 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818005560 con 0x55dc8c5030e0 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f681ea00640 1 -- 172.21.15.175:0/506093287 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818004b70 con 0x55dc8c5030e0 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 -- 172.21.15.175:0/506093287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c5030e0 msgr2=0x55dc8c524b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c5030e0 0x55dc8c524b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f681801ca60 tx=0x7f6818009ef0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 -- 172.21.15.175:0/506093287 shutdown_connections 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c50c910 0x55dc8c4fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c5030e0 0x55dc8c524b40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 --2- 172.21.15.175:0/506093287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c524600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.266+0000 7f6837a63740 1 -- 172.21.15.175:0/506093287 >> 172.21.15.175:0/506093287 conn(0x55dc8c4ca950 msgr2=0x55dc8c4f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.267+0000 7f6837a63740 1 -- 172.21.15.175:0/506093287 shutdown_connections 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.267+0000 7f6837a63740 1 -- 172.21.15.175:0/506093287 wait complete. 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.267+0000 7f6837a63740 1 -- start start 2024-07-29T00:44:03.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.267+0000 7f6837a63740 1 Processor -- start 2024-07-29T00:44:03.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c515130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 0x55dc8c506490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c5069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b0c70 con 0x55dc8c5030e0 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b05a0 con 0x55dc8c501b70 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68301ad800 con 0x55dc8c50c910 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f6837a63740 10 client.0 _ll_register_callbacks cb 0x7f6818009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f682d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 0x55dc8c506490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f682d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 0x55dc8c506490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37720/0 (socket says 172.21.15.175:37720) 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f682d400640 1 -- 172.21.15.175:0/3020117521 learned_addr learned my addr 172.21.15.175:0/3020117521 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f682e800640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c5069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.268+0000 7f682e800640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c515130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f682d400640 1 -- 172.21.15.175:0/3020117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 msgr2=0x55dc8c5069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f682d400640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c5069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f682d400640 1 -- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 msgr2=0x55dc8c515130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f682d400640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c515130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f682d400640 1 -- 172.21.15.175:0/3020117521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8c5070c0 con 0x55dc8c5030e0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f682d400640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 0x55dc8c506490 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6818010df0 tx=0x7f6818010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3748fcdd547f2c89 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f6813e00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f6813e00640 1 -- 172.21.15.175:0/3020117521 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818039060 con 0x55dc8c5030e0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818005560 con 0x55dc8c5030e0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.269+0000 7f6813e00640 1 -- 172.21.15.175:0/3020117521 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7f6818004040 con 0x55dc8c5030e0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6813e00640 1 -- 172.21.15.175:0/3020117521 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f68180053c0 con 0x55dc8c5030e0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 10 client.16782 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dc8c5874d0 con 0x55dc8c5030e0 2024-07-29T00:44:03.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6813e00640 1 -- 172.21.15.175:0/3020117521 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818005560 con 0x55dc8c5030e0 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6813e00640 1 -- 172.21.15.175:0/3020117521 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f68180047f0 con 0x55dc8c5030e0 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 10 client.16782 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 20 client.16782 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c587b10 0x55dc8c589f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 4 client.16782 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.270+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc8c46f530 con 0x55dc8c587b10 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.271+0000 7f682e800640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c587b10 0x55dc8c589f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:03.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.271+0000 7f682e800640 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c587b10 0x55dc8c589f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aacc3a9ae108fe31 server_cookie=54495ebc68b1350b in_seq=0 out_seq=0 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.271+0000 7f6813e00640 10 client.16782 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6813e00640 1 -- 172.21.15.175:0/3020117521 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc8c46f530 con 0x55dc8c587b10 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6813e00640 10 client.16782 handle_command_reply: tid=0 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 client.16782 shutdown 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 msgr2=0x55dc8c506490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 0x55dc8c506490 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6818010df0 tx=0x7f6818010e20 comp rx=0 tx=0).stop 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 shutdown_connections 2024-07-29T00:44:03.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8c50c910 0x55dc8c5069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc8c5030e0 0x55dc8c506490 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x55dc8c587b10 0x55dc8c589f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 --2- 172.21.15.175:0/3020117521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc8c501b70 0x55dc8c515130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 >> 172.21.15.175:0/3020117521 conn(0x55dc8c4ca950 msgr2=0x55dc8c5036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6813e00640 0 client.16782 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 shutdown_connections 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 1 -- 172.21.15.175:0/3020117521 wait complete. 2024-07-29T00:44:03.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.272+0000 7f6837a63740 20 client.16782 trim_cache size 0 max 16384 2024-07-29T00:44:03.279 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:03.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68140a2d70 msgr2=0x7f68140a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68140a2d70 0x7f68140a5230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55dc8c4ff060 tx=0x7f68200f4530 comp rx=0 tx=0).stop 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301016a0 msgr2=0x7f683019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301016a0 0x7f683019a160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f682002a860 tx=0x7f6820033b60 comp rx=0 tx=0).stop 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 shutdown_connections 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f68140a2d70 0x7f68140a5230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68301076a0 0x7f683019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6830102040 0x7f683019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 --2- 172.21.15.175:0/499573408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68301016a0 0x7f683019a160 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:03.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 >> 172.21.15.175:0/499573408 conn(0x7f68300fd3a0 msgr2=0x7f683010c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:03.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 shutdown_connections 2024-07-29T00:44:03.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:03.273+0000 7f682f200640 1 -- 172.21.15.175:0/499573408 wait complete. 2024-07-29T00:44:04.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:03 smithi175 ceph-mon[32138]: pgmap v231: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 65 KiB/s wr, 32 op/s 2024-07-29T00:44:04.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:03 smithi175 ceph-mon[32138]: osdmap e95: 12 total, 12 up, 12 in 2024-07-29T00:44:04.156 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:44:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:03 smithi181 ceph-mon[36706]: pgmap v231: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 65 KiB/s wr, 32 op/s 2024-07-29T00:44:04.352 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:03 smithi181 ceph-mon[36706]: osdmap e95: 12 total, 12 up, 12 in 2024-07-29T00:44:04.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:03 smithi204 ceph-mon[37368]: pgmap v231: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 65 KiB/s wr, 32 op/s 2024-07-29T00:44:04.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:03 smithi204 ceph-mon[37368]: osdmap e95: 12 total, 12 up, 12 in 2024-07-29T00:44:04.617 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:05.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 -- 172.21.15.175:0/3482062616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 msgr2=0x7fb710114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 --2- 172.21.15.175:0/3482062616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb710114b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb708009be0 tx=0x7fb70802a6d0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 -- 172.21.15.175:0/3482062616 shutdown_connections 2024-07-29T00:44:05.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 --2- 172.21.15.175:0/3482062616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb710114b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 --2- 172.21.15.175:0/3482062616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb710108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 --2- 172.21.15.175:0/3482062616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 0x7fb71010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 -- 172.21.15.175:0/3482062616 >> 172.21.15.175:0/3482062616 conn(0x7fb710077670 msgr2=0x7fb710079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 -- 172.21.15.175:0/3482062616 shutdown_connections 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.261+0000 7fb714c00640 1 -- 172.21.15.175:0/3482062616 wait complete. 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 Processor -- start 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 -- start start 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb71019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb71019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 0x7fb7101a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb710118060 con 0x7fb710107d10 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb710117ee0 con 0x7fb710108670 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.263+0000 7fb714c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101181e0 con 0x7fb71010dd50 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb71019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb71019e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37752/0 (socket says 172.21.15.175:37752) 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb71019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 -- 172.21.15.175:0/2773350756 learned_addr learned my addr 172.21.15.175:0/2773350756 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:05.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70fe00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 0x7fb7101a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 -- 172.21.15.175:0/2773350756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 msgr2=0x7fb7101a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 0x7fb7101a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 -- 172.21.15.175:0/2773350756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 msgr2=0x7fb71019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb71019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70f400640 1 -- 172.21.15.175:0/2773350756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7101a3470 con 0x7fb710107d10 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70fe00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 0x7fb7101a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.264+0000 7fb70ea00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb71019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:05.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.265+0000 7fb70f400640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb71019e4c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb70000e9f0 tx=0x7fb70000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:05.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.265+0000 7fb707e00640 1 -- 172.21.15.175:0/2773350756 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000cd80 con 0x7fb710107d10 2024-07-29T00:44:05.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.265+0000 7fb707e00640 1 -- 172.21.15.175:0/2773350756 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700010910 con 0x7fb710107d10 2024-07-29T00:44:05.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.265+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7101a3760 con 0x7fb710107d10 2024-07-29T00:44:05.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.265+0000 7fb707e00640 1 -- 172.21.15.175:0/2773350756 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700018990 con 0x7fb710107d10 2024-07-29T00:44:05.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.265+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7101ab040 con 0x7fb710107d10 2024-07-29T00:44:05.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.268+0000 7fb718e2a740 1 Processor -- start 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.268+0000 7fb707e00640 1 -- 172.21.15.175:0/2773350756 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb700010ac0 con 0x7fb710107d10 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 -- start start 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba237630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba237fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba244d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556bba19dab0 con 0x556bba242910 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556bba135a80 con 0x556bba237b70 2024-07-29T00:44:05.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb718e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556bba199a70 con 0x556bba2390e0 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba237630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb70ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba244d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba237630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:57984/0 (socket says 172.21.15.175:57984) 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb70fe00640 1 -- 172.21.15.175:0/1537291298 learned_addr learned my addr 172.21.15.175:0/1537291298 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.269+0000 7fb70f400640 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba237fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70fe00640 1 -- 172.21.15.175:0/1537291298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 msgr2=0x556bba237fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70fe00640 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba237fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70fe00640 1 -- 172.21.15.175:0/1537291298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 msgr2=0x556bba244d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70fe00640 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba244d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70fe00640 1 -- 172.21.15.175:0/1537291298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb9f0f3c0 con 0x556bba2390e0 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70f400640 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba237fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb707e00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb6f40a2d70 0x7fb6f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb707e00640 1 -- 172.21.15.175:0/2773350756 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7fb7000cf7b0 con 0x7fb710107d10 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70fe00640 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba237630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb708005f00 tx=0x7fb7080304d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c33b3895bd2034c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:05.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.270+0000 7fb70f400640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb6f40a2d70 0x7fb6f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.271+0000 7fb6fbe00640 1 -- 172.21.15.175:0/1537291298 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708040070 con 0x556bba2390e0 2024-07-29T00:44:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.271+0000 7fb6fbe00640 1 -- 172.21.15.175:0/1537291298 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708003970 con 0x556bba2390e0 2024-07-29T00:44:05.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.271+0000 7fb6fbe00640 1 -- 172.21.15.175:0/1537291298 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708036040 con 0x556bba2390e0 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.271+0000 7fb718e2a740 1 -- 172.21.15.175:0/1537291298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 msgr2=0x556bba237630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.271+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba237630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb708005f00 tx=0x7fb7080304d0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.271+0000 7fb70f400640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb6f40a2d70 0x7fb6f40a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556bba235060 tx=0x7fb700000bc0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 -- 172.21.15.175:0/1537291298 shutdown_connections 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba244d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba237fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1537291298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba237630 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 -- 172.21.15.175:0/1537291298 >> 172.21.15.175:0/1537291298 conn(0x556bba200950 msgr2=0x556bba234200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 -- 172.21.15.175:0/1537291298 shutdown_connections 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 -- 172.21.15.175:0/1537291298 wait complete. 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.272+0000 7fb718e2a740 1 -- start start 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 Processor -- start 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba23f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba23fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba240330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556bba245650 con 0x556bba242910 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556bba158c10 con 0x556bba237b70 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556bba2457d0 con 0x556bba2390e0 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.273+0000 7fb718e2a740 10 client.0 _ll_register_callbacks cb 0x556bba121a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba23f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba23f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55224/0 (socket says 172.21.15.175:55224) 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70fe00640 1 -- 172.21.15.175:0/3551579459 learned_addr learned my addr 172.21.15.175:0/3551579459 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:05.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba23fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70fe00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba240330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 -- 172.21.15.175:0/3551579459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 msgr2=0x556bba23f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba23f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 -- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 msgr2=0x556bba240330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba240330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 -- 172.21.15.175:0/3551579459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bba15c1e0 con 0x556bba2390e0 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70fe00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba23f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:05.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb70ea00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba23fdf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb71010d400 tx=0x7fb6fc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90e682e3d7c61e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.274+0000 7fb6f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb6f8c00640 1 -- 172.21.15.175:0/3551579459 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc01e020 con 0x556bba2390e0 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb708003970 con 0x556bba2390e0 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb6f8c00640 1 -- 172.21.15.175:0/3551579459 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7fb6fc0047a0 con 0x556bba2390e0 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb6f8c00640 1 -- 172.21.15.175:0/3551579459 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6fc026050 con 0x556bba2390e0 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb718e2a740 10 client.26893 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:05.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556bba251490 con 0x556bba2390e0 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.275+0000 7fb6f8c00640 1 -- 172.21.15.175:0/3551579459 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708003970 con 0x556bba2390e0 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb6f8c00640 1 -- 172.21.15.175:0/3551579459 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb6fc019070 con 0x556bba2390e0 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb718e2a740 10 client.26893 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb718e2a740 10 client.26893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb718e2a740 20 client.26893 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb718e2a740 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2528f0 0x556bba254d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb718e2a740 4 client.26893 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:05.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bba1ab020 con 0x556bba2528f0 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.276+0000 7fb70fe00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2528f0 0x556bba254d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.277+0000 7fb70fe00640 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2528f0 0x556bba254d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bdb07bf2248a0ad server_cookie=c2f81b63d1bf2dbd in_seq=0 out_seq=0 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.277+0000 7fb6f8c00640 10 client.26893 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.277+0000 7fb6f8c00640 1 -- 172.21.15.175:0/3551579459 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556bba1ab020 con 0x556bba2528f0 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.277+0000 7fb6f8c00640 10 client.26893 handle_command_reply: tid=0 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.277+0000 7fb718e2a740 1 client.26893 shutdown 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 msgr2=0x556bba23fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba23fdf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb71010d400 tx=0x7fb6fc00f790 comp rx=0 tx=0).stop 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 shutdown_connections 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2528f0 0x556bba254d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba242910 0x556bba240330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba23fdf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb6f8c00640 0 client.26893 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 --2- 172.21.15.175:0/3551579459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba237b70 0x556bba23f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 >> 172.21.15.175:0/3551579459 conn(0x556bba200950 msgr2=0x556bba239f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:05.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 shutdown_connections 2024-07-29T00:44:05.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 1 -- 172.21.15.175:0/3551579459 wait complete. 2024-07-29T00:44:05.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.278+0000 7fb718e2a740 20 client.26893 trim_cache size 0 max 16384 2024-07-29T00:44:05.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.292+0000 7fb718e2a740 1 Processor -- start 2024-07-29T00:44:05.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 -- start start 2024-07-29T00:44:05.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba25a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba25ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba242910 0x556bba232b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101b3c20 con 0x556bba237b70 2024-07-29T00:44:05.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101b5760 con 0x556bba242910 2024-07-29T00:44:05.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb718e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101b42f0 con 0x556bba2390e0 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba25a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb70ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba25ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba25a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37776/0 (socket says 172.21.15.175:37776) 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb70fe00640 1 -- 172.21.15.175:0/1901220688 learned_addr learned my addr 172.21.15.175:0/1901220688 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.293+0000 7fb70ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba25ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58006/0 (socket says 172.21.15.175:58006) 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70fe00640 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba242910 0x556bba232b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70ea00640 1 -- 172.21.15.175:0/1901220688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba242910 msgr2=0x556bba232b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70ea00640 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba242910 0x556bba232b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70ea00640 1 -- 172.21.15.175:0/1901220688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 msgr2=0x556bba25a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70ea00640 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba25a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70ea00640 1 -- 172.21.15.175:0/1901220688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bba23a6c0 con 0x556bba2390e0 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb70ea00640 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba25ab20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009530 tx=0x7fb6fc019590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f02240c2f1142fc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb6fb400640 1 -- 172.21.15.175:0/1901220688 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc02e020 con 0x556bba2390e0 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb6fb400640 1 -- 172.21.15.175:0/1901220688 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc019cc0 con 0x556bba2390e0 2024-07-29T00:44:05.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.294+0000 7fb6fb400640 1 -- 172.21.15.175:0/1901220688 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc01e040 con 0x556bba2390e0 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 -- 172.21.15.175:0/1901220688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 msgr2=0x556bba25ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba25ab20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009530 tx=0x7fb6fc019590 comp rx=0 tx=0).stop 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 -- 172.21.15.175:0/1901220688 shutdown_connections 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba242910 0x556bba232b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba2390e0 0x556bba25ab20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 --2- 172.21.15.175:0/1901220688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba25a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 -- 172.21.15.175:0/1901220688 >> 172.21.15.175:0/1901220688 conn(0x556bba200950 msgr2=0x556bba23e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:05.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 -- 172.21.15.175:0/1901220688 shutdown_connections 2024-07-29T00:44:05.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.295+0000 7fb718e2a740 1 -- 172.21.15.175:0/1901220688 wait complete. 2024-07-29T00:44:05.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.296+0000 7fb718e2a740 1 -- start start 2024-07-29T00:44:05.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.296+0000 7fb718e2a740 1 Processor -- start 2024-07-29T00:44:05.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba246010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba2390e0 0x556bba246550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba242910 0x556bba2bb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101b5090 con 0x556bba237b70 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101b49c0 con 0x556bba2390e0 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101b1c20 con 0x556bba242910 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb718e2a740 10 client.0 _ll_register_callbacks cb 0x7fb6fc011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba246010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb70fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba246010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37786/0 (socket says 172.21.15.175:37786) 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb70fe00640 1 -- 172.21.15.175:0/447630520 learned_addr learned my addr 172.21.15.175:0/447630520 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb70ea00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba2390e0 0x556bba246550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.297+0000 7fb70fe00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba242910 0x556bba2bb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb70ea00640 1 -- 172.21.15.175:0/447630520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba242910 msgr2=0x556bba2bb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb70ea00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba242910 0x556bba2bb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb70ea00640 1 -- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 msgr2=0x556bba246010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb70ea00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba246010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb70ea00640 1 -- 172.21.15.175:0/447630520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bba2bbae0 con 0x556bba2390e0 2024-07-29T00:44:05.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb70ea00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba2390e0 0x556bba246550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00ffd0 tx=0x7fb6fc019b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4eec7997d2c05a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:05.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb6f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:05.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb6f8c00640 1 -- 172.21.15.175:0/447630520 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc037020 con 0x556bba2390e0 2024-07-29T00:44:05.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb6f8c00640 1 -- 172.21.15.175:0/447630520 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7fb6fc033450 con 0x556bba2390e0 2024-07-29T00:44:05.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.298+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6fc019cc0 con 0x556bba2390e0 2024-07-29T00:44:05.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb6f8c00640 1 -- 172.21.15.175:0/447630520 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6fc042050 con 0x556bba2390e0 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 10 client.26948 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556bba2bcd50 con 0x556bba2390e0 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb6f8c00640 1 -- 172.21.15.175:0/447630520 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc019cc0 con 0x556bba2390e0 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb6f8c00640 1 -- 172.21.15.175:0/447630520 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb6fc019050 con 0x556bba2390e0 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 10 client.26948 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 10 client.26948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 20 client.26948 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:05.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2bd300 0x556bba2bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 4 client.26948 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bba1a5530 con 0x556bba2bd300 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.299+0000 7fb70fe00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2bd300 0x556bba2bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb70fe00640 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2bd300 0x556bba2bf720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d38523a8c5bf334f server_cookie=11becc22439cfd54 in_seq=0 out_seq=0 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb6f8c00640 10 client.26948 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb6f8c00640 1 -- 172.21.15.175:0/447630520 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bba1a5530 con 0x556bba2bd300 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb6f8c00640 10 client.26948 handle_command_reply: tid=0 2024-07-29T00:44:05.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb718e2a740 1 client.26948 shutdown 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba2390e0 msgr2=0x556bba246550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.300+0000 7fb718e2a740 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba2390e0 0x556bba246550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00ffd0 tx=0x7fb6fc019b20 comp rx=0 tx=0).stop 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 shutdown_connections 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x556bba2bd300 0x556bba2bf720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bba242910 0x556bba2bb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556bba2390e0 0x556bba246550 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 --2- 172.21.15.175:0/447630520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bba237b70 0x556bba246010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb6f8c00640 0 client.26948 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:05.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 >> 172.21.15.175:0/447630520 conn(0x556bba200950 msgr2=0x556bba232b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:05.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 shutdown_connections 2024-07-29T00:44:05.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 1 -- 172.21.15.175:0/447630520 wait complete. 2024-07-29T00:44:05.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.301+0000 7fb718e2a740 20 client.26948 trim_cache size 0 max 16384 2024-07-29T00:44:05.309 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:05.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb6f40a2d70 msgr2=0x7fb6f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb6f40a2d70 0x7fb6f40a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556bba235060 tx=0x7fb700000bc0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 msgr2=0x7fb71019e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb71019e4c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb70000e9f0 tx=0x7fb70000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 shutdown_connections 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb71010dd50 0x7fb7101a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb710108670 0x7fb71019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb6f40a2d70 0x7fb6f40a5230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 --2- 172.21.15.175:0/2773350756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb710107d10 0x7fb71019e4c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 >> 172.21.15.175:0/2773350756 conn(0x7fb710077670 msgr2=0x7fb710078dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:05.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 shutdown_connections 2024-07-29T00:44:05.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:05.302+0000 7fb714c00640 1 -- 172.21.15.175:0/2773350756 wait complete. 2024-07-29T00:44:06.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:05 smithi175 ceph-mon[32138]: pgmap v233: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 73 KiB/s wr, 34 op/s 2024-07-29T00:44:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:05 smithi181 ceph-mon[36706]: pgmap v233: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 73 KiB/s wr, 34 op/s 2024-07-29T00:44:06.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:44:06.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:05 smithi204 ceph-mon[37368]: pgmap v233: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 73 KiB/s wr, 34 op/s 2024-07-29T00:44:06.799 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:07.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 -- 172.21.15.175:0/328850814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8541016c0 msgr2=0x7fb854101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 --2- 172.21.15.175:0/328850814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8541016c0 0x7fb854101b20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009860 tx=0x7fb83c02a340 comp rx=0 tx=0).stop 2024-07-29T00:44:07.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 -- 172.21.15.175:0/328850814 shutdown_connections 2024-07-29T00:44:07.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 --2- 172.21.15.175:0/328850814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb854102060 0x7fb85410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 --2- 172.21.15.175:0/328850814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8541016c0 0x7fb854101b20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 --2- 172.21.15.175:0/328850814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541076c0 0x7fb854107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 -- 172.21.15.175:0/328850814 >> 172.21.15.175:0/328850814 conn(0x7fb8540fd400 msgr2=0x7fb8540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:07.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 -- 172.21.15.175:0/328850814 shutdown_connections 2024-07-29T00:44:07.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.472+0000 7fb859a00640 1 -- 172.21.15.175:0/328850814 wait complete. 2024-07-29T00:44:07.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.473+0000 7fb859a00640 1 Processor -- start 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.473+0000 7fb859a00640 1 -- start start 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 0x7fb85419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb854102060 0x7fb85419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8541076c0 0x7fb85419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb854111a80 con 0x7fb8541016c0 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb854111900 con 0x7fb854102060 2024-07-29T00:44:07.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb854111c00 con 0x7fb8541076c0 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb853e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 0x7fb85419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb853e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 0x7fb85419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37812/0 (socket says 172.21.15.175:37812) 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb853e00640 1 -- 172.21.15.175:0/3244769674 learned_addr learned my addr 172.21.15.175:0/3244769674 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb859000640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8541076c0 0x7fb85419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.474+0000 7fb84b400640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb854102060 0x7fb85419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859000640 1 -- 172.21.15.175:0/3244769674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb854102060 msgr2=0x7fb85419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859000640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb854102060 0x7fb85419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859000640 1 -- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 msgr2=0x7fb85419a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859000640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 0x7fb85419a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859000640 1 -- 172.21.15.175:0/3244769674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85419f0f0 con 0x7fb8541076c0 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb853e00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 0x7fb85419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859000640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8541076c0 0x7fb85419ea10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009fd0 tx=0x7fb84c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb851600640 1 -- 172.21.15.175:0/3244769674 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00ce20 con 0x7fb8541076c0 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb851600640 1 -- 172.21.15.175:0/3244769674 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c005020 con 0x7fb8541076c0 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb851600640 1 -- 172.21.15.175:0/3244769674 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c010720 con 0x7fb8541076c0 2024-07-29T00:44:07.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.475+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85419f3e0 con 0x7fb8541076c0 2024-07-29T00:44:07.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.476+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8541a6cc0 con 0x7fb8541076c0 2024-07-29T00:44:07.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 Processor -- start 2024-07-29T00:44:07.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb851600640 1 -- 172.21.15.175:0/3244769674 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb84c005540 con 0x7fb8541076c0 2024-07-29T00:44:07.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 -- start start 2024-07-29T00:44:07.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.480 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8c8910 0x55ad9d8cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d823ab0 con 0x55ad9d8c8910 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d7bba80 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.479+0000 7fb85dc26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d81fa70 con 0x55ad9d8bdb70 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb84b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb84b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8c8910 0x55ad9d8cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58052/0 (socket says 172.21.15.175:58052) 2024-07-29T00:44:07.481 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 -- 172.21.15.175:0/3871866572 learned_addr learned my addr 172.21.15.175:0/3871866572 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:07.482 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb84b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8c8910 0x55ad9d8cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37828/0 (socket says 172.21.15.175:37828) 2024-07-29T00:44:07.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 -- 172.21.15.175:0/3871866572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bf0e0 msgr2=0x55ad9d8bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 -- 172.21.15.175:0/3871866572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8c8910 msgr2=0x55ad9d8cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8c8910 0x55ad9d8cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.484 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.480+0000 7fb853e00640 1 -- 172.21.15.175:0/3871866572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9d5953c0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb853e00640 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bdfb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb844011540 tx=0x7fb844011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8129f005cc253b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb851600640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8380a2cf0 0x7fb8380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb849600640 1 -- 172.21.15.175:0/3871866572 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400f640 con 0x55ad9d8bdb70 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb849600640 1 -- 172.21.15.175:0/3871866572 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84400fcc0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb849600640 1 -- 172.21.15.175:0/3871866572 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440183b0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb851600640 1 -- 172.21.15.175:0/3244769674 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7fb84c0cfc30 con 0x7fb8541076c0 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.481+0000 7fb84b400640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8380a2cf0 0x7fb8380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.485 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 -- 172.21.15.175:0/3871866572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 msgr2=0x55ad9d8bdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bdfb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb844011540 tx=0x7fb844011a00 comp rx=0 tx=0).stop 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 -- 172.21.15.175:0/3871866572 shutdown_connections 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8c8910 0x55ad9d8cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bdfb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3871866572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 -- 172.21.15.175:0/3871866572 >> 172.21.15.175:0/3871866572 conn(0x55ad9d886950 msgr2=0x55ad9d8ba200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb84b400640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8380a2cf0 0x7fb8380a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb83c02a930 tx=0x7fb83c0058d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:07.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 -- 172.21.15.175:0/3871866572 shutdown_connections 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.482+0000 7fb85dc26740 1 -- 172.21.15.175:0/3871866572 wait complete. 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.483+0000 7fb85dc26740 1 -- start start 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.483+0000 7fb85dc26740 1 Processor -- start 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.483+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.483+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb85dc26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d8cb6b0 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb85dc26740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d7dec10 con 0x55ad9d8c8910 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb85dc26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d8cb830 con 0x55ad9d8bdb70 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb85dc26740 10 client.0 _ll_register_callbacks cb 0x55ad9d7a7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb84b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb84b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37844/0 (socket says 172.21.15.175:37844) 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb84b400640 1 -- 172.21.15.175:0/3457056831 learned_addr learned my addr 172.21.15.175:0/3457056831 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb853e00640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb859000640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb853e00640 1 -- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 msgr2=0x55ad9d8c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb853e00640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb853e00640 1 -- 172.21.15.175:0/3457056831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 msgr2=0x55ad9d8c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb853e00640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb853e00640 1 -- 172.21.15.175:0/3457056831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9d7e21e0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.484+0000 7fb84b400640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb853e00640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8c58a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb84401b040 tx=0x7fb844005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be347865d4de53b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:07.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb842000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb842000640 1 -- 172.21.15.175:0/3457056831 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844028030 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb842000640 1 -- 172.21.15.175:0/3457056831 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7fb844016040 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84400fcc0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb842000640 1 -- 172.21.15.175:0/3457056831 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb844032050 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb85dc26740 10 client.26911 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.485+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad9d8d7440 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb842000640 1 -- 172.21.15.175:0/3457056831 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84400fcc0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb842000640 1 -- 172.21.15.175:0/3457056831 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb8440046c0 con 0x55ad9d8bdb70 2024-07-29T00:44:07.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb85dc26740 10 client.26911 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb85dc26740 10 client.26911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb85dc26740 20 client.26911 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8d88a0 0x55ad9d8dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb85dc26740 4 client.26911 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad9d831020 con 0x55ad9d8d88a0 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.486+0000 7fb84b400640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8d88a0 0x55ad9d8dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.487+0000 7fb84b400640 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8d88a0 0x55ad9d8dac90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa154b46101b05 server_cookie=9ed30fc5ea174fb1 in_seq=0 out_seq=0 2024-07-29T00:44:07.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.487+0000 7fb842000640 10 client.26911 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb842000640 1 -- 172.21.15.175:0/3457056831 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad9d831020 con 0x55ad9d8d88a0 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb842000640 10 client.26911 handle_command_reply: tid=0 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 client.26911 shutdown 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 msgr2=0x55ad9d8c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8c58a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb84401b040 tx=0x7fb844005e80 comp rx=0 tx=0).stop 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 shutdown_connections 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8d88a0 0x55ad9d8dac90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 --2- 172.21.15.175:0/3457056831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8c58a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 >> 172.21.15.175:0/3457056831 conn(0x55ad9d886950 msgr2=0x55ad9d8bff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb842000640 0 client.26911 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 shutdown_connections 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.488+0000 7fb85dc26740 1 -- 172.21.15.175:0/3457056831 wait complete. 2024-07-29T00:44:07.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.489+0000 7fb85dc26740 20 client.26911 trim_cache size 0 max 16384 2024-07-29T00:44:07.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 Processor -- start 2024-07-29T00:44:07.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 -- start start 2024-07-29T00:44:07.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541af780 con 0x55ad9d8bdb70 2024-07-29T00:44:07.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541b12c0 con 0x55ad9d8c8910 2024-07-29T00:44:07.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.503+0000 7fb85dc26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541afe50 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:58066/0 (socket says 172.21.15.175:58066) 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 -- 172.21.15.175:0/301492500 learned_addr learned my addr 172.21.15.175:0/301492500 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb853e00640 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb859000640 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 -- 172.21.15.175:0/301492500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 msgr2=0x55ad9d8ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 -- 172.21.15.175:0/301492500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bdb70 msgr2=0x55ad9d8bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 -- 172.21.15.175:0/301492500 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9d8c06c0 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb859000640 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.504+0000 7fb84b400640 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c9d00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005940 tx=0x7fb83c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e01590f5cf81401 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb848c00640 1 -- 172.21.15.175:0/301492500 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c05b070 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb848c00640 1 -- 172.21.15.175:0/301492500 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c003d90 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb848c00640 1 -- 172.21.15.175:0/301492500 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c037040 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 -- 172.21.15.175:0/301492500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 msgr2=0x55ad9d8c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c9d00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005940 tx=0x7fb83c005f90 comp rx=0 tx=0).stop 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 -- 172.21.15.175:0/301492500 shutdown_connections 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8c8910 0x55ad9d8ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8c9d00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 --2- 172.21.15.175:0/301492500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.505+0000 7fb85dc26740 1 -- 172.21.15.175:0/301492500 >> 172.21.15.175:0/301492500 conn(0x55ad9d886950 msgr2=0x55ad9d8da8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:07.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.506+0000 7fb85dc26740 1 -- 172.21.15.175:0/301492500 shutdown_connections 2024-07-29T00:44:07.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.506+0000 7fb85dc26740 1 -- 172.21.15.175:0/301492500 wait complete. 2024-07-29T00:44:07.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.506+0000 7fb85dc26740 1 -- start start 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.506+0000 7fb85dc26740 1 Processor -- start 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8c8910 0x55ad9d8d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541b0bf0 con 0x55ad9d8bf0e0 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541b0520 con 0x55ad9d8bdb70 2024-07-29T00:44:07.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541ad780 con 0x55ad9d8c8910 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb85dc26740 10 client.0 _ll_register_callbacks cb 0x7fb844005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb853e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb853e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8e0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55290/0 (socket says 172.21.15.175:55290) 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb853e00640 1 -- 172.21.15.175:0/1672925988 learned_addr learned my addr 172.21.15.175:0/1672925988 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb84b400640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.507+0000 7fb859000640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8c8910 0x55ad9d8d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb859000640 1 -- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bdb70 msgr2=0x55ad9d8e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb859000640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb859000640 1 -- 172.21.15.175:0/1672925988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 msgr2=0x55ad9d8d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb859000640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb859000640 1 -- 172.21.15.175:0/1672925988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9d8c2b20 con 0x55ad9d8c8910 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb84b400640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb859000640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8c8910 0x55ad9d8d1000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0cd2a0 tx=0x7fb84c09c130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba6d34dfc5dc267 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb842000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:07.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb842000640 1 -- 172.21.15.175:0/1672925988 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0a00e0 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb842000640 1 -- 172.21.15.175:0/1672925988 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7fb84c0986e0 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.508+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83c003d90 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb842000640 1 -- 172.21.15.175:0/1672925988 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb84c096be0 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 10 client.26917 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad9d8df6a0 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb842000640 1 -- 172.21.15.175:0/1672925988 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c003d90 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb842000640 1 -- 172.21.15.175:0/1672925988 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb84c0a4850 con 0x55ad9d8c8910 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 10 client.26917 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:07.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 20 client.26917 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8dfd80 0x55ad9d9453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 4 client.26917 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad9d82b530 con 0x55ad9d8dfd80 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.509+0000 7fb853e00640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8dfd80 0x55ad9d9453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb853e00640 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8dfd80 0x55ad9d9453e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbc2ffddaf61e6 server_cookie=27a65fb49911b3a5 in_seq=0 out_seq=0 2024-07-29T00:44:07.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb842000640 10 client.26917 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb842000640 1 -- 172.21.15.175:0/1672925988 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad9d82b530 con 0x55ad9d8dfd80 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb842000640 10 client.26917 handle_command_reply: tid=0 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 client.26917 shutdown 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8c8910 msgr2=0x55ad9d8d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8c8910 0x55ad9d8d1000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0cd2a0 tx=0x7fb84c09c130 comp rx=0 tx=0).stop 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 shutdown_connections 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ad9d8dfd80 0x55ad9d9453e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad9d8c8910 0x55ad9d8d1000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad9d8bf0e0 0x55ad9d8d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb85dc26740 1 --2- 172.21.15.175:0/1672925988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad9d8bdb70 0x55ad9d8e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.510+0000 7fb842000640 0 client.26917 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 >> 172.21.15.175:0/1672925988 conn(0x55ad9d886950 msgr2=0x55ad9d8c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 shutdown_connections 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb85dc26740 1 -- 172.21.15.175:0/1672925988 wait complete. 2024-07-29T00:44:07.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb85dc26740 20 client.26917 trim_cache size 0 max 16384 2024-07-29T00:44:07.519 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:07.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8380a2cf0 msgr2=0x7fb8380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb859a00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8380a2cf0 0x7fb8380a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb83c02a930 tx=0x7fb83c0058d0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8541076c0 msgr2=0x7fb85419ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.511+0000 7fb859a00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8541076c0 0x7fb85419ea10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009fd0 tx=0x7fb84c00ef10 comp rx=0 tx=0).stop 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 shutdown_connections 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8541076c0 0x7fb85419ea10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb854102060 0x7fb85419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb8380a2cf0 0x7fb8380a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 --2- 172.21.15.175:0/3244769674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8541016c0 0x7fb85419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:07.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 >> 172.21.15.175:0/3244769674 conn(0x7fb8540fd400 msgr2=0x7fb85410cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:07.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 shutdown_connections 2024-07-29T00:44:07.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:07.512+0000 7fb859a00640 1 -- 172.21.15.175:0/3244769674 wait complete. 2024-07-29T00:44:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:07 smithi181 ceph-mon[36706]: pgmap v234: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:08.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:07 smithi175 ceph-mon[32138]: pgmap v234: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:07 smithi204 ceph-mon[37368]: pgmap v234: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:09.492 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:44:09.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:44:09.960 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:09 smithi181 ceph-mon[36706]: pgmap v235: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:09 smithi175 ceph-mon[32138]: pgmap v235: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:10.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:09 smithi204 ceph-mon[37368]: pgmap v235: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:10.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 -- 172.21.15.175:0/3124606395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 msgr2=0x7f80ac103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 --2- 172.21.15.175:0/3124606395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac103d40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8098009930 tx=0x7f809802a420 comp rx=0 tx=0).stop 2024-07-29T00:44:10.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 -- 172.21.15.175:0/3124606395 shutdown_connections 2024-07-29T00:44:10.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 --2- 172.21.15.175:0/3124606395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80ac104280 0x7f80ac1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 --2- 172.21.15.175:0/3124606395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac103d40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 --2- 172.21.15.175:0/3124606395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ac1099f0 0x7f80ac109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 -- 172.21.15.175:0/3124606395 >> 172.21.15.175:0/3124606395 conn(0x7f80ac0ff5a0 msgr2=0x7f80ac1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 -- 172.21.15.175:0/3124606395 shutdown_connections 2024-07-29T00:44:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.626+0000 7f80b1e00640 1 -- 172.21.15.175:0/3124606395 wait complete. 2024-07-29T00:44:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 Processor -- start 2024-07-29T00:44:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 -- start start 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80ac104280 0x7f80ac19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ac1099f0 0x7f80ac19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac113e00 con 0x7f80ac1038e0 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac113c80 con 0x7f80ac1099f0 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.628+0000 7f80b1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac113f80 con 0x7f80ac104280 2024-07-29T00:44:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39218/0 (socket says 172.21.15.175:39218) 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 -- 172.21.15.175:0/532923297 learned_addr learned my addr 172.21.15.175:0/532923297 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80abe00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80ac104280 0x7f80ac19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b1400640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ac1099f0 0x7f80ac19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 -- 172.21.15.175:0/532923297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80ac104280 msgr2=0x7f80ac19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80ac104280 0x7f80ac19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 -- 172.21.15.175:0/532923297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ac1099f0 msgr2=0x7f80ac19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ac1099f0 0x7f80ac19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.629+0000 7f80b0a00640 1 -- 172.21.15.175:0/532923297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac19f2c0 con 0x7f80ac1038e0 2024-07-29T00:44:10.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.630+0000 7f80b0a00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac19a310 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ea60 tx=0x7f80a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:10.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.630+0000 7f80a9600640 1 -- 172.21.15.175:0/532923297 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a000cde0 con 0x7f80ac1038e0 2024-07-29T00:44:10.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.630+0000 7f80a9600640 1 -- 172.21.15.175:0/532923297 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a00035f0 con 0x7f80ac1038e0 2024-07-29T00:44:10.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.630+0000 7f80a9600640 1 -- 172.21.15.175:0/532923297 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0010670 con 0x7f80ac1038e0 2024-07-29T00:44:10.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.630+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ac19f5b0 con 0x7f80ac1038e0 2024-07-29T00:44:10.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.630+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ac1a6e90 con 0x7f80ac1038e0 2024-07-29T00:44:10.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.633+0000 7f80a9600640 1 -- 172.21.15.175:0/532923297 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f80a00037a0 con 0x7f80ac1038e0 2024-07-29T00:44:10.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 Processor -- start 2024-07-29T00:44:10.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 -- start start 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 0x55bd231db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bd231368a0 con 0x55bd231d0d20 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd230ce710 con 0x55bd231d1640 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.634+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bd23132860 con 0x55bd231dbf00 2024-07-29T00:44:10.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 0x55bd231db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 0x55bd231db9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55178/0 (socket says 172.21.15.175:55178) 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 -- 172.21.15.175:0/333556162 learned_addr learned my addr 172.21.15.175:0/333556162 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80abe00640 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 -- 172.21.15.175:0/333556162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 msgr2=0x55bd231de2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231de2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 -- 172.21.15.175:0/333556162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 msgr2=0x55bd231d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.635+0000 7f80b0a00640 1 -- 172.21.15.175:0/333556162 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd22ea86a0 con 0x55bd231d1640 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f80a9600640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f80940a2d70 0x7f80940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f80b0a00640 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 0x55bd231db9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80a00cccb0 tx=0x7f80a00d5ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa2f0fbf46bf000 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f80b1400640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f80940a2d70 0x7f80940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f80a9600640 1 -- 172.21.15.175:0/532923297 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7f80a00cf700 con 0x7f80ac1038e0 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f809dc00640 1 -- 172.21.15.175:0/333556162 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0097de0 con 0x55bd231d1640 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f809dc00640 1 -- 172.21.15.175:0/333556162 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a009c240 con 0x55bd231d1640 2024-07-29T00:44:10.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.636+0000 7f809dc00640 1 -- 172.21.15.175:0/333556162 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a009b4a0 con 0x55bd231d1640 2024-07-29T00:44:10.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 -- 172.21.15.175:0/333556162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 msgr2=0x55bd231db9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b1400640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f80940a2d70 0x7f80940a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005fd0 tx=0x7f80a40096a0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:10.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 0x55bd231db9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80a00cccb0 tx=0x7f80a00d5ef0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 -- 172.21.15.175:0/333556162 shutdown_connections 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231de2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d1640 0x55bd231db9c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/333556162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 -- 172.21.15.175:0/333556162 >> 172.21.15.175:0/333556162 conn(0x55bd231cd2b0 msgr2=0x55bd231cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 -- 172.21.15.175:0/333556162 shutdown_connections 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.637+0000 7f80b5e2e740 1 -- 172.21.15.175:0/333556162 wait complete. 2024-07-29T00:44:10.642 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.638+0000 7f80b5e2e740 1 -- start start 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.638+0000 7f80b5e2e740 1 Processor -- start 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 0x55bd231d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bd231dead0 con 0x55bd231d1640 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd230f18c0 con 0x55bd231d0d20 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bd231dec50 con 0x55bd231dbf00 2024-07-29T00:44:10.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b5e2e740 10 client.0 _ll_register_callbacks cb 0x55bd230dda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 0x55bd231d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 0x55bd231d9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39234/0 (socket says 172.21.15.175:39234) 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 -- 172.21.15.175:0/1459589210 learned_addr learned my addr 172.21.15.175:0/1459589210 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b1400640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 -- 172.21.15.175:0/1459589210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 msgr2=0x55bd231d9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231d9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 -- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 msgr2=0x55bd231d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80abe00640 1 -- 172.21.15.175:0/1459589210 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd230d4bc0 con 0x55bd231d1640 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.639+0000 7f80b1400640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.640+0000 7f80abe00640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 0x55bd231d9310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8098033000 tx=0x7f8098004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81533b00f98d82 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.640+0000 7f8092000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.640+0000 7f8092000640 1 -- 172.21.15.175:0/1459589210 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098035040 con 0x55bd231d1640 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.640+0000 7f8092000640 1 -- 172.21.15.175:0/1459589210 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7f8098005450 con 0x55bd231d1640 2024-07-29T00:44:10.645 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.640+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a009c240 con 0x55bd231d1640 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f8092000640 1 -- 172.21.15.175:0/1459589210 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8098005650 con 0x55bd231d1640 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 10 client.16830 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bd231eaaf0 con 0x55bd231d1640 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f8092000640 1 -- 172.21.15.175:0/1459589210 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a009c240 con 0x55bd231d1640 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f8092000640 1 -- 172.21.15.175:0/1459589210 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f809804c070 con 0x55bd231d1640 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 10 client.16830 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 20 client.16830 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:10.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd231ebfb0 0x55bd231ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 4 client.16830 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:10.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.641+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd230f6c90 con 0x55bd231ebfb0 2024-07-29T00:44:10.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.642+0000 7f80b0a00640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd231ebfb0 0x55bd231ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.642+0000 7f80b0a00640 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd231ebfb0 0x55bd231ee3f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed0365854015d1e server_cookie=bd094003e0715c40 in_seq=0 out_seq=0 2024-07-29T00:44:10.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.642+0000 7f8092000640 10 client.16830 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:10.648 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f8092000640 1 -- 172.21.15.175:0/1459589210 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bd230f6c90 con 0x55bd231ebfb0 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f8092000640 10 client.16830 handle_command_reply: tid=0 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 client.16830 shutdown 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 msgr2=0x55bd231d9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 0x55bd231d9310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8098033000 tx=0x7f8098004440 comp rx=0 tx=0).stop 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 shutdown_connections 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd231ebfb0 0x55bd231ee3f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231dbf00 0x55bd231d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d1640 0x55bd231d9310 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/1459589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.643+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 >> 172.21.15.175:0/1459589210 conn(0x55bd231cd2b0 msgr2=0x55bd231d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:10.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.644+0000 7f8092000640 0 client.16830 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:10.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.644+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 shutdown_connections 2024-07-29T00:44:10.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.644+0000 7f80b5e2e740 1 -- 172.21.15.175:0/1459589210 wait complete. 2024-07-29T00:44:10.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.644+0000 7f80b5e2e740 20 client.16830 trim_cache size 0 max 16384 2024-07-29T00:44:10.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 Processor -- start 2024-07-29T00:44:10.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 -- start start 2024-07-29T00:44:10.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231dbf00 0x55bd231d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac1af870 con 0x55bd231d0d20 2024-07-29T00:44:10.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac1b13b0 con 0x55bd231dbf00 2024-07-29T00:44:10.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.663+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac1aff40 con 0x55bd231d1640 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80b1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231dbf00 0x55bd231d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80b1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231dbf00 0x55bd231d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55196/0 (socket says 172.21.15.175:55196) 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80b1400640 1 -- 172.21.15.175:0/3781315165 learned_addr learned my addr 172.21.15.175:0/3781315165 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80b0a00640 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 -- 172.21.15.175:0/3781315165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231dbf00 msgr2=0x55bd231d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231dbf00 0x55bd231d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 -- 172.21.15.175:0/3781315165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 msgr2=0x55bd231d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 -- 172.21.15.175:0/3781315165 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd231cd0e0 con 0x55bd231d1640 2024-07-29T00:44:10.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f80abe00640 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231d96a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8098031ae0 tx=0x7f8098001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d3996a96ac3053 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:10.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.664+0000 7f809d200640 1 -- 172.21.15.175:0/3781315165 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098052020 con 0x55bd231d1640 2024-07-29T00:44:10.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f809d200640 1 -- 172.21.15.175:0/3781315165 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8098041070 con 0x55bd231d1640 2024-07-29T00:44:10.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f809d200640 1 -- 172.21.15.175:0/3781315165 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098035070 con 0x55bd231d1640 2024-07-29T00:44:10.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3781315165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 msgr2=0x55bd231d96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231d96a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8098031ae0 tx=0x7f8098001e70 comp rx=0 tx=0).stop 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3781315165 shutdown_connections 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231dbf00 0x55bd231d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231d96a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3781315165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231d0d20 0x55bd231d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.665+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3781315165 >> 172.21.15.175:0/3781315165 conn(0x55bd231cd2b0 msgr2=0x55bd231d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.666+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3781315165 shutdown_connections 2024-07-29T00:44:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.666+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3781315165 wait complete. 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.666+0000 7f80b5e2e740 1 -- start start 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.666+0000 7f80b5e2e740 1 Processor -- start 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231e4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231dbf00 0x55bd231df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac1b0ce0 con 0x55bd231dbf00 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac1b0610 con 0x55bd231d0d20 2024-07-29T00:44:10.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac1ad870 con 0x55bd231d1640 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b5e2e740 10 client.0 _ll_register_callbacks cb 0x7f80980040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231e41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55200/0 (socket says 172.21.15.175:55200) 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 -- 172.21.15.175:0/3180648939 learned_addr learned my addr 172.21.15.175:0/3180648939 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b1400640 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231dbf00 0x55bd231df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 -- 172.21.15.175:0/3180648939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 msgr2=0x55bd231e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231e4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 -- 172.21.15.175:0/3180648939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231dbf00 msgr2=0x55bd231df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231dbf00 0x55bd231df5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.667+0000 7f80b0a00640 1 -- 172.21.15.175:0/3180648939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd231dfe20 con 0x55bd231d0d20 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f80b0a00640 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231e41f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a0005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166201a01f29b0dc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f8092000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f8092000640 1 -- 172.21.15.175:0/3180648939 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a009b680 con 0x55bd231d0d20 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f8092000640 1 -- 172.21.15.175:0/3180648939 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8644+0+0 (secure 0 0 0) 0x7f80a00d5300 con 0x55bd231d0d20 2024-07-29T00:44:10.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8098041070 con 0x55bd231d0d20 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f8092000640 1 -- 172.21.15.175:0/3180648939 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f80a009b910 con 0x55bd231d0d20 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f80b5e2e740 10 client.26984 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bd232568d0 con 0x55bd231d0d20 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.668+0000 7f8092000640 1 -- 172.21.15.175:0/3180648939 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8098041070 con 0x55bd231d0d20 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f8092000640 1 -- 172.21.15.175:0/3180648939 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f80a00d2ce0 con 0x55bd231d0d20 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80b5e2e740 10 client.26984 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80b5e2e740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80b5e2e740 20 client.26984 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd23256c10 0x55bd23259000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80b5e2e740 4 client.26984 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:44:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd23143e10 con 0x55bd23256c10 2024-07-29T00:44:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.669+0000 7f80abe00640 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd23256c10 0x55bd23259000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:10.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.670+0000 7f80abe00640 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd23256c10 0x55bd23259000 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3e6e5a5a879abb server_cookie=c86ad9471c158197 in_seq=0 out_seq=0 2024-07-29T00:44:10.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.670+0000 7f8092000640 10 client.26984 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:10.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f8092000640 1 -- 172.21.15.175:0/3180648939 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd23143e10 con 0x55bd23256c10 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f8092000640 10 client.26984 handle_command_reply: tid=0 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 client.26984 shutdown 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 msgr2=0x55bd231e41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231e41f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a0005f10 comp rx=0 tx=0).stop 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 shutdown_connections 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bd23256c10 0x55bd23259000 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd231dbf00 0x55bd231df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd231d1640 0x55bd231e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 --2- 172.21.15.175:0/3180648939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd231d0d20 0x55bd231e41f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f8092000640 0 client.26984 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 >> 172.21.15.175:0/3180648939 conn(0x55bd231cd2b0 msgr2=0x55bd231cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 shutdown_connections 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.671+0000 7f80b5e2e740 1 -- 172.21.15.175:0/3180648939 wait complete. 2024-07-29T00:44:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b5e2e740 20 client.26984 trim_cache size 0 max 16384 2024-07-29T00:44:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:44:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:44:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "266194c6-dbbe-4baf-a054-c63725e6a757", 2024-07-29T00:44:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:44:10.679 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:44:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f80940a2d70 msgr2=0x7f80940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f80940a2d70 0x7f80940a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005fd0 tx=0x7f80a40096a0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 msgr2=0x7f80ac19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac19a310 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ea60 tx=0x7f80a000ef20 comp rx=0 tx=0).stop 2024-07-29T00:44:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 shutdown_connections 2024-07-29T00:44:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ac1099f0 0x7f80ac19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80ac104280 0x7f80ac19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f80940a2d70 0x7f80940a5230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 --2- 172.21.15.175:0/532923297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ac1038e0 0x7f80ac19a310 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.672+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 >> 172.21.15.175:0/532923297 conn(0x7f80ac0ff5a0 msgr2=0x7f80ac0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.673+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 shutdown_connections 2024-07-29T00:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:10.673+0000 7f80b1e00640 1 -- 172.21.15.175:0/532923297 wait complete. 2024-07-29T00:44:11.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:44:11.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: pgmap v236: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:11.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:44:11.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.785 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.786 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: osdmap e96: 12 total, 12 up, 12 in 2024-07-29T00:44:11.787 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.788 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:11 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:11.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: pgmap v236: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.001 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: osdmap e96: 12 total, 12 up, 12 in 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.002 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:11 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.049 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: pgmap v236: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-07-29T00:44:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:44:12.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: osdmap e96: 12 total, 12 up, 12 in 2024-07-29T00:44:12.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:11 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:44:12.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 -- 172.21.15.175:0/3742296957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e4101050 msgr2=0x7f39e410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 --2- 172.21.15.175:0/3742296957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e4101050 0x7f39e410dc80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009900 tx=0x7f39d002a410 comp rx=0 tx=0).stop 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 -- 172.21.15.175:0/3742296957 shutdown_connections 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 --2- 172.21.15.175:0/3742296957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e410e1c0 0x7f39e41105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 --2- 172.21.15.175:0/3742296957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e4101050 0x7f39e410dc80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 --2- 172.21.15.175:0/3742296957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41006a0 0x7f39e4100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 -- 172.21.15.175:0/3742296957 >> 172.21.15.175:0/3742296957 conn(0x7f39e40fc4c0 msgr2=0x7f39e40fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 -- 172.21.15.175:0/3742296957 shutdown_connections 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.769+0000 7f39ea600640 1 -- 172.21.15.175:0/3742296957 wait complete. 2024-07-29T00:44:12.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.770+0000 7f39ea600640 1 Processor -- start 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 -- start start 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41006a0 0x7f39e419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 0x7f39e419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e410e1c0 0x7f39e419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4113f80 con 0x7f39e4101050 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4113e00 con 0x7f39e410e1c0 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39ea600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4114100 con 0x7f39e41006a0 2024-07-29T00:44:12.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 0x7f39e419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 0x7f39e419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39270/0 (socket says 172.21.15.175:39270) 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39e2a00640 1 -- 172.21.15.175:0/3980988548 learned_addr learned my addr 172.21.15.175:0/3980988548 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39e3400640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41006a0 0x7f39e419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.771+0000 7f39e3e00640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e410e1c0 0x7f39e419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e3e00640 1 -- 172.21.15.175:0/3980988548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41006a0 msgr2=0x7f39e419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e3e00640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41006a0 0x7f39e419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e3e00640 1 -- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 msgr2=0x7f39e419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e3e00640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 0x7f39e419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e3e00640 1 -- 172.21.15.175:0/3980988548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e419f3d0 con 0x7f39e410e1c0 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e2a00640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 0x7f39e419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39e3e00640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e410e1c0 0x7f39e419ecf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009fd0 tx=0x7f39dc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.772+0000 7f39dbe00640 1 -- 172.21.15.175:0/3980988548 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00ce20 con 0x7f39e410e1c0 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.773+0000 7f39dbe00640 1 -- 172.21.15.175:0/3980988548 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39dc005020 con 0x7f39e410e1c0 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.773+0000 7f39dbe00640 1 -- 172.21.15.175:0/3980988548 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc010720 con 0x7f39e410e1c0 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.773+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e419f6c0 con 0x7f39e410e1c0 2024-07-29T00:44:12.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.773+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e41a6fa0 con 0x7f39e410e1c0 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.776+0000 7f39ed237740 1 Processor -- start 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.776+0000 7f39dbe00640 1 -- 172.21.15.175:0/3980988548 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f39dc005540 con 0x7f39e410e1c0 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.776+0000 7f39ed237740 1 -- start start 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39ed237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dc9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39ed237740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39ed237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dd6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39ed237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555881d2ead0 con 0x555881dc9930 2024-07-29T00:44:12.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39ed237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555881cc6af0 con 0x555881dd4b10 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39ed237740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555881d2aa90 con 0x555881dca250 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dc9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dc9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39272/0 (socket says 172.21.15.175:39272) 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 -- 172.21.15.175:0/4048556033 learned_addr learned my addr 172.21.15.175:0/4048556033 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e3400640 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dd6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 -- 172.21.15.175:0/4048556033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 msgr2=0x555881dd45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 -- 172.21.15.175:0/4048556033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 msgr2=0x555881dd6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.777+0000 7f39e2a00640 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dd6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39e2a00640 1 -- 172.21.15.175:0/4048556033 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555881aa03c0 con 0x555881dc9930 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39dbe00640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c80a2cf0 0x7f39c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39dbe00640 1 -- 172.21.15.175:0/3980988548 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f39dc0cfb80 con 0x7f39e410e1c0 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39e2a00640 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dc9d10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f39d0031520 tx=0x7f39d00314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c82be0d052e423 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39e3400640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c80a2cf0 0x7f39c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39d8c00640 1 -- 172.21.15.175:0/4048556033 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0035040 con 0x555881dc9930 2024-07-29T00:44:12.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39d8c00640 1 -- 172.21.15.175:0/4048556033 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d004adc0 con 0x555881dc9930 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.778+0000 7f39d8c00640 1 -- 172.21.15.175:0/4048556033 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0003aa0 con 0x555881dc9930 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 -- 172.21.15.175:0/4048556033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 msgr2=0x555881dc9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dc9d10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f39d0031520 tx=0x7f39d00314b0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39e3400640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c80a2cf0 0x7f39c80a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x555881dc6d20 tx=0x7f39d400b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 -- 172.21.15.175:0/4048556033 shutdown_connections 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dd6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 --2- 172.21.15.175:0/4048556033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dc9d10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 -- 172.21.15.175:0/4048556033 >> 172.21.15.175:0/4048556033 conn(0x555881dc5ec0 msgr2=0x555881dc62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 -- 172.21.15.175:0/4048556033 shutdown_connections 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.779+0000 7f39ed237740 1 -- 172.21.15.175:0/4048556033 wait complete. 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.780+0000 7f39ed237740 1 -- start start 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.780+0000 7f39ed237740 1 Processor -- start 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.780+0000 7f39ed237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dc9930 0x555881dd1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39ed237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 0x555881dd1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39ed237740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881dd23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39ed237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555881dd76e0 con 0x555881dca250 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39ed237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555881ce9bc0 con 0x555881dc9930 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39ed237740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555881dd7860 con 0x555881dd4b10 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39ed237740 10 client.0 _ll_register_callbacks cb 0x555881cd5d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 0x555881dd1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 0x555881dd1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39284/0 (socket says 172.21.15.175:39284) 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881dd23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 -- 172.21.15.175:0/870870642 learned_addr learned my addr 172.21.15.175:0/870870642 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e3400640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dc9930 0x555881dd1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 -- 172.21.15.175:0/870870642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 msgr2=0x555881dd23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881dd23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 -- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dc9930 msgr2=0x555881dd1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dc9930 0x555881dd1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e2a00640 1 -- 172.21.15.175:0/870870642 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555881de0b50 con 0x555881dca250 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.781+0000 7f39e3e00640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881dd23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:12.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39e2a00640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 0x555881dd1e90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f39d0005e80 tx=0x7f39d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d580ffd2bec9d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39cd600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39cd600640 1 -- 172.21.15.175:0/870870642 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0058060 con 0x555881dca250 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39cd600640 1 -- 172.21.15.175:0/870870642 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f39d004a8e0 con 0x555881dca250 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d004adc0 con 0x555881dca250 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39cd600640 1 -- 172.21.15.175:0/870870642 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f39d0035040 con 0x555881dca250 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39ed237740 10 client.16848 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555881de34b0 con 0x555881dca250 2024-07-29T00:44:12.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.782+0000 7f39cd600640 1 -- 172.21.15.175:0/870870642 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d004adc0 con 0x555881dca250 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39cd600640 1 -- 172.21.15.175:0/870870642 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f39d0004a40 con 0x555881dca250 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39ed237740 10 client.16848 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39ed237740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39ed237740 20 client.16848 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39ed237740 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881de4800 0x555881de6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39ed237740 4 client.16848 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555881d3c040 con 0x555881de4800 2024-07-29T00:44:12.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.783+0000 7f39e3400640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881de4800 0x555881de6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.784+0000 7f39e3400640 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881de4800 0x555881de6bf0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a51dd3bffe10a96 server_cookie=9954fbc74f6a46fe in_seq=0 out_seq=0 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.784+0000 7f39cd600640 10 client.16848 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39cd600640 1 -- 172.21.15.175:0/870870642 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555881d3c040 con 0x555881de4800 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39cd600640 10 client.16848 handle_command_reply: tid=0 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 client.16848 shutdown 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 msgr2=0x555881dd1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 0x555881dd1e90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f39d0005e80 tx=0x7f39d00046c0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.801 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 shutdown_connections 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881de4800 0x555881de6bf0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881dd23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dca250 0x555881dd1e90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 --2- 172.21.15.175:0/870870642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dc9930 0x555881dd1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 >> 172.21.15.175:0/870870642 conn(0x555881dc5ec0 msgr2=0x555881dcbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39cd600640 0 client.16848 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 shutdown_connections 2024-07-29T00:44:12.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.785+0000 7f39ed237740 1 -- 172.21.15.175:0/870870642 wait complete. 2024-07-29T00:44:12.803 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.786+0000 7f39ed237740 20 client.16848 trim_cache size 0 max 16384 2024-07-29T00:44:12.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.804+0000 7f39ed237740 1 Processor -- start 2024-07-29T00:44:12.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.804+0000 7f39ed237740 1 -- start start 2024-07-29T00:44:12.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39ed237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dcc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39ed237740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39ed237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dcebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39ed237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41ab3a0 con 0x555881dc9930 2024-07-29T00:44:12.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39ed237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41acee0 con 0x555881dd4b10 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39ed237740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41aba70 con 0x555881dca250 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39e3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dcc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39e3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dcc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39286/0 (socket says 172.21.15.175:39286) 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39e3400640 1 -- 172.21.15.175:0/3231905993 learned_addr learned my addr 172.21.15.175:0/3231905993 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39e2a00640 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.805+0000 7f39e3e00640 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dcebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39e3400640 1 -- 172.21.15.175:0/3231905993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 msgr2=0x555881dd2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39e3400640 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39e3400640 1 -- 172.21.15.175:0/3231905993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 msgr2=0x555881dcebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39e3400640 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dcebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39e3400640 1 -- 172.21.15.175:0/3231905993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555881dca080 con 0x555881dc9930 2024-07-29T00:44:12.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39e3400640 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dcc390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f39d401e5b0 tx=0x7f39d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919663080a281931 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:12.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39cfe00640 1 -- 172.21.15.175:0/3231905993 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d402a420 con 0x555881dc9930 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.806+0000 7f39cfe00640 1 -- 172.21.15.175:0/3231905993 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d402aaa0 con 0x555881dc9930 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39cfe00640 1 -- 172.21.15.175:0/3231905993 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d401dc70 con 0x555881dc9930 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 -- 172.21.15.175:0/3231905993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 msgr2=0x555881dcc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dcc390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f39d401e5b0 tx=0x7f39d4019000 comp rx=0 tx=0).stop 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 -- 172.21.15.175:0/3231905993 shutdown_connections 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dd4b10 0x555881dcebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dca250 0x555881dd2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 --2- 172.21.15.175:0/3231905993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881dc9930 0x555881dcc390 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 -- 172.21.15.175:0/3231905993 >> 172.21.15.175:0/3231905993 conn(0x555881dc5ec0 msgr2=0x555881dd0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.807+0000 7f39ed237740 1 -- 172.21.15.175:0/3231905993 shutdown_connections 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.808+0000 7f39ed237740 1 -- 172.21.15.175:0/3231905993 wait complete. 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.808+0000 7f39ed237740 1 -- start start 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.808+0000 7f39ed237740 1 Processor -- start 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dca250 0x555881de9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881de6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881de7470 0x555881de78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41ac810 con 0x555881de7470 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41ac140 con 0x555881dca250 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41a93a0 con 0x555881dd4b10 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39ed237740 10 client.0 _ll_register_callbacks cb 0x7f39d0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39e3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881de7470 0x555881de78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881de6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39e3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dca250 0x555881de9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39e3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881de7470 0x555881de78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39300/0 (socket says 172.21.15.175:39300) 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881de6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:38680/0 (socket says 172.21.15.175:38680) 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.809+0000 7f39e3e00640 1 -- 172.21.15.175:0/3763830966 learned_addr learned my addr 172.21.15.175:0/3763830966 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39e2a00640 1 -- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dca250 msgr2=0x555881de9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39e2a00640 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dca250 0x555881de9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39e2a00640 1 -- 172.21.15.175:0/3763830966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881de7470 msgr2=0x555881de78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:12.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39e2a00640 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881de7470 0x555881de78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39e2a00640 1 -- 172.21.15.175:0/3763830966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555881dea4d0 con 0x555881dd4b10 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39e2a00640 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881de6f30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f39d005d040 tx=0x7f39d00054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91ddcc5aa9d9c40 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39cd600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39cd600640 1 -- 172.21.15.175:0/3763830966 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0058070 con 0x555881dd4b10 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39cd600640 1 -- 172.21.15.175:0/3763830966 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f39d0004fb0 con 0x555881dd4b10 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.810+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d402aaa0 con 0x555881dd4b10 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39cd600640 1 -- 172.21.15.175:0/3763830966 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f39d006a050 con 0x555881dd4b10 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 10 client.26953 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:12.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555881e4f220 con 0x555881dd4b10 2024-07-29T00:44:12.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39cd600640 1 -- 172.21.15.175:0/3763830966 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d402aaa0 con 0x555881dd4b10 2024-07-29T00:44:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39cd600640 1 -- 172.21.15.175:0/3763830966 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f39d0062020 con 0x555881dd4b10 2024-07-29T00:44:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 10 client.26953 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 20 client.26953 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881e4f880 0x555881e51ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 4 client.26953 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.811+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555881d36550 con 0x555881e4f880 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.812+0000 7f39e3400640 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881e4f880 0x555881e51ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.812+0000 7f39e3400640 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881e4f880 0x555881e51ca0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2566e2c8760069b server_cookie=a42e96754c78126b in_seq=0 out_seq=0 2024-07-29T00:44:12.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.812+0000 7f39cd600640 10 client.26953 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.812+0000 7f39cd600640 1 -- 172.21.15.175:0/3763830966 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555881d36550 con 0x555881e4f880 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.812+0000 7f39cd600640 10 client.26953 handle_command_reply: tid=0 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.812+0000 7f39ed237740 1 client.26953 shutdown 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 msgr2=0x555881de6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881de6f30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f39d005d040 tx=0x7f39d00054f0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 shutdown_connections 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555881e4f880 0x555881e51ca0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555881de7470 0x555881de78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555881dd4b10 0x555881de6f30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 --2- 172.21.15.175:0/3763830966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555881dca250 0x555881de9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39cd600640 0 client.26953 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:12.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 >> 172.21.15.175:0/3763830966 conn(0x555881dc5ec0 msgr2=0x555881dc46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:12.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 shutdown_connections 2024-07-29T00:44:12.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 1 -- 172.21.15.175:0/3763830966 wait complete. 2024-07-29T00:44:12.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.813+0000 7f39ed237740 20 client.26953 trim_cache size 0 max 16384 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout: "266194c6-dbbe-4baf-a054-c63725e6a757": { 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "266194c6-dbbe-4baf-a054-c63725e6a757", 2024-07-29T00:44:12.821 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:44:12.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:44:12.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:44:12.822 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:44:12.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c80a2cf0 msgr2=0x7f39c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c80a2cf0 0x7f39c80a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x555881dc6d20 tx=0x7f39d400b410 comp rx=0 tx=0).stop 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e410e1c0 msgr2=0x7f39e419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e410e1c0 0x7f39e419ecf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009fd0 tx=0x7f39dc00ef10 comp rx=0 tx=0).stop 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 shutdown_connections 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e410e1c0 0x7f39e419ecf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39c80a2cf0 0x7f39c80a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e4101050 0x7f39e419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 --2- 172.21.15.175:0/3980988548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e41006a0 0x7f39e419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:12.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 >> 172.21.15.175:0/3980988548 conn(0x7f39e40fc4c0 msgr2=0x7f39e40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:12.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 shutdown_connections 2024-07-29T00:44:12.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:12.814+0000 7f39ea600640 1 -- 172.21.15.175:0/3980988548 wait complete. 2024-07-29T00:44:13.674 INFO:tasks.cephfs.filesystem:scrub status for tag:266194c6-dbbe-4baf-a054-c63725e6a757 - {'path': '/', 'tag': '266194c6-dbbe-4baf-a054-c63725e6a757', 'options': 'recursive,force'} 2024-07-29T00:44:13.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:13 smithi181 ceph-mon[36706]: pgmap v238: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-07-29T00:44:14.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:13 smithi175 ceph-mon[32138]: pgmap v238: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-07-29T00:44:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:13 smithi204 ceph-mon[37368]: pgmap v238: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-07-29T00:44:15.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:15 smithi181 ceph-mon[36706]: pgmap v239: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-07-29T00:44:16.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:15 smithi175 ceph-mon[32138]: pgmap v239: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-07-29T00:44:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:15 smithi204 ceph-mon[37368]: pgmap v239: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-07-29T00:44:17.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:17 smithi181 ceph-mon[36706]: pgmap v240: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:17.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:17 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:44:18.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:17 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:44:18.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:17 smithi175 ceph-mon[32138]: pgmap v240: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:18.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:17 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:44:18.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:17 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:44:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:17 smithi204 ceph-mon[37368]: pgmap v240: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:17 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:44:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:17 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:44:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:18 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:44:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:18 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:44:18.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:18 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:44:19.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:18 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:44:19.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:18 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:44:19.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:18 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:44:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:18 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:44:19.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:18 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:44:19.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:18 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:44:19.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:19 smithi181 ceph-mon[36706]: pgmap v241: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:20.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:19 smithi175 ceph-mon[32138]: pgmap v241: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:19 smithi204 ceph-mon[37368]: pgmap v241: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:21.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:21 smithi181 ceph-mon[36706]: pgmap v242: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:22.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:21 smithi175 ceph-mon[32138]: pgmap v242: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:21 smithi204 ceph-mon[37368]: pgmap v242: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-07-29T00:44:23.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:23 smithi181 ceph-mon[36706]: pgmap v243: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-07-29T00:44:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:23 smithi175 ceph-mon[32138]: pgmap v243: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-07-29T00:44:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:23 smithi204 ceph-mon[37368]: pgmap v243: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-07-29T00:44:25.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:25 smithi181 ceph-mon[36706]: pgmap v244: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-07-29T00:44:26.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:25 smithi175 ceph-mon[32138]: pgmap v244: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-07-29T00:44:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:25 smithi204 ceph-mon[37368]: pgmap v244: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-07-29T00:44:27.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:27 smithi181 ceph-mon[36706]: pgmap v245: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-07-29T00:44:28.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:27 smithi175 ceph-mon[32138]: pgmap v245: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-07-29T00:44:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:27 smithi204 ceph-mon[37368]: pgmap v245: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-07-29T00:44:29.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:29 smithi181 ceph-mon[36706]: pgmap v246: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:29 smithi175 ceph-mon[32138]: pgmap v246: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:29 smithi204 ceph-mon[37368]: pgmap v246: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:31.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:31 smithi181 ceph-mon[36706]: pgmap v247: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:32.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:31 smithi175 ceph-mon[32138]: pgmap v247: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:31 smithi204 ceph-mon[37368]: pgmap v247: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:34.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:33 smithi175 ceph-mon[32138]: pgmap v248: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:34.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:33 smithi204 ceph-mon[37368]: pgmap v248: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:34.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:33 smithi181 ceph-mon[36706]: pgmap v248: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:36.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:35 smithi175 ceph-mon[32138]: pgmap v249: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:35 smithi204 ceph-mon[37368]: pgmap v249: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:36.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:35 smithi181 ceph-mon[36706]: pgmap v249: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:38.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:37 smithi175 ceph-mon[32138]: pgmap v250: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:38.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:37 smithi204 ceph-mon[37368]: pgmap v250: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:37 smithi181 ceph-mon[36706]: pgmap v250: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:40.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:39 smithi175 ceph-mon[32138]: pgmap v251: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:40.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:39 smithi204 ceph-mon[37368]: pgmap v251: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:39 smithi181 ceph-mon[36706]: pgmap v251: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:42.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:41 smithi175 ceph-mon[32138]: pgmap v252: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:42.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:41 smithi204 ceph-mon[37368]: pgmap v252: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:41 smithi181 ceph-mon[36706]: pgmap v252: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:43.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:44:44.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:43 smithi175 ceph-mon[32138]: pgmap v253: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:44.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:43 smithi204 ceph-mon[37368]: pgmap v253: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:44.135 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:43 smithi181 ceph-mon[36706]: pgmap v253: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:44.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.823+0000 7ff622a00640 1 -- 172.21.15.175:0/1293476258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c1076a0 msgr2=0x7ff61c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.823+0000 7ff622a00640 1 --2- 172.21.15.175:0/1293476258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c1076a0 0x7ff61c107a80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff610009930 tx=0x7ff61002a440 comp rx=0 tx=0).stop 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 -- 172.21.15.175:0/1293476258 shutdown_connections 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 --2- 172.21.15.175:0/1293476258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c102040 0x7ff61c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 --2- 172.21.15.175:0/1293476258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1016a0 0x7ff61c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 --2- 172.21.15.175:0/1293476258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c1076a0 0x7ff61c107a80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 -- 172.21.15.175:0/1293476258 >> 172.21.15.175:0/1293476258 conn(0x7ff61c0fd3a0 msgr2=0x7ff61c0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 -- 172.21.15.175:0/1293476258 shutdown_connections 2024-07-29T00:44:44.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.824+0000 7ff622a00640 1 -- 172.21.15.175:0/1293476258 wait complete. 2024-07-29T00:44:44.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.825+0000 7ff622a00640 1 Processor -- start 2024-07-29T00:44:44.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.825+0000 7ff622a00640 1 -- start start 2024-07-29T00:44:44.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 0x7ff61c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 0x7ff61c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1076a0 0x7ff61c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111aa0 con 0x7ff61c102040 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111920 con 0x7ff61c1076a0 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111c20 con 0x7ff61c1016a0 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff620c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 0x7ff61c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff621600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 0x7ff61c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff621600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 0x7ff61c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:56910/0 (socket says 172.21.15.175:56910) 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff620c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 0x7ff61c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41788/0 (socket says 172.21.15.175:41788) 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff621600640 1 -- 172.21.15.175:0/261268670 learned_addr learned my addr 172.21.15.175:0/261268670 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:44.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.826+0000 7ff622000640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1076a0 0x7ff61c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff620c00640 1 -- 172.21.15.175:0/261268670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 msgr2=0x7ff61c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff620c00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 0x7ff61c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff620c00640 1 -- 172.21.15.175:0/261268670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1076a0 msgr2=0x7ff61c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff620c00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1076a0 0x7ff61c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff620c00640 1 -- 172.21.15.175:0/261268670 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c19f150 con 0x7ff61c102040 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff621600640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 0x7ff61c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:44.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff620c00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 0x7ff61c19a6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff60800e9f0 tx=0x7ff60800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:44.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff61a000640 1 -- 172.21.15.175:0/261268670 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60800ccc0 con 0x7ff61c102040 2024-07-29T00:44:44.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.828+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c19f440 con 0x7ff61c102040 2024-07-29T00:44:44.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.827+0000 7ff61a000640 1 -- 172.21.15.175:0/261268670 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff608010910 con 0x7ff61c102040 2024-07-29T00:44:44.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.828+0000 7ff61a000640 1 -- 172.21.15.175:0/261268670 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608018960 con 0x7ff61c102040 2024-07-29T00:44:44.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.828+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a6d20 con 0x7ff61c102040 2024-07-29T00:44:44.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 Processor -- start 2024-07-29T00:44:44.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 -- start start 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff61a000640 1 -- 172.21.15.175:0/261268670 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff608010ac0 con 0x7ff61c102040 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b49ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f33aaead0 con 0x561f33b49b10 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561f33a46af0 con 0x561f33b4a430 2024-07-29T00:44:44.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.831+0000 7ff626c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561f33aaaa90 con 0x561f33b54cf0 2024-07-29T00:44:44.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff622000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b49ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39278/0 (socket says 172.21.15.175:39278) 2024-07-29T00:44:44.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 -- 172.21.15.175:0/3962081795 learned_addr learned my addr 172.21.15.175:0/3962081795 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:44.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff620c00640 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 -- 172.21.15.175:0/3962081795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 msgr2=0x561f33b570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 -- 172.21.15.175:0/3962081795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 msgr2=0x561f33b49ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b49ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.832+0000 7ff621600640 1 -- 172.21.15.175:0/3962081795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f338203c0 con 0x561f33b4a430 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff620c00640 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff622000640 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:44.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff621600640 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b547b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff610006010 tx=0x7ff610005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ff4b6f55685185 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:44.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff60e600640 1 -- 172.21.15.175:0/3962081795 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610035040 con 0x561f33b4a430 2024-07-29T00:44:44.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff61a000640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff6040a2d70 0x7ff6040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff60e600640 1 -- 172.21.15.175:0/3962081795 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff610031830 con 0x561f33b4a430 2024-07-29T00:44:44.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff60e600640 1 -- 172.21.15.175:0/3962081795 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610004a00 con 0x561f33b4a430 2024-07-29T00:44:44.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.833+0000 7ff61a000640 1 -- 172.21.15.175:0/261268670 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7ff6080cf7f0 con 0x7ff61c102040 2024-07-29T00:44:44.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff622000640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff6040a2d70 0x7ff6040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 -- 172.21.15.175:0/3962081795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 msgr2=0x561f33b547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b547b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff610006010 tx=0x7ff610005dc0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 -- 172.21.15.175:0/3962081795 shutdown_connections 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b547b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 --2- 172.21.15.175:0/3962081795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff626c35740 1 -- 172.21.15.175:0/3962081795 >> 172.21.15.175:0/3962081795 conn(0x561f33b460a0 msgr2=0x561f33b46480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.834+0000 7ff622000640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff6040a2d70 0x7ff6040a5230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff614009fd0 tx=0x7ff614009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.835+0000 7ff626c35740 1 -- 172.21.15.175:0/3962081795 shutdown_connections 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.835+0000 7ff626c35740 1 -- 172.21.15.175:0/3962081795 wait complete. 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.835+0000 7ff626c35740 1 -- start start 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.835+0000 7ff626c35740 1 Processor -- start 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b51bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f33b578c0 con 0x561f33b49b10 2024-07-29T00:44:44.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561f33a69bc0 con 0x561f33b4a430 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561f33b57a40 con 0x561f33b54cf0 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff626c35740 10 client.0 _ll_register_callbacks cb 0x561f33a55d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff621600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b51bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff620c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff621600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b51bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41808/0 (socket says 172.21.15.175:41808) 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff621600640 1 -- 172.21.15.175:0/1914786407 learned_addr learned my addr 172.21.15.175:0/1914786407 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.836+0000 7ff622000640 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff620c00640 1 -- 172.21.15.175:0/1914786407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 msgr2=0x561f33b52650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff620c00640 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b52650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff620c00640 1 -- 172.21.15.175:0/1914786407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 msgr2=0x561f33b51bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff620c00640 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b51bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff620c00640 1 -- 172.21.15.175:0/1914786407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f33b60db0 con 0x561f33b4a430 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff620c00640 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b52110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cce90 tx=0x7ff60809b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a978fd8f66bc7502 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff602a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff602a00640 1 -- 172.21.15.175:0/1914786407 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080cd320 con 0x561f33b4a430 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff610031830 con 0x561f33b4a430 2024-07-29T00:44:44.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.837+0000 7ff602a00640 1 -- 172.21.15.175:0/1914786407 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7ff6080cfde0 con 0x561f33b4a430 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff602a00640 1 -- 172.21.15.175:0/1914786407 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff6080b4e60 con 0x561f33b4a430 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 10 client.27014 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561f33b63700 con 0x561f33b4a430 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff602a00640 1 -- 172.21.15.175:0/1914786407 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff610031830 con 0x561f33b4a430 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff602a00640 1 -- 172.21.15.175:0/1914786407 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff608097a60 con 0x561f33b4a430 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 10 client.27014 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 10 client.27014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:44.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 20 client.27014 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:44.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33b64b10 0x561f33b66fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 4 client.27014 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:44.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.838+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f33abc040 con 0x561f33b64b10 2024-07-29T00:44:44.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.839+0000 7ff621600640 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33b64b10 0x561f33b66fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.839+0000 7ff621600640 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33b64b10 0x561f33b66fa0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a858e50e270f00c server_cookie=252461bbda87dc18 in_seq=0 out_seq=0 2024-07-29T00:44:44.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.839+0000 7ff602a00640 10 client.27014 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:44.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff602a00640 1 -- 172.21.15.175:0/1914786407 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561f33abc040 con 0x561f33b64b10 2024-07-29T00:44:44.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff602a00640 10 client.27014 handle_command_reply: tid=0 2024-07-29T00:44:44.846 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 client.27014 shutdown 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 msgr2=0x561f33b52110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b52110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cce90 tx=0x7ff60809b950 comp rx=0 tx=0).stop 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 shutdown_connections 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33b64b10 0x561f33b66fa0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b54cf0 0x561f33b52650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b4a430 0x561f33b52110 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 --2- 172.21.15.175:0/1914786407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b49b10 0x561f33b51bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 >> 172.21.15.175:0/1914786407 conn(0x561f33b460a0 msgr2=0x561f33b4bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:44.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.840+0000 7ff602a00640 0 client.27014 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:44.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.841+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 shutdown_connections 2024-07-29T00:44:44.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.841+0000 7ff626c35740 1 -- 172.21.15.175:0/1914786407 wait complete. 2024-07-29T00:44:44.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.841+0000 7ff626c35740 20 client.27014 trim_cache size 0 max 16384 2024-07-29T00:44:44.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.859+0000 7ff626c35740 1 Processor -- start 2024-07-29T00:44:44.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.859+0000 7ff626c35740 1 -- start start 2024-07-29T00:44:44.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff626c35740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff626c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b4bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff626c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b6c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff626c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1af810 con 0x561f33b54cf0 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff626c35740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1b1350 con 0x561f33b49b10 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff626c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1afee0 con 0x561f33b4a430 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff622000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b6c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff621600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff621600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39290/0 (socket says 172.21.15.175:39290) 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff621600640 1 -- 172.21.15.175:0/1207729394 learned_addr learned my addr 172.21.15.175:0/1207729394 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.860+0000 7ff620c00640 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b4bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff620c00640 1 -- 172.21.15.175:0/1207729394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 msgr2=0x561f33b524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff620c00640 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff620c00640 1 -- 172.21.15.175:0/1207729394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 msgr2=0x561f33b6c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff620c00640 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b6c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.862 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff620c00640 1 -- 172.21.15.175:0/1207729394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f33b4a260 con 0x561f33b4a430 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff620c00640 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b4bcc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c880 tx=0x7ff608097f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec037d30021dfa8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff60dc00640 1 -- 172.21.15.175:0/1207729394 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60807e020 con 0x561f33b4a430 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff60dc00640 1 -- 172.21.15.175:0/1207729394 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6080b1060 con 0x561f33b4a430 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.861+0000 7ff60dc00640 1 -- 172.21.15.175:0/1207729394 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60809b160 con 0x561f33b4a430 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 -- 172.21.15.175:0/1207729394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 msgr2=0x561f33b4bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b4bcc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c880 tx=0x7ff608097f10 comp rx=0 tx=0).stop 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 -- 172.21.15.175:0/1207729394 shutdown_connections 2024-07-29T00:44:44.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b6c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b4bcc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 --2- 172.21.15.175:0/1207729394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 -- 172.21.15.175:0/1207729394 >> 172.21.15.175:0/1207729394 conn(0x561f33b460a0 msgr2=0x561f33b4b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 -- 172.21.15.175:0/1207729394 shutdown_connections 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.862+0000 7ff626c35740 1 -- 172.21.15.175:0/1207729394 wait complete. 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 -- start start 2024-07-29T00:44:44.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 Processor -- start 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b58930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b5c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1b0c80 con 0x561f33b54cf0 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1b05b0 con 0x561f33b49b10 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1ad810 con 0x561f33b4a430 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.863+0000 7ff626c35740 10 client.0 _ll_register_callbacks cb 0x7ff60809bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff620c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b58930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:39294/0 (socket says 172.21.15.175:39294) 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff620c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b58930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:56950/0 (socket says 172.21.15.175:56950) 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 -- 172.21.15.175:0/2790242085 learned_addr learned my addr 172.21.15.175:0/2790242085 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff622000640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b5c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 -- 172.21.15.175:0/2790242085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 msgr2=0x561f33b58930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b58930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 -- 172.21.15.175:0/2790242085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 msgr2=0x561f33b5c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:44.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b5c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 -- 172.21.15.175:0/2790242085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f33b5d020 con 0x561f33b49b10 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff622000640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b5c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff620c00640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b58930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff621600640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b583f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff610000c00 tx=0x7ff610005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d99a2e32cd6da5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.864+0000 7ff602a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff602a00640 1 -- 172.21.15.175:0/2790242085 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610057020 con 0x561f33b49b10 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff602a00640 1 -- 172.21.15.175:0/2790242085 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7ff610004040 con 0x561f33b49b10 2024-07-29T00:44:44.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6080b1060 con 0x561f33b49b10 2024-07-29T00:44:44.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff602a00640 1 -- 172.21.15.175:0/2790242085 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff610062050 con 0x561f33b49b10 2024-07-29T00:44:44.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff626c35740 10 client.27020 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:44.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561f33bcf2b0 con 0x561f33b49b10 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.865+0000 7ff602a00640 1 -- 172.21.15.175:0/2790242085 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6080b1060 con 0x561f33b49b10 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff602a00640 1 -- 172.21.15.175:0/2790242085 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff6100523b0 con 0x561f33b49b10 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff626c35740 10 client.27020 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff626c35740 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff626c35740 20 client.27020 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff626c35740 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33bcf6b0 0x561f33bd1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff626c35740 4 client.27020 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:44:44.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f33ab6550 con 0x561f33bcf6b0 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.866+0000 7ff620c00640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33bcf6b0 0x561f33bd1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff620c00640 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33bcf6b0 0x561f33bd1aa0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26b8e143ccea5b2 server_cookie=f935a24afb911c71 in_seq=0 out_seq=0 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff602a00640 10 client.27020 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff602a00640 1 -- 172.21.15.175:0/2790242085 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f33ab6550 con 0x561f33bcf6b0 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff602a00640 10 client.27020 handle_command_reply: tid=0 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 client.27020 shutdown 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 msgr2=0x561f33b583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b583f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff610000c00 tx=0x7ff610005670 comp rx=0 tx=0).stop 2024-07-29T00:44:44.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 shutdown_connections 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x561f33bcf6b0 0x561f33bd1aa0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f33b54cf0 0x561f33b5c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f33b4a430 0x561f33b58930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 --2- 172.21.15.175:0/2790242085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f33b49b10 0x561f33b583f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff602a00640 0 client.27020 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 >> 172.21.15.175:0/2790242085 conn(0x561f33b460a0 msgr2=0x561f33b50070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 shutdown_connections 2024-07-29T00:44:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 1 -- 172.21.15.175:0/2790242085 wait complete. 2024-07-29T00:44:44.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.867+0000 7ff626c35740 20 client.27020 trim_cache size 0 max 16384 2024-07-29T00:44:44.875 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:44:44.875 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:44:44.875 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:44:44.875 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:44:44.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff6040a2d70 msgr2=0x7ff6040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff6040a2d70 0x7ff6040a5230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff614009fd0 tx=0x7ff614009500 comp rx=0 tx=0).stop 2024-07-29T00:44:44.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 msgr2=0x7ff61c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:44.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 0x7ff61c19a6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff60800e9f0 tx=0x7ff60800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 shutdown_connections 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1076a0 0x7ff61c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff6040a2d70 0x7ff6040a5230 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff61c102040 0x7ff61c19a6e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 --2- 172.21.15.175:0/261268670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61c1016a0 0x7ff61c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 >> 172.21.15.175:0/261268670 conn(0x7ff61c0fd3a0 msgr2=0x7ff61c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 shutdown_connections 2024-07-29T00:44:44.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:44.868+0000 7ff622a00640 1 -- 172.21.15.175:0/261268670 wait complete. 2024-07-29T00:44:45.886 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:44:45.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:44:46.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:45 smithi204 ceph-mon[37368]: pgmap v254: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:46.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:45 smithi181 ceph-mon[36706]: pgmap v254: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:46.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:45 smithi175 ceph-mon[32138]: pgmap v254: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:46.339 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:46.980 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 -- 172.21.15.175:0/3370929820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074ba0 msgr2=0x7fe570112190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 --2- 172.21.15.175:0/3370929820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074ba0 0x7fe570112190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe558009900 tx=0x7fe55802a480 comp rx=0 tx=0).stop 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 -- 172.21.15.175:0/3370929820 shutdown_connections 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 --2- 172.21.15.175:0/3370929820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5701126d0 0x7fe570114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 --2- 172.21.15.175:0/3370929820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074ba0 0x7fe570112190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 --2- 172.21.15.175:0/3370929820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe570074280 0x7fe570074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 -- 172.21.15.175:0/3370929820 >> 172.21.15.175:0/3370929820 conn(0x7fe5700fd400 msgr2=0x7fe5700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:46.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 -- 172.21.15.175:0/3370929820 shutdown_connections 2024-07-29T00:44:46.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.980+0000 7fe575400640 1 -- 172.21.15.175:0/3370929820 wait complete. 2024-07-29T00:44:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.981+0000 7fe575400640 1 Processor -- start 2024-07-29T00:44:46.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 -- start start 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 0x7fe57019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe570074ba0 0x7fe57019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5701126d0 0x7fe5701a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701180f0 con 0x7fe570074280 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe570117f70 con 0x7fe5701126d0 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe575400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe570118270 con 0x7fe570074ba0 2024-07-29T00:44:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe56fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 0x7fe57019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe56fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 0x7fe57019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41848/0 (socket says 172.21.15.175:41848) 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe56fe00640 1 -- 172.21.15.175:0/3253646971 learned_addr learned my addr 172.21.15.175:0/3253646971 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.982+0000 7fe567400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe570074ba0 0x7fe57019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe574a00640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5701126d0 0x7fe5701a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56fe00640 1 -- 172.21.15.175:0/3253646971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe570074ba0 msgr2=0x7fe57019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56fe00640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe570074ba0 0x7fe57019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56fe00640 1 -- 172.21.15.175:0/3253646971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5701126d0 msgr2=0x7fe5701a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56fe00640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5701126d0 0x7fe5701a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:46.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56fe00640 1 -- 172.21.15.175:0/3253646971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5701a3590 con 0x7fe570074280 2024-07-29T00:44:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56fe00640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 0x7fe57019e5e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe56000d950 tx=0x7fe56000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560017030 con 0x7fe570074280 2024-07-29T00:44:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe560004020 con 0x7fe570074280 2024-07-29T00:44:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600047a0 con 0x7fe570074280 2024-07-29T00:44:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.983+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5701a3880 con 0x7fe570074280 2024-07-29T00:44:46.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.984+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5701ab160 con 0x7fe570074280 2024-07-29T00:44:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.985+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe570074660 con 0x7fe570074280 2024-07-29T00:44:46.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.987+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe5600041d0 con 0x7fe570074280 2024-07-29T00:44:46.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.988+0000 7fe56d600640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe5540a2d70 0x7fe5540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:46.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.988+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7fe5600cee70 con 0x7fe570074280 2024-07-29T00:44:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.988+0000 7fe567400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe5540a2d70 0x7fe5540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:46.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.989+0000 7fe567400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe5540a2d70 0x7fe5540a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe558000c00 tx=0x7fe558005b50 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:46.993 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:46.993+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe56009b9e0 con 0x7fe570074280 2024-07-29T00:44:47.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.222+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe57019f940 con 0x7fe570074280 2024-07-29T00:44:47.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.224+0000 7fe56d600640 1 -- 172.21.15.175:0/3253646971 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11994 (secure 0 0 0) 0x7fe56009b2e0 con 0x7fe570074280 2024-07-29T00:44:47.225 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:44:47.225 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 43 2024-07-29T00:44:47.226 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":43,"btime":"2024-07-29T00:43:57:876956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:57.442709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe5540a2d70 msgr2=0x7fe5540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe5540a2d70 0x7fe5540a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe558000c00 tx=0x7fe558005b50 comp rx=0 tx=0).stop 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 msgr2=0x7fe57019e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 0x7fe57019e5e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe56000d950 tx=0x7fe56000de10 comp rx=0 tx=0).stop 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 shutdown_connections 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5701126d0 0x7fe5701a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe570074ba0 0x7fe57019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:47.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe5540a2d70 0x7fe5540a5230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:47.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 --2- 172.21.15.175:0/3253646971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe570074280 0x7fe57019e5e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:47.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.228+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 >> 172.21.15.175:0/3253646971 conn(0x7fe5700fd400 msgr2=0x7fe5700fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:47.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.229+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 shutdown_connections 2024-07-29T00:44:47.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:47.229+0000 7fe575400640 1 -- 172.21.15.175:0/3253646971 wait complete. 2024-07-29T00:44:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:47 smithi175 ceph-mon[32138]: pgmap v255: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:47 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3253646971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:44:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:47 smithi204 ceph-mon[37368]: pgmap v255: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:48.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:47 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3253646971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:44:48.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:44:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:47 smithi181 ceph-mon[36706]: pgmap v255: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:47 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3253646971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:44:48.561 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:49.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.231+0000 7f36b3200640 1 -- 172.21.15.175:0/2806822568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101b20 msgr2=0x7f36b410d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.231+0000 7f36b3200640 1 --2- 172.21.15.175:0/2806822568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101b20 0x7f36b410d4a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009860 tx=0x7f36ac02a340 comp rx=0 tx=0).stop 2024-07-29T00:44:49.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 -- 172.21.15.175:0/2806822568 shutdown_connections 2024-07-29T00:44:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 --2- 172.21.15.175:0/2806822568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101b20 0x7f36b410d4a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 --2- 172.21.15.175:0/2806822568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4101180 0x7f36b41015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 --2- 172.21.15.175:0/2806822568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b4105600 0x7f36b41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 -- 172.21.15.175:0/2806822568 >> 172.21.15.175:0/2806822568 conn(0x7f36b40fb610 msgr2=0x7f36b40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 -- 172.21.15.175:0/2806822568 shutdown_connections 2024-07-29T00:44:49.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.232+0000 7f36b3200640 1 -- 172.21.15.175:0/2806822568 wait complete. 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.233+0000 7f36b3200640 1 Processor -- start 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.233+0000 7f36b3200640 1 -- start start 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101180 0x7f36b419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b4101b20 0x7f36b419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 0x7f36b419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4111b10 con 0x7f36b4105600 2024-07-29T00:44:49.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4111990 con 0x7f36b4101180 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4111c90 con 0x7f36b4101b20 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 0x7f36b419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 0x7f36b419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57650/0 (socket says 172.21.15.175:57650) 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b2800640 1 -- 172.21.15.175:0/1654994135 learned_addr learned my addr 172.21.15.175:0/1654994135 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b1e00640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101180 0x7f36b419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.234+0000 7f36b1400640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b4101b20 0x7f36b419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36b2800640 1 -- 172.21.15.175:0/1654994135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b4101b20 msgr2=0x7f36b419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36b2800640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b4101b20 0x7f36b419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36b2800640 1 -- 172.21.15.175:0/1654994135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101180 msgr2=0x7f36b419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36b2800640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101180 0x7f36b419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36b2800640 1 -- 172.21.15.175:0/1654994135 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b419f0d0 con 0x7f36b4105600 2024-07-29T00:44:49.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36b2800640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 0x7f36b419e9f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f36ac02a930 tx=0x7f36ac033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:49.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.235+0000 7f36aaa00640 1 -- 172.21.15.175:0/1654994135 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac037040 con 0x7f36b4105600 2024-07-29T00:44:49.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.236+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b419f360 con 0x7f36b4105600 2024-07-29T00:44:49.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.236+0000 7f36aaa00640 1 -- 172.21.15.175:0/1654994135 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac032770 con 0x7f36b4105600 2024-07-29T00:44:49.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.236+0000 7f36aaa00640 1 -- 172.21.15.175:0/1654994135 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0054d0 con 0x7f36b4105600 2024-07-29T00:44:49.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.236+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b41a6ca0 con 0x7f36b4105600 2024-07-29T00:44:49.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.239+0000 7f36aaa00640 1 -- 172.21.15.175:0/1654994135 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f36ac004250 con 0x7f36b4105600 2024-07-29T00:44:49.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.239+0000 7f36bb4bd740 1 Processor -- start 2024-07-29T00:44:49.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 -- start start 2024-07-29T00:44:49.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 0x555e1b6cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555e1b635ab0 con 0x555e1b6cfb70 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555e1b5cda80 con 0x555e1b6d10e0 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555e1b631a70 con 0x555e1b6da910 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36b1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 0x555e1b6cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57658/0 (socket says 172.21.15.175:57658) 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36b1e00640 1 -- 172.21.15.175:0/4278562931 learned_addr learned my addr 172.21.15.175:0/4278562931 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:49.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36b1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 0x555e1b6cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38588/0 (socket says 172.21.15.175:38588) 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.240+0000 7f36b1400640 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1e00640 1 -- 172.21.15.175:0/4278562931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 msgr2=0x555e1b6dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36aaa00640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36980a2cf0 0x7f36980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1e00640 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1e00640 1 -- 172.21.15.175:0/4278562931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 msgr2=0x555e1b6cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1e00640 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 0x555e1b6cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1e00640 1 -- 172.21.15.175:0/4278562931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1b3a73c0 con 0x555e1b6cfb70 2024-07-29T00:44:49.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1400640 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 0x555e1b6cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:49.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36aaa00640 1 -- 172.21.15.175:0/1654994135 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f36ac0ef1c0 con 0x7f36b4105600 2024-07-29T00:44:49.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b2800640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36980a2cf0 0x7f36980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.241+0000 7f36b1e00640 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cffb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f36a40114f0 tx=0x7f36a40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb423153a07ece0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.242+0000 7f36a3000640 1 -- 172.21.15.175:0/4278562931 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a401e070 con 0x555e1b6cfb70 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.242+0000 7f36a3000640 1 -- 172.21.15.175:0/4278562931 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a40110f0 con 0x555e1b6cfb70 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.242+0000 7f36a3000640 1 -- 172.21.15.175:0/4278562931 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40107a0 con 0x555e1b6cfb70 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 -- 172.21.15.175:0/4278562931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 msgr2=0x555e1b6cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36b2800640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36980a2cf0 0x7f36980a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0ec9d0 tx=0x7f36ac02a340 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cffb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f36a40114f0 tx=0x7f36a40119b0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 -- 172.21.15.175:0/4278562931 shutdown_connections 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cffb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/4278562931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6d10e0 0x555e1b6cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 -- 172.21.15.175:0/4278562931 >> 172.21.15.175:0/4278562931 conn(0x555e1b698950 msgr2=0x555e1b6cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 -- 172.21.15.175:0/4278562931 shutdown_connections 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 -- 172.21.15.175:0/4278562931 wait complete. 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.243+0000 7f36bb4bd740 1 -- start start 2024-07-29T00:44:49.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 Processor -- start 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6cfb70 0x555e1b6d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 0x555e1b6d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6d8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555e1b6dd6b0 con 0x555e1b6d10e0 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555e1b5f0c10 con 0x555e1b6cfb70 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555e1b6dd830 con 0x555e1b6da910 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.244+0000 7f36bb4bd740 10 client.0 _ll_register_callbacks cb 0x555e1b5b9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:49.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 0x555e1b6d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 0x555e1b6d7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57672/0 (socket says 172.21.15.175:57672) 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 -- 172.21.15.175:0/3695268463 learned_addr learned my addr 172.21.15.175:0/3695268463 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1400640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6cfb70 0x555e1b6d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1400640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 -- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 msgr2=0x555e1b6d8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6d8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 -- 172.21.15.175:0/3695268463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6cfb70 msgr2=0x555e1b6d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6cfb70 0x555e1b6d7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 -- 172.21.15.175:0/3695268463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1b5f41e0 con 0x555e1b6d10e0 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1400640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6cfb70 0x555e1b6d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f36b1e00640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 0x555e1b6d7e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36a4011eb0 tx=0x7f36a40092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07beffdcfa5d02d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.245+0000 7f3697400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f3697400640 1 -- 172.21.15.175:0/3695268463 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a401e040 con 0x555e1b6d10e0 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f3697400640 1 -- 172.21.15.175:0/3695268463 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f36a4004e00 con 0x555e1b6d10e0 2024-07-29T00:44:49.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a40110f0 con 0x555e1b6d10e0 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f3697400640 1 -- 172.21.15.175:0/3695268463 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f36a4031050 con 0x555e1b6d10e0 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f36bb4bd740 10 client.16890 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555e1b6e94c0 con 0x555e1b6d10e0 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.246+0000 7f3697400640 1 -- 172.21.15.175:0/3695268463 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a40110f0 con 0x555e1b6d10e0 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f3697400640 1 -- 172.21.15.175:0/3695268463 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f36a402c030 con 0x555e1b6d10e0 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36bb4bd740 10 client.16890 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36bb4bd740 10 client.16890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:44:49.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36bb4bd740 20 client.16890 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:49.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6ea8c0 0x555e1b6eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36bb4bd740 4 client.16890 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:49.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e1b643020 con 0x555e1b6ea8c0 2024-07-29T00:44:49.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.247+0000 7f36b1400640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6ea8c0 0x555e1b6eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.248+0000 7f36b1400640 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6ea8c0 0x555e1b6eccb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32052edb9c7a948e server_cookie=cf5f18be9b2a07c4 in_seq=0 out_seq=0 2024-07-29T00:44:49.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.248+0000 7f3697400640 10 client.16890 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f3697400640 1 -- 172.21.15.175:0/3695268463 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555e1b643020 con 0x555e1b6ea8c0 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f3697400640 10 client.16890 handle_command_reply: tid=0 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 client.16890 shutdown 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 msgr2=0x555e1b6d7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 0x555e1b6d7e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36a4011eb0 tx=0x7f36a40092a0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 shutdown_connections 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6da910 0x555e1b6d8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6d10e0 0x555e1b6d7e50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6ea8c0 0x555e1b6eccb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f3697400640 0 client.16890 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/3695268463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6cfb70 0x555e1b6d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 >> 172.21.15.175:0/3695268463 conn(0x555e1b698950 msgr2=0x555e1b6d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 shutdown_connections 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 1 -- 172.21.15.175:0/3695268463 wait complete. 2024-07-29T00:44:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.249+0000 7f36bb4bd740 20 client.16890 trim_cache size 0 max 16384 2024-07-29T00:44:49.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 Processor -- start 2024-07-29T00:44:49.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 -- start start 2024-07-29T00:44:49.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41af720 con 0x555e1b6cfb70 2024-07-29T00:44:49.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41b1260 con 0x555e1b6da910 2024-07-29T00:44:49.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41afdf0 con 0x555e1b6d10e0 2024-07-29T00:44:49.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57680/0 (socket says 172.21.15.175:57680) 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.264+0000 7f36b1400640 1 -- 172.21.15.175:0/1057796910 learned_addr learned my addr 172.21.15.175:0/1057796910 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1400640 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1e00640 1 -- 172.21.15.175:0/1057796910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 msgr2=0x555e1b6dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1e00640 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6dc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1e00640 1 -- 172.21.15.175:0/1057796910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 msgr2=0x555e1b6cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1e00640 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1e00640 1 -- 172.21.15.175:0/1057796910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1b6cc470 con 0x555e1b6d10e0 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1400640 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36b1e00640 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6dbd00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009270 tx=0x7f36a4005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106f3259a6e8ace1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36a2600640 1 -- 172.21.15.175:0/1057796910 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a401e070 con 0x555e1b6d10e0 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36a2600640 1 -- 172.21.15.175:0/1057796910 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a40110f0 con 0x555e1b6d10e0 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.265+0000 7f36a2600640 1 -- 172.21.15.175:0/1057796910 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4003bd0 con 0x555e1b6d10e0 2024-07-29T00:44:49.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 -- 172.21.15.175:0/1057796910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 msgr2=0x555e1b6dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6dbd00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009270 tx=0x7f36a4005bd0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 -- 172.21.15.175:0/1057796910 shutdown_connections 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6dbd00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/1057796910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 -- 172.21.15.175:0/1057796910 >> 172.21.15.175:0/1057796910 conn(0x555e1b698950 msgr2=0x555e1b6d61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 -- 172.21.15.175:0/1057796910 shutdown_connections 2024-07-29T00:44:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.266+0000 7f36bb4bd740 1 -- 172.21.15.175:0/1057796910 wait complete. 2024-07-29T00:44:49.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 -- start start 2024-07-29T00:44:49.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 Processor -- start 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41b0b90 con 0x555e1b6cfb70 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41b04c0 con 0x555e1b6da910 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41ad720 con 0x555e1b6d10e0 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.267+0000 7f36bb4bd740 10 client.0 _ll_register_callbacks cb 0x7f36a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6f13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57688/0 (socket says 172.21.15.175:57688) 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1400640 1 -- 172.21.15.175:0/899765709 learned_addr learned my addr 172.21.15.175:0/899765709 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1400640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1e00640 1 -- 172.21.15.175:0/899765709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 msgr2=0x555e1b6f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1e00640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1e00640 1 -- 172.21.15.175:0/899765709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 msgr2=0x555e1b6f13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:49.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1e00640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6f13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1e00640 1 -- 172.21.15.175:0/899765709 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1b6e1af0 con 0x555e1b6d10e0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.268+0000 7f36b1400640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f36b1e00640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6f1920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009270 tx=0x7f36a4009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1752e9a82bbef80 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f3697400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f3697400640 1 -- 172.21.15.175:0/899765709 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a401e070 con 0x555e1b6d10e0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f3697400640 1 -- 172.21.15.175:0/899765709 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f36a40042e0 con 0x555e1b6d10e0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a40110f0 con 0x555e1b6d10e0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f3697400640 1 -- 172.21.15.175:0/899765709 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f36a404c050 con 0x555e1b6d10e0 2024-07-29T00:44:49.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f36bb4bd740 10 client.26992 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.269+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555e1b6e2db0 con 0x555e1b6d10e0 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f3697400640 1 -- 172.21.15.175:0/899765709 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a40110f0 con 0x555e1b6d10e0 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f3697400640 1 -- 172.21.15.175:0/899765709 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f36a4047030 con 0x555e1b6d10e0 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36bb4bd740 10 client.26992 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36bb4bd740 10 client.26992 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36bb4bd740 20 client.26992 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6e34d0 0x555e1b75fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36bb4bd740 4 client.26992 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:49.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e1b63d530 con 0x555e1b6e34d0 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.270+0000 7f36b1400640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6e34d0 0x555e1b75fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36b1400640 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6e34d0 0x555e1b75fbe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55888f5f9ad8ae70 server_cookie=cb0bc479b6f51d6 in_seq=0 out_seq=0 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f3697400640 10 client.26992 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f3697400640 1 -- 172.21.15.175:0/899765709 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e1b63d530 con 0x555e1b6e34d0 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f3697400640 10 client.26992 handle_command_reply: tid=0 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 client.26992 shutdown 2024-07-29T00:44:49.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 msgr2=0x555e1b6f1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6f1920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009270 tx=0x7f36a4009610 comp rx=0 tx=0).stop 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 shutdown_connections 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x555e1b6e34d0 0x555e1b75fbe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e1b6da910 0x555e1b6f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e1b6d10e0 0x555e1b6f1920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 --2- 172.21.15.175:0/899765709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1b6cfb70 0x555e1b6f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f3697400640 0 client.26992 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:44:49.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.271+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 >> 172.21.15.175:0/899765709 conn(0x555e1b698950 msgr2=0x555e1b6cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:49.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 shutdown_connections 2024-07-29T00:44:49.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36bb4bd740 1 -- 172.21.15.175:0/899765709 wait complete. 2024-07-29T00:44:49.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36bb4bd740 20 client.26992 trim_cache size 0 max 16384 2024-07-29T00:44:49.279 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:49.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36980a2cf0 msgr2=0x7f36980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36980a2cf0 0x7f36980a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0ec9d0 tx=0x7f36ac02a340 comp rx=0 tx=0).stop 2024-07-29T00:44:49.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 msgr2=0x7f36b419e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:49.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 0x7f36b419e9f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f36ac02a930 tx=0x7f36ac033d10 comp rx=0 tx=0).stop 2024-07-29T00:44:49.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 shutdown_connections 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36980a2cf0 0x7f36980a51b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b4105600 0x7f36b419e9f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b4101b20 0x7f36b419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 --2- 172.21.15.175:0/1654994135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b4101180 0x7f36b419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.272+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 >> 172.21.15.175:0/1654994135 conn(0x7f36b40fb610 msgr2=0x7f36b410e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.273+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 shutdown_connections 2024-07-29T00:44:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:49.273+0000 7f36b3200640 1 -- 172.21.15.175:0/1654994135 wait complete. 2024-07-29T00:44:50.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:49 smithi204 ceph-mon[37368]: pgmap v256: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:50.190 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:49 smithi175 ceph-mon[32138]: pgmap v256: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:50.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:44:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:49 smithi181 ceph-mon[36706]: pgmap v256: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:50.693 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:51.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 -- 172.21.15.175:0/3317253534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 msgr2=0x7f2f84110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 --2- 172.21.15.175:0/3317253534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 0x7f2f84110330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009be0 tx=0x7f2f7c02a750 comp rx=0 tx=0).stop 2024-07-29T00:44:51.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 -- 172.21.15.175:0/3317253534 shutdown_connections 2024-07-29T00:44:51.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 --2- 172.21.15.175:0/3317253534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 0x7f2f84110330 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 --2- 172.21.15.175:0/3317253534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f84074bd0 0x7f2f8410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 --2- 172.21.15.175:0/3317253534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f840742b0 0x7f2f84074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 -- 172.21.15.175:0/3317253534 >> 172.21.15.175:0/3317253534 conn(0x7f2f840fd3a0 msgr2=0x7f2f840ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 -- 172.21.15.175:0/3317253534 shutdown_connections 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.379+0000 7f2f89000640 1 -- 172.21.15.175:0/3317253534 wait complete. 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.380+0000 7f2f89000640 1 Processor -- start 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.380+0000 7f2f89000640 1 -- start start 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f89000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f840742b0 0x7f2f8419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f89000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 0x7f2f8419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f89000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 0x7f2f8419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f89000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f84113c50 con 0x7f2f84074bd0 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f89000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f84113ad0 con 0x7f2f840742b0 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f89000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f84113dd0 con 0x7f2f8410df40 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f82a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 0x7f2f8419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f82a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 0x7f2f8419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57702/0 (socket says 172.21.15.175:57702) 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f82a00640 1 -- 172.21.15.175:0/3923561466 learned_addr learned my addr 172.21.15.175:0/3923561466 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f83e00640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 0x7f2f8419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.381+0000 7f2f83400640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f840742b0 0x7f2f8419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f82a00640 1 -- 172.21.15.175:0/3923561466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 msgr2=0x7f2f8419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f82a00640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 0x7f2f8419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f82a00640 1 -- 172.21.15.175:0/3923561466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f840742b0 msgr2=0x7f2f8419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f82a00640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f840742b0 0x7f2f8419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f82a00640 1 -- 172.21.15.175:0/3923561466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8419f100 con 0x7f2f84074bd0 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f82a00640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 0x7f2f8419a690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000e9f0 tx=0x7f2f7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:51.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f7be00640 1 -- 172.21.15.175:0/3923561466 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7000cd80 con 0x7f2f84074bd0 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f7be00640 1 -- 172.21.15.175:0/3923561466 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f70010910 con 0x7f2f84074bd0 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f7be00640 1 -- 172.21.15.175:0/3923561466 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70018990 con 0x7f2f84074bd0 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.382+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8419f3f0 con 0x7f2f84074bd0 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.383+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f841a6cd0 con 0x7f2f84074bd0 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 Processor -- start 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 -- start start 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f7be00640 1 -- 172.21.15.175:0/3923561466 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2f70010ac0 con 0x7f2f84074bd0 2024-07-29T00:44:51.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e340630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e340fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e34dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ad5e2a6ab0 con 0x55ad5e34b910 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ad5e23ea80 con 0x55ad5e3420e0 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.387+0000 7f2f8d004740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad5e2a2a70 con 0x55ad5e340b70 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e340630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e340630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38650/0 (socket says 172.21.15.175:38650) 2024-07-29T00:44:51.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 -- 172.21.15.175:0/3951759811 learned_addr learned my addr 172.21.15.175:0/3951759811 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83400640 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e340fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f82a00640 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e34dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 -- 172.21.15.175:0/3951759811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 msgr2=0x55ad5e340fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e340fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 -- 172.21.15.175:0/3951759811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 msgr2=0x55ad5e34dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e34dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.388+0000 7f2f83e00640 1 -- 172.21.15.175:0/3951759811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5e0183c0 con 0x55ad5e3420e0 2024-07-29T00:44:51.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.389+0000 7f2f83e00640 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e340630 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009be0 tx=0x7f2f7c005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9769667db3ec8147 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.389+0000 7f2f78c00640 1 -- 172.21.15.175:0/3951759811 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c041030 con 0x55ad5e3420e0 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.389+0000 7f2f78c00640 1 -- 172.21.15.175:0/3951759811 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c030bf0 con 0x55ad5e3420e0 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.389+0000 7f2f78c00640 1 -- 172.21.15.175:0/3951759811 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c036040 con 0x55ad5e3420e0 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.389+0000 7f2f7be00640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2f680a2d70 0x7f2f680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.389+0000 7f2f7be00640 1 -- 172.21.15.175:0/3923561466 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f2f700cf6d0 con 0x7f2f84074bd0 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f83400640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2f680a2d70 0x7f2f680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 -- 172.21.15.175:0/3951759811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 msgr2=0x55ad5e340630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e340630 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009be0 tx=0x7f2f7c005ec0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 -- 172.21.15.175:0/3951759811 shutdown_connections 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e34dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e340fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 --2- 172.21.15.175:0/3951759811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e340630 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 -- 172.21.15.175:0/3951759811 >> 172.21.15.175:0/3951759811 conn(0x55ad5e309950 msgr2=0x55ad5e33d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 -- 172.21.15.175:0/3951759811 shutdown_connections 2024-07-29T00:44:51.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.390+0000 7f2f8d004740 1 -- 172.21.15.175:0/3951759811 wait complete. 2024-07-29T00:44:51.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.391+0000 7f2f83400640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2f680a2d70 0x7f2f680a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55ad5e33e060 tx=0x7f2f74009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:51.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.391+0000 7f2f8d004740 1 -- start start 2024-07-29T00:44:51.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.391+0000 7f2f8d004740 1 Processor -- start 2024-07-29T00:44:51.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e340b70 0x55ad5e348810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e348d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 0x55ad5e349290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ad5e34e650 con 0x55ad5e340b70 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ad5e261c10 con 0x55ad5e34b910 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad5e34e7d0 con 0x55ad5e3420e0 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f8d004740 10 client.0 _ll_register_callbacks cb 0x55ad5e22aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e340b70 0x55ad5e348810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 0x55ad5e349290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 0x55ad5e349290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38664/0 (socket says 172.21.15.175:38664) 2024-07-29T00:44:51.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 -- 172.21.15.175:0/297111790 learned_addr learned my addr 172.21.15.175:0/297111790 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f82a00640 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e348d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 -- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 msgr2=0x55ad5e348d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e348d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 -- 172.21.15.175:0/297111790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e340b70 msgr2=0x55ad5e348810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e340b70 0x55ad5e348810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.392+0000 7f2f83e00640 1 -- 172.21.15.175:0/297111790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad5e2651e0 con 0x55ad5e34b910 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f83e00640 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 0x55ad5e349290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c02a790 tx=0x7f2f7c009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd2865199989303 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:51.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f6d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f6d600640 1 -- 172.21.15.175:0/297111790 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c041040 con 0x55ad5e34b910 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f7c030bf0 con 0x55ad5e34b910 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f6d600640 1 -- 172.21.15.175:0/297111790 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f2f7c0303a0 con 0x55ad5e34b910 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f6d600640 1 -- 172.21.15.175:0/297111790 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c036040 con 0x55ad5e34b910 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f8d004740 10 client.27056 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.393+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad5e35a350 con 0x55ad5e34b910 2024-07-29T00:44:51.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f6d600640 1 -- 172.21.15.175:0/297111790 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c030bf0 con 0x55ad5e34b910 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f6d600640 1 -- 172.21.15.175:0/297111790 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f2f7c031440 con 0x55ad5e34b910 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f8d004740 10 client.27056 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f8d004740 10 client.27056 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f8d004740 20 client.27056 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f8d004740 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e35b6a0 0x55ad5e35da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f8d004740 4 client.27056 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.394+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad5e2b4020 con 0x55ad5e35b6a0 2024-07-29T00:44:51.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.395+0000 7f2f83400640 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e35b6a0 0x55ad5e35da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.395+0000 7f2f83400640 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e35b6a0 0x55ad5e35da90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f8cd5d7b025c7a4 server_cookie=4eff2d0fe9d38e41 in_seq=0 out_seq=0 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.395+0000 7f2f6d600640 10 client.27056 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f6d600640 1 -- 172.21.15.175:0/297111790 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad5e2b4020 con 0x55ad5e35b6a0 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f6d600640 10 client.27056 handle_command_reply: tid=0 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 client.27056 shutdown 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 msgr2=0x55ad5e349290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 0x55ad5e349290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c02a790 tx=0x7f2f7c009960 comp rx=0 tx=0).stop 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 shutdown_connections 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e35b6a0 0x55ad5e35da90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e34b910 0x55ad5e349290 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e348d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 --2- 172.21.15.175:0/297111790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e340b70 0x55ad5e348810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 >> 172.21.15.175:0/297111790 conn(0x55ad5e309950 msgr2=0x55ad5e342f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.396+0000 7f2f6d600640 0 client.27056 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.397+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 shutdown_connections 2024-07-29T00:44:51.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.397+0000 7f2f8d004740 1 -- 172.21.15.175:0/297111790 wait complete. 2024-07-29T00:44:51.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.397+0000 7f2f8d004740 20 client.27056 trim_cache size 0 max 16384 2024-07-29T00:44:51.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 Processor -- start 2024-07-29T00:44:51.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 -- start start 2024-07-29T00:44:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e363060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e3635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e3487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841af620 con 0x55ad5e34b910 2024-07-29T00:44:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841b1160 con 0x55ad5e3420e0 2024-07-29T00:44:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.411+0000 7f2f8d004740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841afcf0 con 0x55ad5e340b70 2024-07-29T00:44:51.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e363060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e3487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e363060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42812/0 (socket says 172.21.15.175:42812) 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 -- 172.21.15.175:0/2742897371 learned_addr learned my addr 172.21.15.175:0/2742897371 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f82a00640 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e3635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 -- 172.21.15.175:0/2742897371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 msgr2=0x55ad5e3635a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e3635a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 -- 172.21.15.175:0/2742897371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 msgr2=0x55ad5e3487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e3487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83400640 1 -- 172.21.15.175:0/2742897371 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5e3436c0 con 0x55ad5e340b70 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.412+0000 7f2f83e00640 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e3487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.413+0000 7f2f83400640 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e363060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005fd0 tx=0x7f2f7401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c385891bd9d1146 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.413+0000 7f2f6fe00640 1 -- 172.21.15.175:0/2742897371 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7401dc00 con 0x55ad5e340b70 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.413+0000 7f2f6fe00640 1 -- 172.21.15.175:0/2742897371 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7401dda0 con 0x55ad5e340b70 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.413+0000 7f2f6fe00640 1 -- 172.21.15.175:0/2742897371 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7402b690 con 0x55ad5e340b70 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.413+0000 7f2f8d004740 1 -- 172.21.15.175:0/2742897371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 msgr2=0x55ad5e363060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.413+0000 7f2f8d004740 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e363060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005fd0 tx=0x7f2f7401b000 comp rx=0 tx=0).stop 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 -- 172.21.15.175:0/2742897371 shutdown_connections 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e3487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e3420e0 0x55ad5e3635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 --2- 172.21.15.175:0/2742897371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e340b70 0x55ad5e363060 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 -- 172.21.15.175:0/2742897371 >> 172.21.15.175:0/2742897371 conn(0x55ad5e309950 msgr2=0x55ad5e347900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 -- 172.21.15.175:0/2742897371 shutdown_connections 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 -- 172.21.15.175:0/2742897371 wait complete. 2024-07-29T00:44:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 -- start start 2024-07-29T00:44:51.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.414+0000 7f2f8d004740 1 Processor -- start 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e340b70 0x55ad5e353ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e345200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e345740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841b0a90 con 0x55ad5e34b910 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841b03c0 con 0x55ad5e340b70 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f841ad620 con 0x55ad5e3420e0 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f8d004740 10 client.0 _ll_register_callbacks cb 0x7f2f74009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:51.421 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e345740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f83e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e345740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57744/0 (socket says 172.21.15.175:57744) 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f83e00640 1 -- 172.21.15.175:0/1016033212 learned_addr learned my addr 172.21.15.175:0/1016033212 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f82a00640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e345200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.415+0000 7f2f83400640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e340b70 0x55ad5e353ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f83e00640 1 -- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 msgr2=0x55ad5e345200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f83e00640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e345200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f83e00640 1 -- 172.21.15.175:0/1016033212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e340b70 msgr2=0x55ad5e353ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:51.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f83e00640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e340b70 0x55ad5e353ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f83e00640 1 -- 172.21.15.175:0/1016033212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad5e345cb0 con 0x55ad5e34b910 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f83e00640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e345740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c031a50 tx=0x7f2f7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f70a9fa9d8eb096 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f6d600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f6d600640 1 -- 172.21.15.175:0/1016033212 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c041050 con 0x55ad5e34b910 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f7401dda0 con 0x55ad5e34b910 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.416+0000 7f2f6d600640 1 -- 172.21.15.175:0/1016033212 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f2f7c0305a0 con 0x55ad5e34b910 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f6d600640 1 -- 172.21.15.175:0/1016033212 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c031650 con 0x55ad5e34b910 2024-07-29T00:44:51.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 10 client.16914 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad5e3c5d70 con 0x55ad5e34b910 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f6d600640 1 -- 172.21.15.175:0/1016033212 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7401dda0 con 0x55ad5e34b910 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f6d600640 1 -- 172.21.15.175:0/1016033212 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f2f7c05d070 con 0x55ad5e34b910 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 10 client.16914 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 10 client.16914 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 20 client.16914 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e3c64a0 0x55ad5e3c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:51.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 4 client.16914 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.417+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad5e2ae530 con 0x55ad5e3c64a0 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.418+0000 7f2f83400640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e3c64a0 0x55ad5e3c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.418+0000 7f2f83400640 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e3c64a0 0x55ad5e3c88c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=165a9deed5038e6 server_cookie=1c24098e134f0d36 in_seq=0 out_seq=0 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.418+0000 7f2f6d600640 10 client.16914 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.418+0000 7f2f6d600640 1 -- 172.21.15.175:0/1016033212 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad5e2ae530 con 0x55ad5e3c64a0 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.418+0000 7f2f6d600640 10 client.16914 handle_command_reply: tid=0 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.418+0000 7f2f8d004740 1 client.16914 shutdown 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 msgr2=0x55ad5e345740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.425 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e345740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c031a50 tx=0x7f2f7c005fd0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 shutdown_connections 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55ad5e3c64a0 0x55ad5e3c88c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad5e34b910 0x55ad5e345740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5e3420e0 0x55ad5e345200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 --2- 172.21.15.175:0/1016033212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad5e340b70 0x55ad5e353ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 >> 172.21.15.175:0/1016033212 conn(0x55ad5e309950 msgr2=0x55ad5e34a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:51.426 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f6d600640 0 client.16914 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:44:51.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 shutdown_connections 2024-07-29T00:44:51.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 1 -- 172.21.15.175:0/1016033212 wait complete. 2024-07-29T00:44:51.427 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.419+0000 7f2f8d004740 20 client.16914 trim_cache size 0 max 16384 2024-07-29T00:44:51.427 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:51.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2f680a2d70 msgr2=0x7f2f680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2f680a2d70 0x7f2f680a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55ad5e33e060 tx=0x7f2f74009260 comp rx=0 tx=0).stop 2024-07-29T00:44:51.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 msgr2=0x7f2f8419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:51.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 0x7f2f8419a690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000e9f0 tx=0x7f2f7000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.428 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 shutdown_connections 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f8410df40 0x7f2f8419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2f680a2d70 0x7f2f680a5230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f84074bd0 0x7f2f8419a690 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 --2- 172.21.15.175:0/3923561466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f840742b0 0x7f2f8419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 >> 172.21.15.175:0/3923561466 conn(0x7f2f840fd3a0 msgr2=0x7f2f840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 shutdown_connections 2024-07-29T00:44:51.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:51.420+0000 7f2f89000640 1 -- 172.21.15.175:0/3923561466 wait complete. 2024-07-29T00:44:52.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:51 smithi204 ceph-mon[37368]: pgmap v257: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:52.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:51 smithi175 ceph-mon[32138]: pgmap v257: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:51 smithi181 ceph-mon[36706]: pgmap v257: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:52.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:44:52.764 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:53.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:52 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:44:53.120 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:52 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:44:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:52 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:44:53.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 -- 172.21.15.175:0/2775632938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c81098c0 msgr2=0x7f20c8109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 --2- 172.21.15.175:0/2775632938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c81098c0 0x7f20c8109ca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009900 tx=0x7f20b802a400 comp rx=0 tx=0).stop 2024-07-29T00:44:53.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 -- 172.21.15.175:0/2775632938 shutdown_connections 2024-07-29T00:44:53.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 --2- 172.21.15.175:0/2775632938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8104260 0x7f20c8110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 --2- 172.21.15.175:0/2775632938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 --2- 172.21.15.175:0/2775632938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c81098c0 0x7f20c8109ca0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 -- 172.21.15.175:0/2775632938 >> 172.21.15.175:0/2775632938 conn(0x7f20c80ff5c0 msgr2=0x7f20c81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:53.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 -- 172.21.15.175:0/2775632938 shutdown_connections 2024-07-29T00:44:53.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.473+0000 7f20cd600640 1 -- 172.21.15.175:0/2775632938 wait complete. 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.475+0000 7f20cd600640 1 Processor -- start 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.475+0000 7f20cd600640 1 -- start start 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20cd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20cd600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c8104260 0x7f20c819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20cd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c81098c0 0x7f20c819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20cd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8113d10 con 0x7f20c81098c0 2024-07-29T00:44:53.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20cd600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8113b90 con 0x7f20c8104260 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20cd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8113e90 con 0x7f20c81038c0 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42834/0 (socket says 172.21.15.175:42834) 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 -- 172.21.15.175:0/4272937222 learned_addr learned my addr 172.21.15.175:0/4272937222 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c81098c0 0x7f20c819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20bfe00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c8104260 0x7f20c819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 -- 172.21.15.175:0/4272937222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c8104260 msgr2=0x7f20c819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c8104260 0x7f20c819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 -- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c81098c0 msgr2=0x7f20c819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.476+0000 7f20c7e00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c81098c0 0x7f20c819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20c7e00640 1 -- 172.21.15.175:0/4272937222 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c819f1c0 con 0x7f20c81038c0 2024-07-29T00:44:53.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20ccc00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c81098c0 0x7f20c819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:53.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20c7e00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c819a210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f20b802a9f0 tx=0x7f20b8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:53.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20c5600640 1 -- 172.21.15.175:0/4272937222 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b803f070 con 0x7f20c81038c0 2024-07-29T00:44:53.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20c5600640 1 -- 172.21.15.175:0/4272937222 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b8031c30 con 0x7f20c81038c0 2024-07-29T00:44:53.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c819f450 con 0x7f20c81038c0 2024-07-29T00:44:53.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20c5600640 1 -- 172.21.15.175:0/4272937222 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8035030 con 0x7f20c81038c0 2024-07-29T00:44:53.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.477+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c81a6d90 con 0x7f20c81038c0 2024-07-29T00:44:53.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.480+0000 7f20c5600640 1 -- 172.21.15.175:0/4272937222 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f20b8042070 con 0x7f20c81038c0 2024-07-29T00:44:53.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 Processor -- start 2024-07-29T00:44:53.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 -- start start 2024-07-29T00:44:53.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x56069784e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x56069784efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069785bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5606977b4ab0 con 0x5606978500e0 2024-07-29T00:44:53.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56069774ca80 con 0x56069784eb70 2024-07-29T00:44:53.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20d185f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5606977b0a70 con 0x560697859910 2024-07-29T00:44:53.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x56069784e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x56069784e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57768/0 (socket says 172.21.15.175:57768) 2024-07-29T00:44:53.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.481+0000 7f20ccc00640 1 -- 172.21.15.175:0/135819504 learned_addr learned my addr 172.21.15.175:0/135819504 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:53.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069785bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20c7e00640 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x56069784efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 -- 172.21.15.175:0/135819504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 msgr2=0x56069784efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x56069784efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.496 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 -- 172.21.15.175:0/135819504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 msgr2=0x56069784e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x56069784e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 -- 172.21.15.175:0/135819504 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606975263c0 con 0x560697859910 2024-07-29T00:44:53.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bfe00640 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069785bd00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009bf0 tx=0x7f20b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8889ea020a9f022d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:53.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.482+0000 7f20bd200640 1 -- 172.21.15.175:0/135819504 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b001e070 con 0x560697859910 2024-07-29T00:44:53.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20bd200640 1 -- 172.21.15.175:0/135819504 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b0009da0 con 0x560697859910 2024-07-29T00:44:53.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20bd200640 1 -- 172.21.15.175:0/135819504 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0018870 con 0x560697859910 2024-07-29T00:44:53.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20c5600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f20ac0a2d70 0x7f20ac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20c5600640 1 -- 172.21.15.175:0/4272937222 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f20b80efa80 con 0x7f20c81038c0 2024-07-29T00:44:53.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20ccc00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f20ac0a2d70 0x7f20ac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20d185f740 1 -- 172.21.15.175:0/135819504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 msgr2=0x56069785bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.483+0000 7f20d185f740 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069785bd00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009bf0 tx=0x7f20b0011810 comp rx=0 tx=0).stop 2024-07-29T00:44:53.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 -- 172.21.15.175:0/135819504 shutdown_connections 2024-07-29T00:44:53.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069785bd00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x56069784efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 --2- 172.21.15.175:0/135819504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x56069784e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 -- 172.21.15.175:0/135819504 >> 172.21.15.175:0/135819504 conn(0x560697817950 msgr2=0x56069784b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 -- 172.21.15.175:0/135819504 shutdown_connections 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 -- 172.21.15.175:0/135819504 wait complete. 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20ccc00640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f20ac0a2d70 0x7f20ac0a5230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20c00099a0 tx=0x7f20c0009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.484+0000 7f20d185f740 1 -- start start 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 Processor -- start 2024-07-29T00:44:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697856e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x560697857390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56069785c6b0 con 0x5606978500e0 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56069776fc10 con 0x56069784eb70 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56069785c830 con 0x560697859910 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20d185f740 10 client.0 _ll_register_callbacks cb 0x560697738a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x560697857390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x560697857390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42848/0 (socket says 172.21.15.175:42848) 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.485+0000 7f20ccc00640 1 -- 172.21.15.175:0/2748439138 learned_addr learned my addr 172.21.15.175:0/2748439138 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20c7e00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20bfe00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697856e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20ccc00640 1 -- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 msgr2=0x560697856910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20ccc00640 1 -- 172.21.15.175:0/2748439138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 msgr2=0x560697856e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697856e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20ccc00640 1 -- 172.21.15.175:0/2748439138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606977731e0 con 0x560697859910 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20c7e00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x560697857390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f20c0010bd0 tx=0x7f20c0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ad4e64073942a0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.486+0000 7f20b5600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20b5600640 1 -- 172.21.15.175:0/2748439138 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c001fd40 con 0x560697859910 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20b5600640 1 -- 172.21.15.175:0/2748439138 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f20c001c3f0 con 0x560697859910 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b0009da0 con 0x560697859910 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20b5600640 1 -- 172.21.15.175:0/2748439138 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f20c001c5f0 con 0x560697859910 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20d185f740 10 client.27022 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:53.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560697870d30 con 0x560697859910 2024-07-29T00:44:53.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20b5600640 1 -- 172.21.15.175:0/2748439138 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b0009da0 con 0x560697859910 2024-07-29T00:44:53.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.487+0000 7f20b5600640 1 -- 172.21.15.175:0/2748439138 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f20c0026070 con 0x560697859910 2024-07-29T00:44:53.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20d185f740 10 client.27022 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:53.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20d185f740 10 client.27022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:44:53.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20d185f740 20 client.27022 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:53.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20d185f740 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978721f0 0x560697874660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20d185f740 4 client.27022 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606977c2020 con 0x5606978721f0 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.488+0000 7f20c7e00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978721f0 0x560697874660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.489+0000 7f20c7e00640 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978721f0 0x560697874660 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b896772f5a7e8cc3 server_cookie=a7b6c73b82cbc2ec in_seq=0 out_seq=0 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.489+0000 7f20b5600640 10 client.27022 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.489+0000 7f20b5600640 1 -- 172.21.15.175:0/2748439138 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5606977c2020 con 0x5606978721f0 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.489+0000 7f20b5600640 10 client.27022 handle_command_reply: tid=0 2024-07-29T00:44:53.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.489+0000 7f20d185f740 1 client.27022 shutdown 2024-07-29T00:44:53.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 msgr2=0x560697857390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x560697857390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f20c0010bd0 tx=0x7f20c0004ab0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 shutdown_connections 2024-07-29T00:44:53.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x560697857390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697856e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.507 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978721f0 0x560697874660 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 --2- 172.21.15.175:0/2748439138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20b5600640 0 client.27022 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:53.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 >> 172.21.15.175:0/2748439138 conn(0x560697817950 msgr2=0x560697850f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:53.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 shutdown_connections 2024-07-29T00:44:53.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 1 -- 172.21.15.175:0/2748439138 wait complete. 2024-07-29T00:44:53.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.490+0000 7f20d185f740 20 client.27022 trim_cache size 0 max 16384 2024-07-29T00:44:53.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 Processor -- start 2024-07-29T00:44:53.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 -- start start 2024-07-29T00:44:53.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697879f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069787a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ab340 con 0x5606978500e0 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ace80 con 0x56069784eb70 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20d185f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81aba10 con 0x560697859910 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697879f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697879f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57776/0 (socket says 172.21.15.175:57776) 2024-07-29T00:44:53.514 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20bfe00640 1 -- 172.21.15.175:0/2389488175 learned_addr learned my addr 172.21.15.175:0/2389488175 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.505+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069787a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20c7e00640 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20ccc00640 1 -- 172.21.15.175:0/2389488175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 msgr2=0x560697856fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20ccc00640 1 -- 172.21.15.175:0/2389488175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 msgr2=0x560697879f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697879f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20ccc00640 1 -- 172.21.15.175:0/2389488175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069784bf00 con 0x560697859910 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20bfe00640 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697879f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:53.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20ccc00640 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069787a4c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f20c0011f40 tx=0x7f20c0010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15da21f2706d3faa server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20b7e00640 1 -- 172.21.15.175:0/2389488175 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0028bf0 con 0x560697859910 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20b7e00640 1 -- 172.21.15.175:0/2389488175 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0028d90 con 0x560697859910 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.506+0000 7f20b7e00640 1 -- 172.21.15.175:0/2389488175 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c002f660 con 0x560697859910 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 -- 172.21.15.175:0/2389488175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 msgr2=0x56069787a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069787a4c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f20c0011f40 tx=0x7f20c0010fd0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 -- 172.21.15.175:0/2389488175 shutdown_connections 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560697859910 0x56069787a4c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606978500e0 0x560697879f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 --2- 172.21.15.175:0/2389488175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x560697856fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 -- 172.21.15.175:0/2389488175 >> 172.21.15.175:0/2389488175 conn(0x560697817950 msgr2=0x5606978568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 -- 172.21.15.175:0/2389488175 shutdown_connections 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.507+0000 7f20d185f740 1 -- 172.21.15.175:0/2389488175 wait complete. 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 -- start start 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 Processor -- start 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x5606978653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606978500e0 0x560697865930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 0x5606978daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ac7b0 con 0x560697859910 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ac0e0 con 0x56069784eb70 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81a9340 con 0x5606978500e0 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.508+0000 7f20d185f740 10 client.0 _ll_register_callbacks cb 0x7f20c0009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 0x5606978daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 0x5606978daf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57780/0 (socket says 172.21.15.175:57780) 2024-07-29T00:44:53.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 -- 172.21.15.175:0/1065417276 learned_addr learned my addr 172.21.15.175:0/1065417276 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:53.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20c7e00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x5606978653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20bfe00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606978500e0 0x560697865930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 -- 172.21.15.175:0/1065417276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606978500e0 msgr2=0x560697865930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606978500e0 0x560697865930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 -- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 msgr2=0x5606978653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:53.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x5606978653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20ccc00640 1 -- 172.21.15.175:0/1065417276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606978661d0 con 0x560697859910 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.509+0000 7f20c7e00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x5606978653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20ccc00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 0x5606978daf40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f20c0021040 tx=0x7f20c0028860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ce56d2b815dbb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20b5600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20b5600640 1 -- 172.21.15.175:0/1065417276 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c003f070 con 0x560697859910 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20b5600640 1 -- 172.21.15.175:0/1065417276 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f20c001c9a0 con 0x560697859910 2024-07-29T00:44:53.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c0028d90 con 0x560697859910 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20b5600640 1 -- 172.21.15.175:0/1065417276 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f20c004a050 con 0x560697859910 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20d185f740 10 client.16932 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.510+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5606978dc550 con 0x560697859910 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20b5600640 1 -- 172.21.15.175:0/1065417276 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0028d90 con 0x560697859910 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20b5600640 1 -- 172.21.15.175:0/1065417276 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f20c0037020 con 0x560697859910 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20d185f740 10 client.16932 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20d185f740 10 client.16932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20d185f740 20 client.16932 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:53.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20d185f740 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978dcb60 0x5606978def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20d185f740 4 client.16932 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606977bc530 con 0x5606978dcb60 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.511+0000 7f20c7e00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978dcb60 0x5606978def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20c7e00640 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978dcb60 0x5606978def50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f89f0db34226124 server_cookie=f4ebce07769b3208 in_seq=0 out_seq=0 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20b5600640 10 client.16932 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20b5600640 1 -- 172.21.15.175:0/1065417276 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606977bc530 con 0x5606978dcb60 2024-07-29T00:44:53.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20b5600640 10 client.16932 handle_command_reply: tid=0 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 client.16932 shutdown 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 msgr2=0x5606978daf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 0x5606978daf40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f20c0021040 tx=0x7f20c0028860 comp rx=0 tx=0).stop 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 shutdown_connections 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560697859910 0x5606978daf40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606978500e0 0x560697865930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5606978dcb60 0x5606978def50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 --2- 172.21.15.175:0/1065417276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069784eb70 0x5606978653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20b5600640 0 client.16932 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:44:53.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 >> 172.21.15.175:0/1065417276 conn(0x560697817950 msgr2=0x560697850ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:53.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 shutdown_connections 2024-07-29T00:44:53.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.512+0000 7f20d185f740 1 -- 172.21.15.175:0/1065417276 wait complete. 2024-07-29T00:44:53.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20d185f740 20 client.16932 trim_cache size 0 max 16384 2024-07-29T00:44:53.524 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:53.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f20ac0a2d70 msgr2=0x7f20ac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f20ac0a2d70 0x7f20ac0a5230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20c00099a0 tx=0x7f20c0009500 comp rx=0 tx=0).stop 2024-07-29T00:44:53.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 msgr2=0x7f20c819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:53.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c819a210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f20b802a9f0 tx=0x7f20b8003c10 comp rx=0 tx=0).stop 2024-07-29T00:44:53.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 shutdown_connections 2024-07-29T00:44:53.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f20ac0a2d70 0x7f20ac0a5230 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c81098c0 0x7f20c819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c8104260 0x7f20c819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 --2- 172.21.15.175:0/4272937222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c81038c0 0x7f20c819a210 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:53.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.513+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 >> 172.21.15.175:0/4272937222 conn(0x7f20c80ff5c0 msgr2=0x7f20c8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:53.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.514+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 shutdown_connections 2024-07-29T00:44:53.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:53.514+0000 7f20cd600640 1 -- 172.21.15.175:0/4272937222 wait complete. 2024-07-29T00:44:54.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:53 smithi204 ceph-mon[37368]: pgmap v258: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:54.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:53 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:44:54.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:53 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:44:54.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:53 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:44:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:53 smithi181 ceph-mon[36706]: pgmap v258: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:53 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:44:54.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:53 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:44:54.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:53 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:44:54.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:53 smithi175 ceph-mon[32138]: pgmap v258: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:54.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:53 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:44:54.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:53 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:44:54.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:53 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:44:54.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:44:55.057 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:55.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.749+0000 7f75a2c00640 1 -- 172.21.15.175:0/3126850920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 msgr2=0x7f759c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.749+0000 7f75a2c00640 1 --2- 172.21.15.175:0/3126850920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c108130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7588009930 tx=0x7f758802a440 comp rx=0 tx=0).stop 2024-07-29T00:44:55.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 -- 172.21.15.175:0/3126850920 shutdown_connections 2024-07-29T00:44:55.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 --2- 172.21.15.175:0/3126850920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c108670 0x7f759c114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 --2- 172.21.15.175:0/3126850920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c108130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 --2- 172.21.15.175:0/3126850920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c10dd50 0x7f759c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 -- 172.21.15.175:0/3126850920 >> 172.21.15.175:0/3126850920 conn(0x7f759c077670 msgr2=0x7f759c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:55.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 -- 172.21.15.175:0/3126850920 shutdown_connections 2024-07-29T00:44:55.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.750+0000 7f75a2c00640 1 -- 172.21.15.175:0/3126850920 wait complete. 2024-07-29T00:44:55.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.751+0000 7f75a2c00640 1 Processor -- start 2024-07-29T00:44:55.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.751+0000 7f75a2c00640 1 -- start start 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 0x7f759c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c10dd50 0x7f759c1a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f759c118070 con 0x7f759c107d10 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f759c117ef0 con 0x7f759c10dd50 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1181f0 con 0x7f759c108670 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 0x7f759c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57808/0 (socket says 172.21.15.175:57808) 2024-07-29T00:44:55.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 0x7f759c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42878/0 (socket says 172.21.15.175:42878) 2024-07-29T00:44:55.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a1800640 1 -- 172.21.15.175:0/2805435805 learned_addr learned my addr 172.21.15.175:0/2805435805 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:55.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.752+0000 7f75a2200640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c10dd50 0x7f759c1a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a0e00640 1 -- 172.21.15.175:0/2805435805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c10dd50 msgr2=0x7f759c1a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a0e00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c10dd50 0x7f759c1a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a0e00640 1 -- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 msgr2=0x7f759c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a0e00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a0e00640 1 -- 172.21.15.175:0/2805435805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c1a3560 con 0x7f759c108670 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a2200640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c10dd50 0x7f759c1a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a1800640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a0e00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 0x7f759c19eaf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7588005940 tx=0x7f75880040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f759a000640 1 -- 172.21.15.175:0/2805435805 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758803f070 con 0x7f759c108670 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.753+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c1a37f0 con 0x7f759c108670 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.754+0000 7f759a000640 1 -- 172.21.15.175:0/2805435805 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75880044c0 con 0x7f759c108670 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.754+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1ab130 con 0x7f759c108670 2024-07-29T00:44:55.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.754+0000 7f759a000640 1 -- 172.21.15.175:0/2805435805 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588048060 con 0x7f759c108670 2024-07-29T00:44:55.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f759a000640 1 -- 172.21.15.175:0/2805435805 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f758803b460 con 0x7f759c108670 2024-07-29T00:44:55.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 Processor -- start 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 -- start start 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 0x564ef238d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef238dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ef22f3ab0 con 0x564ef238f0e0 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564ef228ba80 con 0x564ef238db70 2024-07-29T00:44:55.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.757+0000 7f75a6deb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564ef22efa70 con 0x564ef2398910 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 0x564ef238d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 0x564ef238d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57824/0 (socket says 172.21.15.175:57824) 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a2200640 1 -- 172.21.15.175:0/1529975111 learned_addr learned my addr 172.21.15.175:0/1529975111 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a1800640 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef238dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a0e00640 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a1800640 1 -- 172.21.15.175:0/1529975111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 msgr2=0x564ef239ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a1800640 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a1800640 1 -- 172.21.15.175:0/1529975111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 msgr2=0x564ef238d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a1800640 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 0x564ef238d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a1800640 1 -- 172.21.15.175:0/1529975111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef20653c0 con 0x564ef238db70 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.758+0000 7f75a2200640 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 0x564ef238d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f759a000640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75840a2cf0 0x7f75840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f75a1800640 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef238dfb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f75900114f0 tx=0x7f75900119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d17e0ba20e8d218 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f759a000640 1 -- 172.21.15.175:0/2805435805 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f75880efdb0 con 0x7f759c108670 2024-07-29T00:44:55.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f75a2200640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75840a2cf0 0x7f75840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f758ea00640 1 -- 172.21.15.175:0/1529975111 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759001e070 con 0x564ef238db70 2024-07-29T00:44:55.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f758ea00640 1 -- 172.21.15.175:0/1529975111 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759000fb60 con 0x564ef238db70 2024-07-29T00:44:55.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.759+0000 7f758ea00640 1 -- 172.21.15.175:0/1529975111 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590019c90 con 0x564ef238db70 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 -- 172.21.15.175:0/1529975111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 msgr2=0x564ef238dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a2200640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75840a2cf0 0x7f75840a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75940127a0 tx=0x7f7594009930 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef238dfb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f75900114f0 tx=0x7f75900119b0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 -- 172.21.15.175:0/1529975111 shutdown_connections 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef238dfb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1529975111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238f0e0 0x564ef238d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 -- 172.21.15.175:0/1529975111 >> 172.21.15.175:0/1529975111 conn(0x564ef2356950 msgr2=0x564ef238a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 -- 172.21.15.175:0/1529975111 shutdown_connections 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.760+0000 7f75a6deb740 1 -- 172.21.15.175:0/1529975111 wait complete. 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 -- start start 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 Processor -- start 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238db70 0x564ef2395900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.768 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef2395e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2396380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ef239b6b0 con 0x564ef2398910 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564ef22aec10 con 0x564ef238f0e0 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.761+0000 7f75a6deb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564ef239b830 con 0x564ef238db70 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a6deb740 10 client.0 _ll_register_callbacks cb 0x564ef2277a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef2395e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238db70 0x564ef2395900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.769 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef2395e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38764/0 (socket says 172.21.15.175:38764) 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a0e00640 1 -- 172.21.15.175:0/1878334247 learned_addr learned my addr 172.21.15.175:0/1878334247 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a2200640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2396380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 -- 172.21.15.175:0/1878334247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 msgr2=0x564ef2395e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef2395e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 -- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 msgr2=0x564ef2396380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2396380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 -- 172.21.15.175:0/1878334247 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef22b21e0 con 0x564ef238db70 2024-07-29T00:44:55.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a2200640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2396380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a0e00640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef2395e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f75a1800640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238db70 0x564ef2395900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f75900114c0 tx=0x7f759000ff40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee30332d845dd54 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.762+0000 7f7583400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f7583400640 1 -- 172.21.15.175:0/1878334247 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759001e040 con 0x564ef238db70 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759000fb60 con 0x564ef238db70 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f7583400640 1 -- 172.21.15.175:0/1878334247 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f759000f740 con 0x564ef238db70 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f7583400640 1 -- 172.21.15.175:0/1878334247 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7590032050 con 0x564ef238db70 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f75a6deb740 10 client.27040 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:55.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564ef23a74a0 con 0x564ef238db70 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.763+0000 7f7583400640 1 -- 172.21.15.175:0/1878334247 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759000fb60 con 0x564ef238db70 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f7583400640 1 -- 172.21.15.175:0/1878334247 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f75900046c0 con 0x564ef238db70 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a6deb740 10 client.27040 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a6deb740 10 client.27040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a6deb740 20 client.27040 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef23a8850 0x564ef23aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a6deb740 4 client.27040 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:55.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef2301020 con 0x564ef23a8850 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.764+0000 7f75a0e00640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef23a8850 0x564ef23aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.765+0000 7f75a0e00640 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef23a8850 0x564ef23aac40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cba54bdcf39d6643 server_cookie=16501d82f242e968 in_seq=0 out_seq=0 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.765+0000 7f7583400640 10 client.27040 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.765+0000 7f7583400640 1 -- 172.21.15.175:0/1878334247 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ef2301020 con 0x564ef23a8850 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.765+0000 7f7583400640 10 client.27040 handle_command_reply: tid=0 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.765+0000 7f75a6deb740 1 client.27040 shutdown 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238db70 msgr2=0x564ef2395900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238db70 0x564ef2395900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f75900114c0 tx=0x7f759000ff40 comp rx=0 tx=0).stop 2024-07-29T00:44:55.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 shutdown_connections 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef23a8850 0x564ef23aac40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2396380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef2395e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 --2- 172.21.15.175:0/1878334247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238db70 0x564ef2395900 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 >> 172.21.15.175:0/1878334247 conn(0x564ef2356950 msgr2=0x564ef238fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f7583400640 0 client.27040 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 shutdown_connections 2024-07-29T00:44:55.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 1 -- 172.21.15.175:0/1878334247 wait complete. 2024-07-29T00:44:55.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.766+0000 7f75a6deb740 20 client.27040 trim_cache size 0 max 16384 2024-07-29T00:44:55.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.780+0000 7f75a6deb740 1 Processor -- start 2024-07-29T00:44:55.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 -- start start 2024-07-29T00:44:55.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef23b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238f0e0 0x564ef23b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2388b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1b3ae0 con 0x564ef2398910 2024-07-29T00:44:55.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1b5620 con 0x564ef238db70 2024-07-29T00:44:55.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a6deb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1b41b0 con 0x564ef238f0e0 2024-07-29T00:44:55.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2388b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238f0e0 0x564ef23b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2388b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57848/0 (socket says 172.21.15.175:57848) 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a2200640 1 -- 172.21.15.175:0/3692272880 learned_addr learned my addr 172.21.15.175:0/3692272880 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.781+0000 7f75a1800640 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef23b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f75a0e00640 1 -- 172.21.15.175:0/3692272880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 msgr2=0x564ef23b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef23b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f75a0e00640 1 -- 172.21.15.175:0/3692272880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 msgr2=0x564ef2388b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2388b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f75a0e00640 1 -- 172.21.15.175:0/3692272880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef23ab1d0 con 0x564ef238f0e0 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238f0e0 0x564ef23b0940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f75880bc1b0 tx=0x7f75880f2270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a4ccfa8a88bfff server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f758e000640 1 -- 172.21.15.175:0/3692272880 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588035040 con 0x564ef238f0e0 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f758e000640 1 -- 172.21.15.175:0/3692272880 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75880bc920 con 0x564ef238f0e0 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.782+0000 7f758e000640 1 -- 172.21.15.175:0/3692272880 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880dae60 con 0x564ef238f0e0 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 -- 172.21.15.175:0/3692272880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238f0e0 msgr2=0x564ef23b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238f0e0 0x564ef23b0940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f75880bc1b0 tx=0x7f75880f2270 comp rx=0 tx=0).stop 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 -- 172.21.15.175:0/3692272880 shutdown_connections 2024-07-29T00:44:55.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef2398910 0x564ef2388b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef238f0e0 0x564ef23b0940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3692272880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238db70 0x564ef23b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 -- 172.21.15.175:0/3692272880 >> 172.21.15.175:0/3692272880 conn(0x564ef2356950 msgr2=0x564ef23960e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:55.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 -- 172.21.15.175:0/3692272880 shutdown_connections 2024-07-29T00:44:55.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.783+0000 7f75a6deb740 1 -- 172.21.15.175:0/3692272880 wait complete. 2024-07-29T00:44:55.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.784+0000 7f75a6deb740 1 -- start start 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.784+0000 7f75a6deb740 1 Processor -- start 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 0x564ef23a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef23a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1b4f50 con 0x564ef238db70 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1b4880 con 0x564ef238f0e0 2024-07-29T00:44:55.790 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1b1ae0 con 0x564ef2398910 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a6deb740 10 client.0 _ll_register_callbacks cb 0x7f759001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 0x564ef23a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 0x564ef23a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57856/0 (socket says 172.21.15.175:57856) 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a1800640 1 -- 172.21.15.175:0/3979008828 learned_addr learned my addr 172.21.15.175:0/3979008828 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef23a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a2200640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a0e00640 1 -- 172.21.15.175:0/3979008828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 msgr2=0x564ef239c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.791 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a0e00640 1 -- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 msgr2=0x564ef23a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 0x564ef23a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a0e00640 1 -- 172.21.15.175:0/3979008828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef23906c0 con 0x564ef238f0e0 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.785+0000 7f75a1800640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 0x564ef23a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f75a0e00640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef23a10c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x564ef2396b50 tx=0x7f758802a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53fe9e12127e43f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f7583400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f7583400640 1 -- 172.21.15.175:0/3979008828 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588035070 con 0x564ef238f0e0 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f7583400640 1 -- 172.21.15.175:0/3979008828 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f75880ed6d0 con 0x564ef238f0e0 2024-07-29T00:44:55.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75880bc920 con 0x564ef238f0e0 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f7583400640 1 -- 172.21.15.175:0/3979008828 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75880dae60 con 0x564ef238f0e0 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f75a6deb740 10 client.27098 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.786+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564ef238af00 con 0x564ef238f0e0 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f7583400640 1 -- 172.21.15.175:0/3979008828 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75880bc920 con 0x564ef238f0e0 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f7583400640 1 -- 172.21.15.175:0/3979008828 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f75880dc0b0 con 0x564ef238f0e0 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a6deb740 10 client.27098 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a6deb740 10 client.27098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:44:55.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a6deb740 20 client.27098 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef2412fb0 0x564ef24153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a6deb740 4 client.27098 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ef22fb530 con 0x564ef2412fb0 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.787+0000 7f75a1800640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef2412fb0 0x564ef24153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a1800640 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef2412fb0 0x564ef24153d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ae20bd8d1699015 server_cookie=7799f9425d10d7a6 in_seq=0 out_seq=0 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f7583400640 10 client.27098 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:55.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f7583400640 1 -- 172.21.15.175:0/3979008828 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ef22fb530 con 0x564ef2412fb0 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f7583400640 10 client.27098 handle_command_reply: tid=0 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 client.27098 shutdown 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 msgr2=0x564ef23a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef23a10c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x564ef2396b50 tx=0x7f758802a440 comp rx=0 tx=0).stop 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 shutdown_connections 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564ef2412fb0 0x564ef24153d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ef2398910 0x564ef239c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.795 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ef238f0e0 0x564ef23a10c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 --2- 172.21.15.175:0/3979008828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ef238db70 0x564ef23a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f7583400640 0 client.27098 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.788+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 >> 172.21.15.175:0/3979008828 conn(0x564ef2356950 msgr2=0x564ef238f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 shutdown_connections 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a6deb740 1 -- 172.21.15.175:0/3979008828 wait complete. 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a6deb740 20 client.27098 trim_cache size 0 max 16384 2024-07-29T00:44:55.796 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:55.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75840a2cf0 msgr2=0x7f75840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a2c00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75840a2cf0 0x7f75840a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75940127a0 tx=0x7f7594009930 comp rx=0 tx=0).stop 2024-07-29T00:44:55.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 msgr2=0x7f759c19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.789+0000 7f75a2c00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 0x7f759c19eaf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7588005940 tx=0x7f75880040c0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 shutdown_connections 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f759c10dd50 0x7f759c1a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c108670 0x7f759c19eaf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f75840a2cf0 0x7f75840a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 --2- 172.21.15.175:0/2805435805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f759c107d10 0x7f759c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:55.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 >> 172.21.15.175:0/2805435805 conn(0x7f759c077670 msgr2=0x7f759c078f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:55.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 shutdown_connections 2024-07-29T00:44:55.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:55.790+0000 7f75a2c00640 1 -- 172.21.15.175:0/2805435805 wait complete. 2024-07-29T00:44:56.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:55 smithi175 ceph-mon[32138]: pgmap v259: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:56.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:55 smithi204 ceph-mon[37368]: pgmap v259: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:55 smithi181 ceph-mon[36706]: pgmap v259: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:44:56.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:44:57.169 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:44:57.864 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 -- 172.21.15.175:0/321613052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 msgr2=0x7f6da8114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 --2- 172.21.15.175:0/321613052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da8114b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009cd0 tx=0x7f6da402a8f0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 -- 172.21.15.175:0/321613052 shutdown_connections 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 --2- 172.21.15.175:0/321613052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da8114b90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 --2- 172.21.15.175:0/321613052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 0x7f6da8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 --2- 172.21.15.175:0/321613052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da810dd50 0x7f6da810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 -- 172.21.15.175:0/321613052 >> 172.21.15.175:0/321613052 conn(0x7f6da8077670 msgr2=0x7f6da8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 -- 172.21.15.175:0/321613052 shutdown_connections 2024-07-29T00:44:57.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.864+0000 7f6daf200640 1 -- 172.21.15.175:0/321613052 wait complete. 2024-07-29T00:44:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.865+0000 7f6daf200640 1 Processor -- start 2024-07-29T00:44:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.865+0000 7f6daf200640 1 -- start start 2024-07-29T00:44:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6daf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 0x7f6da819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6daf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6daf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da810dd50 0x7f6da81a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6daf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6da8118060 con 0x7f6da8108670 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6daf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6da8117ee0 con 0x7f6da8107d10 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6daf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81181e0 con 0x7f6da810dd50 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dade00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 0x7f6da819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57866/0 (socket says 172.21.15.175:57866) 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dad400640 1 -- 172.21.15.175:0/1496729733 learned_addr learned my addr 172.21.15.175:0/1496729733 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dae800640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da810dd50 0x7f6da81a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dade00640 1 -- 172.21.15.175:0/1496729733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da810dd50 msgr2=0x7f6da81a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dade00640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da810dd50 0x7f6da81a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dade00640 1 -- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 msgr2=0x7f6da819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dade00640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.866+0000 7f6dade00640 1 -- 172.21.15.175:0/1496729733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da81a34d0 con 0x7f6da8107d10 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.867+0000 7f6dade00640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 0x7f6da819e520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00e9f0 tx=0x7f6d9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.867+0000 7f6da2a00640 1 -- 172.21.15.175:0/1496729733 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00ccc0 con 0x7f6da8107d10 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.867+0000 7f6da2a00640 1 -- 172.21.15.175:0/1496729733 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9c010910 con 0x7f6da8107d10 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.867+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81a37c0 con 0x7f6da8107d10 2024-07-29T00:44:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.867+0000 7f6da2a00640 1 -- 172.21.15.175:0/1496729733 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c018960 con 0x7f6da8107d10 2024-07-29T00:44:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.867+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da81ab0a0 con 0x7f6da8107d10 2024-07-29T00:44:57.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.870+0000 7f6da2a00640 1 -- 172.21.15.175:0/1496729733 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6d9c010ac0 con 0x7f6da8107d10 2024-07-29T00:44:57.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.870+0000 7f6db33d6740 1 Processor -- start 2024-07-29T00:44:57.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 -- start start 2024-07-29T00:44:57.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d3ac11bab0 con 0x55d3ac1c0910 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d3ac0b3a80 con 0x55d3ac1b70e0 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6db33d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3ac117a70 con 0x55d3ac1b5b70 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6dade00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6dae800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.871+0000 7f6dae800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38814/0 (socket says 172.21.15.175:38814) 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57870/0 (socket says 172.21.15.175:57870) 2024-07-29T00:44:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dade00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42942/0 (socket says 172.21.15.175:42942) 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 -- 172.21.15.175:0/2630247515 learned_addr learned my addr 172.21.15.175:0/2630247515 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 -- 172.21.15.175:0/2630247515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b5b70 msgr2=0x55d3ac1b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 -- 172.21.15.175:0/2630247515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 msgr2=0x55d3ac1c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 -- 172.21.15.175:0/2630247515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3abe8d3c0 con 0x55d3ac1b70e0 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6da2a00640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d900a2cf0 0x7f6d900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.872+0000 7f6dae800640 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1b5630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009380 tx=0x7f6da40093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2e45d1131c2aaf server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:57.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.873+0000 7f6da2a00640 1 -- 172.21.15.175:0/1496729733 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f6d9c0cf820 con 0x7f6da8107d10 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.873+0000 7f6dade00640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d900a2cf0 0x7f6d900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.873+0000 7f6d97000640 1 -- 172.21.15.175:0/2630247515 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4030690 con 0x55d3ac1b70e0 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.873+0000 7f6d97000640 1 -- 172.21.15.175:0/2630247515 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da4036070 con 0x55d3ac1b70e0 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.873+0000 7f6d97000640 1 -- 172.21.15.175:0/2630247515 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4004430 con 0x55d3ac1b70e0 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 -- 172.21.15.175:0/2630247515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 msgr2=0x55d3ac1b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6dade00640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d900a2cf0 0x7f6d900a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55d3ac1b3060 tx=0x7f6d9c0d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1b5630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009380 tx=0x7f6da40093b0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 -- 172.21.15.175:0/2630247515 shutdown_connections 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2630247515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1b5630 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 -- 172.21.15.175:0/2630247515 >> 172.21.15.175:0/2630247515 conn(0x55d3ac17e950 msgr2=0x55d3ac1b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 -- 172.21.15.175:0/2630247515 shutdown_connections 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 -- 172.21.15.175:0/2630247515 wait complete. 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.874+0000 7f6db33d6740 1 -- start start 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 Processor -- start 2024-07-29T00:44:57.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1bde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d3ac1c3650 con 0x55d3ac1b70e0 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d3ac0d6c10 con 0x55d3ac1b5b70 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d3ac1c37d0 con 0x55d3ac1c0910 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6db33d6740 10 client.0 _ll_register_callbacks cb 0x55d3ac09fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6dae800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1bde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57884/0 (socket says 172.21.15.175:57884) 2024-07-29T00:44:57.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.875+0000 7f6dad400640 1 -- 172.21.15.175:0/2006985258 learned_addr learned my addr 172.21.15.175:0/2006985258 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:57.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dae800640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dad400640 1 -- 172.21.15.175:0/2006985258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 msgr2=0x55d3ac1be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dad400640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dad400640 1 -- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 msgr2=0x55d3ac1bd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dad400640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1bd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dad400640 1 -- 172.21.15.175:0/2006985258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ac0da1e0 con 0x55d3ac1b70e0 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dae800640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6dad400640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1bde00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d98012750 tx=0x7f6d98012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd3f0374933498c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6d8b400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:44:57.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6d8b400640 1 -- 172.21.15.175:0/2006985258 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98012e60 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6d8b400640 1 -- 172.21.15.175:0/2006985258 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f6d98004050 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.876+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da4036070 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6d8b400640 1 -- 172.21.15.175:0/2006985258 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6d98004250 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6db33d6740 10 client.16962 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d3ac1cf460 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6d8b400640 1 -- 172.21.15.175:0/2006985258 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da4036070 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6d8b400640 1 -- 172.21.15.175:0/2006985258 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6d98003b10 con 0x55d3ac1b70e0 2024-07-29T00:44:57.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6db33d6740 10 client.16962 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:57.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.877+0000 7f6db33d6740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:57.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6db33d6740 20 client.16962 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:57.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac1d0920 0x55d3ac1d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6db33d6740 4 client.16962 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:44:57.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3ac129020 con 0x55d3ac1d0920 2024-07-29T00:44:57.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6dae800640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac1d0920 0x55d3ac1d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6dae800640 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac1d0920 0x55d3ac1d2d50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c5db4fb26cb7cf server_cookie=bb5e012943c75fe1 in_seq=0 out_seq=0 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.878+0000 7f6d8b400640 10 client.16962 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.879+0000 7f6d8b400640 1 -- 172.21.15.175:0/2006985258 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3ac129020 con 0x55d3ac1d0920 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.879+0000 7f6d8b400640 10 client.16962 handle_command_reply: tid=0 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.879+0000 7f6db33d6740 1 client.16962 shutdown 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.879+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 msgr2=0x55d3ac1bde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.879+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1bde00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d98012750 tx=0x7f6d98012c10 comp rx=0 tx=0).stop 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 shutdown_connections 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac1d0920 0x55d3ac1d2d50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1bde00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 --2- 172.21.15.175:0/2006985258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 >> 172.21.15.175:0/2006985258 conn(0x55d3ac17e950 msgr2=0x55d3ac1b7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:57.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6d8b400640 0 client.16962 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:57.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 shutdown_connections 2024-07-29T00:44:57.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 1 -- 172.21.15.175:0/2006985258 wait complete. 2024-07-29T00:44:57.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.880+0000 7f6db33d6740 20 client.16962 trim_cache size 0 max 16384 2024-07-29T00:44:57.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.894+0000 7f6db33d6740 1 Processor -- start 2024-07-29T00:44:57.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.894+0000 7f6db33d6740 1 -- start start 2024-07-29T00:44:57.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1d8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1d8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6db33d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b3b20 con 0x55d3ac1c0910 2024-07-29T00:44:57.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6db33d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b5660 con 0x55d3ac1b5b70 2024-07-29T00:44:57.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6db33d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b41f0 con 0x55d3ac1b70e0 2024-07-29T00:44:57.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dae800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dae800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1d8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38826/0 (socket says 172.21.15.175:38826) 2024-07-29T00:44:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dae800640 1 -- 172.21.15.175:0/477889487 learned_addr learned my addr 172.21.15.175:0/477889487 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dad400640 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1d8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dae800640 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dad400640 1 -- 172.21.15.175:0/477889487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 msgr2=0x55d3ac1d8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dad400640 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1d8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dad400640 1 -- 172.21.15.175:0/477889487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 msgr2=0x55d3ac1b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dad400640 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.895+0000 7f6dad400640 1 -- 172.21.15.175:0/477889487 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ac1b62f0 con 0x55d3ac1b70e0 2024-07-29T00:44:57.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.896+0000 7f6dad400640 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1d8ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55d3ac1bee70 tx=0x7f6d9800f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efec642b519aff50 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:57.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.896+0000 7f6d96600640 1 -- 172.21.15.175:0/477889487 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98028040 con 0x55d3ac1b70e0 2024-07-29T00:44:57.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.896+0000 7f6d96600640 1 -- 172.21.15.175:0/477889487 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d98010e50 con 0x55d3ac1b70e0 2024-07-29T00:44:57.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.896+0000 7f6d96600640 1 -- 172.21.15.175:0/477889487 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800fdf0 con 0x55d3ac1b70e0 2024-07-29T00:44:57.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 -- 172.21.15.175:0/477889487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b70e0 msgr2=0x55d3ac1d8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1d8ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55d3ac1bee70 tx=0x7f6d9800f810 comp rx=0 tx=0).stop 2024-07-29T00:44:57.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 -- 172.21.15.175:0/477889487 shutdown_connections 2024-07-29T00:44:57.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1c0910 0x55d3ac1b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1d8ba0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 --2- 172.21.15.175:0/477889487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1d8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 -- 172.21.15.175:0/477889487 >> 172.21.15.175:0/477889487 conn(0x55d3ac17e950 msgr2=0x55d3ac1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 -- 172.21.15.175:0/477889487 shutdown_connections 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 -- 172.21.15.175:0/477889487 wait complete. 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.897+0000 7f6db33d6740 1 -- start start 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 Processor -- start 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b4f90 con 0x55d3ac1b70e0 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b48c0 con 0x55d3ac1b5b70 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b1b20 con 0x55d3ac1c0910 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6db33d6740 10 client.0 _ll_register_callbacks cb 0x7f6d980099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.898+0000 7f6dae800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:42972/0 (socket says 172.21.15.175:42972) 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 -- 172.21.15.175:0/1141254799 learned_addr learned my addr 172.21.15.175:0/1141254799 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57906/0 (socket says 172.21.15.175:57906) 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 -- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 msgr2=0x55d3ac1c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 -- 172.21.15.175:0/1141254799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 msgr2=0x55d3ac1c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 -- 172.21.15.175:0/1141254799 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ac1c4ff0 con 0x55d3ac1c0910 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6dae800640 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c4900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009620 tx=0x7f6da404d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c169a5d2f88e32e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6d8b400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.899+0000 7f6d8b400640 1 -- 172.21.15.175:0/1141254799 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da404e040 con 0x55d3ac1c0910 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d98010e50 con 0x55d3ac1c0910 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6d8b400640 1 -- 172.21.15.175:0/1141254799 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f6da4030930 con 0x55d3ac1c0910 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6d8b400640 1 -- 172.21.15.175:0/1141254799 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6da4030b30 con 0x55d3ac1c0910 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 10 client.27064 fetch_fsmap learned FSMap version 43 2024-07-29T00:44:57.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d3ac23b570 con 0x55d3ac1c0910 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6d8b400640 1 -- 172.21.15.175:0/1141254799 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d98010e50 con 0x55d3ac1c0910 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6d8b400640 1 -- 172.21.15.175:0/1141254799 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6da4054040 con 0x55d3ac1c0910 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 10 client.27064 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 20 client.27064 populate_metadata read hostname 'smithi175' 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac23b860 0x55d3ac23dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 4 client.27064 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:44:57.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.900+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3ac123530 con 0x55d3ac23b860 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.901+0000 7f6dad400640 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac23b860 0x55d3ac23dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.901+0000 7f6dad400640 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac23b860 0x55d3ac23dc50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ae8f5c4bd23098 server_cookie=b0936fd8fcdd220a in_seq=0 out_seq=0 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.901+0000 7f6d8b400640 10 client.27064 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.901+0000 7f6d8b400640 1 -- 172.21.15.175:0/1141254799 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3ac123530 con 0x55d3ac23b860 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.901+0000 7f6d8b400640 10 client.27064 handle_command_reply: tid=0 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.901+0000 7f6db33d6740 1 client.27064 shutdown 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 msgr2=0x55d3ac1c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c4900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009620 tx=0x7f6da404d040 comp rx=0 tx=0).stop 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 shutdown_connections 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55d3ac23b860 0x55d3ac23dc50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3ac1c0910 0x55d3ac1c4900 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3ac1b70e0 0x55d3ac1c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 --2- 172.21.15.175:0/1141254799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3ac1b5b70 0x55d3ac1c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6d8b400640 0 client.27064 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 >> 172.21.15.175:0/1141254799 conn(0x55d3ac17e950 msgr2=0x55d3ac1bf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:57.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 shutdown_connections 2024-07-29T00:44:57.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 1 -- 172.21.15.175:0/1141254799 wait complete. 2024-07-29T00:44:57.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.902+0000 7f6db33d6740 20 client.27064 trim_cache size 0 max 16384 2024-07-29T00:44:57.913 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:44:57.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d900a2cf0 msgr2=0x7f6d900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d900a2cf0 0x7f6d900a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55d3ac1b3060 tx=0x7f6d9c0d5530 comp rx=0 tx=0).stop 2024-07-29T00:44:57.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 msgr2=0x7f6da819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:44:57.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 0x7f6da819e520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00e9f0 tx=0x7f6d9c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 shutdown_connections 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da810dd50 0x7f6da81a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d900a2cf0 0x7f6d900a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6da8108670 0x7f6da819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 --2- 172.21.15.175:0/1496729733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6da8107d10 0x7f6da819e520 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 >> 172.21.15.175:0/1496729733 conn(0x7f6da8077670 msgr2=0x7f6da8078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 shutdown_connections 2024-07-29T00:44:57.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:44:57.903+0000 7f6daf200640 1 -- 172.21.15.175:0/1496729733 wait complete. 2024-07-29T00:44:58.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:57 smithi175 ceph-mon[32138]: pgmap v260: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:57 smithi204 ceph-mon[37368]: pgmap v260: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:58.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:57 smithi181 ceph-mon[36706]: pgmap v260: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:44:59.862 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:44:59.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:45:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:44:59 smithi204 ceph-mon[37368]: pgmap v261: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:00.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:44:59 smithi181 ceph-mon[36706]: pgmap v261: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:00.283 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:44:59 smithi175 ceph-mon[32138]: pgmap v261: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:00.319 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:00.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 -- 172.21.15.175:0/2399193129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c1076a0 msgr2=0x7f073c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:00.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 --2- 172.21.15.175:0/2399193129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c1076a0 0x7f073c107a80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0728009930 tx=0x7f072802a420 comp rx=0 tx=0).stop 2024-07-29T00:45:00.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 -- 172.21.15.175:0/2399193129 shutdown_connections 2024-07-29T00:45:00.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 --2- 172.21.15.175:0/2399193129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c102040 0x7f073c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:00.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 --2- 172.21.15.175:0/2399193129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:00.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 --2- 172.21.15.175:0/2399193129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c1076a0 0x7f073c107a80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:00.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 -- 172.21.15.175:0/2399193129 >> 172.21.15.175:0/2399193129 conn(0x7f073c0fd3a0 msgr2=0x7f073c0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:00.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 -- 172.21.15.175:0/2399193129 shutdown_connections 2024-07-29T00:45:00.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.997+0000 7f0742400640 1 -- 172.21.15.175:0/2399193129 wait complete. 2024-07-29T00:45:01.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.998+0000 7f0742400640 1 Processor -- start 2024-07-29T00:45:01.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.998+0000 7f0742400640 1 -- start start 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0742400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0742400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 0x7f073c074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0742400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c1076a0 0x7f073c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0742400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f073c111aa0 con 0x7f073c102040 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0742400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f073c111920 con 0x7f073c1016a0 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0742400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f073c111c20 con 0x7f073c1076a0 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f073be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 0x7f073c074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0741000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0741a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c1076a0 0x7f073c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f073be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 0x7f073c074720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47180/0 (socket says 172.21.15.175:47180) 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f0741000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c0741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48874/0 (socket says 172.21.15.175:48874) 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:00.999+0000 7f073be00640 1 -- 172.21.15.175:0/622885494 learned_addr learned my addr 172.21.15.175:0/622885494 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741000640 1 -- 172.21.15.175:0/622885494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c1076a0 msgr2=0x7f073c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741000640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c1076a0 0x7f073c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741000640 1 -- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 msgr2=0x7f073c074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741000640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 0x7f073c074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741000640 1 -- 172.21.15.175:0/622885494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c1aa8f0 con 0x7f073c1016a0 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741a00640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c1076a0 0x7f073c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f073be00640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 0x7f073c074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0741000640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c0741e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0728005940 tx=0x7f07280040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0739600640 1 -- 172.21.15.175:0/622885494 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072803f070 con 0x7f073c1016a0 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0739600640 1 -- 172.21.15.175:0/622885494 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07280044c0 con 0x7f073c1016a0 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0739600640 1 -- 172.21.15.175:0/622885494 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728035070 con 0x7f073c1016a0 2024-07-29T00:45:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c1aaac0 con 0x7f073c1016a0 2024-07-29T00:45:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.000+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c1aafb0 con 0x7f073c1016a0 2024-07-29T00:45:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 Processor -- start 2024-07-29T00:45:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f0739600640 1 -- 172.21.15.175:0/622885494 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f072803b460 con 0x7f073c1016a0 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 -- start start 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c8640 0x55bdb45d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45d2f00 0x55bdb45d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdb452d8a0 con 0x55bdb45c7d20 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bdb44c5710 con 0x55bdb45c8640 2024-07-29T00:45:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f07463ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bdb4529860 con 0x55bdb45d2f00 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.004+0000 7f0741a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47196/0 (socket says 172.21.15.175:47196) 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 -- 172.21.15.175:0/2138057128 learned_addr learned my addr 172.21.15.175:0/2138057128 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c8640 0x55bdb45d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f073be00640 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45d2f00 0x55bdb45d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 -- 172.21.15.175:0/2138057128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45d2f00 msgr2=0x55bdb45d52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45d2f00 0x55bdb45d52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 -- 172.21.15.175:0/2138057128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c8640 msgr2=0x55bdb45d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c8640 0x55bdb45d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741a00640 1 -- 172.21.15.175:0/2138057128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb429f6a0 con 0x55bdb45c7d20 2024-07-29T00:45:01.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f073be00640 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45d2f00 0x55bdb45d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:45:01.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0741000640 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c8640 0x55bdb45d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:01.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.005+0000 7f0739600640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f07240a2d70 0x7f07240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.006+0000 7f0741a00640 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45c8100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0734010710 tx=0x7f0734010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8422e37d8c814ddc server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:01.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.006+0000 7f0739600640 1 -- 172.21.15.175:0/622885494 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f07280ef8e0 con 0x7f073c1016a0 2024-07-29T00:45:01.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.006+0000 7f0741000640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f07240a2d70 0x7f07240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.006+0000 7f072e000640 1 -- 172.21.15.175:0/2138057128 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073400e8d0 con 0x55bdb45c7d20 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.006+0000 7f072e000640 1 -- 172.21.15.175:0/2138057128 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0734012450 con 0x55bdb45c7d20 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.006+0000 7f072e000640 1 -- 172.21.15.175:0/2138057128 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073401c650 con 0x55bdb45c7d20 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 -- 172.21.15.175:0/2138057128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 msgr2=0x55bdb45c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45c8100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0734010710 tx=0x7f0734010bd0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f0741000640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f07240a2d70 0x7f07240a5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55bdb45c5110 tx=0x7f07280f5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 -- 172.21.15.175:0/2138057128 shutdown_connections 2024-07-29T00:45:01.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45d2f00 0x55bdb45d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c8640 0x55bdb45d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 --2- 172.21.15.175:0/2138057128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45c8100 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 -- 172.21.15.175:0/2138057128 >> 172.21.15.175:0/2138057128 conn(0x55bdb45c42b0 msgr2=0x55bdb45c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 -- 172.21.15.175:0/2138057128 shutdown_connections 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.007+0000 7f07463ec740 1 -- 172.21.15.175:0/2138057128 wait complete. 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.008+0000 7f07463ec740 1 -- start start 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.008+0000 7f07463ec740 1 Processor -- start 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdb45d5ad0 con 0x55bdb45c7d20 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bdb44e88c0 con 0x55bdb45d2f00 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bdb45d5c50 con 0x55bdb45c8640 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f07463ec740 10 client.0 _ll_register_callbacks cb 0x55bdb44d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f0741a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f0741a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47212/0 (socket says 172.21.15.175:47212) 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f0741a00640 1 -- 172.21.15.175:0/3736079452 learned_addr learned my addr 172.21.15.175:0/3736079452 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f073be00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f0741a00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f073be00640 1 -- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 msgr2=0x55bdb45d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f073be00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f073be00640 1 -- 172.21.15.175:0/3736079452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 msgr2=0x55bdb45cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.009+0000 7f073be00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f073be00640 1 -- 172.21.15.175:0/3736079452 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb44cbbc0 con 0x55bdb45c8640 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f0741a00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f073be00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d0350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f073c075a30 tx=0x7f0730012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88b314d927ab9fb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f0722a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f0722a00640 1 -- 172.21.15.175:0/3736079452 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073000f9d0 con 0x55bdb45c8640 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f0722a00640 1 -- 172.21.15.175:0/3736079452 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f073000fb70 con 0x55bdb45c8640 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.010+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0734012450 con 0x55bdb45c8640 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f0722a00640 1 -- 172.21.15.175:0/3736079452 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f073000fd70 con 0x55bdb45c8640 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 10 client.27076 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:01.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bdb45e1bd0 con 0x55bdb45c8640 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f0722a00640 1 -- 172.21.15.175:0/3736079452 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0734012450 con 0x55bdb45c8640 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f0722a00640 1 -- 172.21.15.175:0/3736079452 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f0730003cd0 con 0x55bdb45c8640 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 10 client.27076 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 20 client.27076 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb45e3090 0x55bdb45e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 4 client.27076 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:01.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.011+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb44edc90 con 0x55bdb45e3090 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.012+0000 7f0741a00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb45e3090 0x55bdb45e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.012+0000 7f0741a00640 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb45e3090 0x55bdb45e54c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bef24c032ef29491 server_cookie=1889437655337e93 in_seq=0 out_seq=0 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.012+0000 7f0722a00640 10 client.27076 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f0722a00640 1 -- 172.21.15.175:0/3736079452 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bdb44edc90 con 0x55bdb45e3090 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f0722a00640 10 client.27076 handle_command_reply: tid=0 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 client.27076 shutdown 2024-07-29T00:45:01.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 msgr2=0x55bdb45d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d0350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f073c075a30 tx=0x7f0730012e70 comp rx=0 tx=0).stop 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 shutdown_connections 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb45e3090 0x55bdb45e54c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d0350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f0722a00640 0 client.27076 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:01.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 --2- 172.21.15.175:0/3736079452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c7d20 0x55bdb45cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.013+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 >> 172.21.15.175:0/3736079452 conn(0x55bdb45c42b0 msgr2=0x55bdb45c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:01.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.014+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 shutdown_connections 2024-07-29T00:45:01.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.014+0000 7f07463ec740 1 -- 172.21.15.175:0/3736079452 wait complete. 2024-07-29T00:45:01.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.014+0000 7f07463ec740 20 client.27076 trim_cache size 0 max 16384 2024-07-29T00:45:01.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 Processor -- start 2024-07-29T00:45:01.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 -- start start 2024-07-29T00:45:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c7d20 0x55bdb45ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 0x55bdb45c3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b3bc0 con 0x55bdb45c8640 2024-07-29T00:45:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b5700 con 0x55bdb45d2f00 2024-07-29T00:45:01.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.033+0000 7f07463ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b4290 con 0x55bdb45c7d20 2024-07-29T00:45:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 0x55bdb45c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 0x55bdb45c3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47222/0 (socket says 172.21.15.175:47222) 2024-07-29T00:45:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 -- 172.21.15.175:0/1686775234 learned_addr learned my addr 172.21.15.175:0/1686775234 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:01.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f0741a00640 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c7d20 0x55bdb45ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f0741a00640 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 -- 172.21.15.175:0/1686775234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c7d20 msgr2=0x55bdb45ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c7d20 0x55bdb45ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 -- 172.21.15.175:0/1686775234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 msgr2=0x55bdb45eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.034+0000 7f073be00640 1 -- 172.21.15.175:0/1686775234 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb45bfc70 con 0x55bdb45c8640 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.035+0000 7f073be00640 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 0x55bdb45c3340 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f07300106f0 tx=0x7f073000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d01ce74af54a1f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:01.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.035+0000 7f072d600640 1 -- 172.21.15.175:0/1686775234 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730028070 con 0x55bdb45c8640 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.035+0000 7f072d600640 1 -- 172.21.15.175:0/1686775234 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07300055b0 con 0x55bdb45c8640 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.035+0000 7f072d600640 1 -- 172.21.15.175:0/1686775234 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300035f0 con 0x55bdb45c8640 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 -- 172.21.15.175:0/1686775234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 msgr2=0x55bdb45c3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 0x55bdb45c3340 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f07300106f0 tx=0x7f073000fc00 comp rx=0 tx=0).stop 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 -- 172.21.15.175:0/1686775234 shutdown_connections 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45d2f00 0x55bdb45eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45c8640 0x55bdb45c3340 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 --2- 172.21.15.175:0/1686775234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c7d20 0x55bdb45ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 -- 172.21.15.175:0/1686775234 >> 172.21.15.175:0/1686775234 conn(0x55bdb45c42b0 msgr2=0x55bdb45c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:01.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 -- 172.21.15.175:0/1686775234 shutdown_connections 2024-07-29T00:45:01.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.036+0000 7f07463ec740 1 -- 172.21.15.175:0/1686775234 wait complete. 2024-07-29T00:45:01.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 -- start start 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 Processor -- start 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c7d20 0x55bdb45d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45d2f00 0x55bdb45d7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b5030 con 0x55bdb45d2f00 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b4960 con 0x55bdb45c7d20 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b1bc0 con 0x55bdb45c8640 2024-07-29T00:45:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.037+0000 7f07463ec740 10 client.0 _ll_register_callbacks cb 0x7f0730012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f0741a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c7d20 0x55bdb45d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:36174/0 (socket says 172.21.15.175:36174) 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 -- 172.21.15.175:0/2865394686 learned_addr learned my addr 172.21.15.175:0/2865394686 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f0741a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c7d20 0x55bdb45d6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48914/0 (socket says 172.21.15.175:48914) 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f0741a00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45d2f00 0x55bdb45d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 -- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c7d20 msgr2=0x55bdb45d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c7d20 0x55bdb45d6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 -- 172.21.15.175:0/2865394686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45d2f00 msgr2=0x55bdb45d7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45d2f00 0x55bdb45d7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 -- 172.21.15.175:0/2865394686 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb464ce20 con 0x55bdb45c8640 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f0741a00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45d2f00 0x55bdb45d7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f073be00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d71e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f073000a010 tx=0x7f07300095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61af46346d989379 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.038+0000 7f0722a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f0722a00640 1 -- 172.21.15.175:0/2865394686 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730033060 con 0x55bdb45c8640 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07300055b0 con 0x55bdb45c8640 2024-07-29T00:45:01.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f0722a00640 1 -- 172.21.15.175:0/2865394686 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f0730010ad0 con 0x55bdb45c8640 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f0722a00640 1 -- 172.21.15.175:0/2865394686 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0730028040 con 0x55bdb45c8640 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f07463ec740 10 client.27082 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bdb464e140 con 0x55bdb45c8640 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f0722a00640 1 -- 172.21.15.175:0/2865394686 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07300055b0 con 0x55bdb45c8640 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.039+0000 7f0722a00640 1 -- 172.21.15.175:0/2865394686 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f073002e460 con 0x55bdb45c8640 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f07463ec740 10 client.27082 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f07463ec740 10 client.27082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f07463ec740 20 client.27082 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f07463ec740 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb464e760 0x55bdb4650b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:01.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f07463ec740 4 client.27082 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bdb453ae10 con 0x55bdb464e760 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.040+0000 7f0741a00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb464e760 0x55bdb4650b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.041+0000 7f0741a00640 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb464e760 0x55bdb4650b80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b57ff0b4f06667 server_cookie=32494bd78affe9ac in_seq=0 out_seq=0 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.041+0000 7f0722a00640 10 client.27082 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f0722a00640 1 -- 172.21.15.175:0/2865394686 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bdb453ae10 con 0x55bdb464e760 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f0722a00640 10 client.27082 handle_command_reply: tid=0 2024-07-29T00:45:01.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 client.27082 shutdown 2024-07-29T00:45:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 msgr2=0x55bdb45d71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d71e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f073000a010 tx=0x7f07300095a0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 shutdown_connections 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55bdb464e760 0x55bdb4650b80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb45d2f00 0x55bdb45d7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb45c8640 0x55bdb45d71e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 --2- 172.21.15.175:0/2865394686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bdb45c7d20 0x55bdb45d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 >> 172.21.15.175:0/2865394686 conn(0x55bdb45c42b0 msgr2=0x55bdb45d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:01.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f0722a00640 0 client.27082 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:01.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 shutdown_connections 2024-07-29T00:45:01.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 1 -- 172.21.15.175:0/2865394686 wait complete. 2024-07-29T00:45:01.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.042+0000 7f07463ec740 20 client.27082 trim_cache size 0 max 16384 2024-07-29T00:45:01.049 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:45:01.049 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:45:01.049 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "dc418360-7e40-4ff7-9f62-15d35f767578", 2024-07-29T00:45:01.050 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:45:01.050 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:45:01.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f07240a2d70 msgr2=0x7f07240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f07240a2d70 0x7f07240a5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55bdb45c5110 tx=0x7f07280f5530 comp rx=0 tx=0).stop 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 msgr2=0x7f073c0741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c0741e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0728005940 tx=0x7f07280040c0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 shutdown_connections 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f073c1076a0 0x7f073c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f07240a2d70 0x7f07240a5230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f073c102040 0x7f073c074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 --2- 172.21.15.175:0/622885494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f073c1016a0 0x7f073c0741e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:01.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 >> 172.21.15.175:0/622885494 conn(0x7f073c0fd3a0 msgr2=0x7f073c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:01.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 shutdown_connections 2024-07-29T00:45:01.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:01.043+0000 7f0742400640 1 -- 172.21.15.175:0/622885494 wait complete. 2024-07-29T00:45:01.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:45:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: pgmap v262: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:01 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: pgmap v262: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:01 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: pgmap v262: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:45:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:02.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:01 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:02.433 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:03.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 -- 172.21.15.175:0/2035532843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c107d10 msgr2=0x7f306c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 --2- 172.21.15.175:0/2035532843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c107d10 0x7f306c108130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f305c009900 tx=0x7f305c02a3f0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 -- 172.21.15.175:0/2035532843 shutdown_connections 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 --2- 172.21.15.175:0/2035532843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 --2- 172.21.15.175:0/2035532843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c107d10 0x7f306c108130 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 --2- 172.21.15.175:0/2035532843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f306c10dd50 0x7f306c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 -- 172.21.15.175:0/2035532843 >> 172.21.15.175:0/2035532843 conn(0x7f306c077670 msgr2=0x7f306c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:03.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 -- 172.21.15.175:0/2035532843 shutdown_connections 2024-07-29T00:45:03.107 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:02 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:03.107 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:02 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.107 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:02 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:03.107 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:02 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.107 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:02 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:03.108 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:02 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.105+0000 7f3072a00640 1 -- 172.21.15.175:0/2035532843 wait complete. 2024-07-29T00:45:03.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.106+0000 7f3072a00640 1 Processor -- start 2024-07-29T00:45:03.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 -- start start 2024-07-29T00:45:03.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f306c107d10 0x7f306c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c10dd50 0x7f306c1a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1180c0 con 0x7f306c108670 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f306c117f40 con 0x7f306c10dd50 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f306c118240 con 0x7f306c107d10 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3070c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3070c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c19ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47256/0 (socket says 172.21.15.175:47256) 2024-07-29T00:45:03.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3070c00640 1 -- 172.21.15.175:0/2496184195 learned_addr learned my addr 172.21.15.175:0/2496184195 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3071600640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f306c107d10 0x7f306c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.107+0000 7f3072000640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c10dd50 0x7f306c1a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3070c00640 1 -- 172.21.15.175:0/2496184195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f306c107d10 msgr2=0x7f306c19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3070c00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f306c107d10 0x7f306c19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3070c00640 1 -- 172.21.15.175:0/2496184195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c10dd50 msgr2=0x7f306c1a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3070c00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c10dd50 0x7f306c1a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3070c00640 1 -- 172.21.15.175:0/2496184195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c1a34e0 con 0x7f306c108670 2024-07-29T00:45:03.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3072000640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c10dd50 0x7f306c1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:03.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.108+0000 7f3070c00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c19ea70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9e0 tx=0x7f305c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:03.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.109+0000 7f306a000640 1 -- 172.21.15.175:0/2496184195 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c03f070 con 0x7f306c108670 2024-07-29T00:45:03.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.109+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c1a3770 con 0x7f306c108670 2024-07-29T00:45:03.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.109+0000 7f306a000640 1 -- 172.21.15.175:0/2496184195 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c031c30 con 0x7f306c108670 2024-07-29T00:45:03.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.109+0000 7f306a000640 1 -- 172.21.15.175:0/2496184195 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c035030 con 0x7f306c108670 2024-07-29T00:45:03.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.109+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1ab0b0 con 0x7f306c108670 2024-07-29T00:45:03.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.112+0000 7f306a000640 1 -- 172.21.15.175:0/2496184195 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f305c042070 con 0x7f306c108670 2024-07-29T00:45:03.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.112+0000 7f3076bc4740 1 Processor -- start 2024-07-29T00:45:03.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.112+0000 7f3076bc4740 1 -- start start 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b0feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b1097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b10c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3076bc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b063ad0 con 0x55cb5b0feb10 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3076bc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cb5affbaf0 con 0x55cb5b109cf0 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3076bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b05fa90 con 0x55cb5b0ff430 2024-07-29T00:45:03.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3072000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b0feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3072000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b0feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47270/0 (socket says 172.21.15.175:47270) 2024-07-29T00:45:03.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3071600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b1097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3072000640 1 -- 172.21.15.175:0/3684821742 learned_addr learned my addr 172.21.15.175:0/3684821742 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:03.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.113+0000 7f3070c00640 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b10c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3071600640 1 -- 172.21.15.175:0/3684821742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 msgr2=0x55cb5b10c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3071600640 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b10c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3071600640 1 -- 172.21.15.175:0/3684821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 msgr2=0x55cb5b0feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3071600640 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b0feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3071600640 1 -- 172.21.15.175:0/3684821742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5add53c0 con 0x55cb5b0ff430 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f306a000640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f30540a2d70 0x7f30540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3071600640 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b1097b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3060009c00 tx=0x7f3060011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf5f121d97fdf07 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f305a600640 1 -- 172.21.15.175:0/3684821742 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060004050 con 0x55cb5b0ff430 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f306a000640 1 -- 172.21.15.175:0/2496184195 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f305c0efa30 con 0x7f306c108670 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f3072000640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f30540a2d70 0x7f30540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f305a600640 1 -- 172.21.15.175:0/3684821742 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060005500 con 0x55cb5b0ff430 2024-07-29T00:45:03.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.114+0000 7f305a600640 1 -- 172.21.15.175:0/3684821742 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060004d30 con 0x55cb5b0ff430 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 -- 172.21.15.175:0/3684821742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 msgr2=0x55cb5b1097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3072000640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f30540a2d70 0x7f30540a5230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3064003f80 tx=0x7f3064009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b1097b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3060009c00 tx=0x7f3060011810 comp rx=0 tx=0).stop 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 -- 172.21.15.175:0/3684821742 shutdown_connections 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b10c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b1097b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3684821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b0feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 -- 172.21.15.175:0/3684821742 >> 172.21.15.175:0/3684821742 conn(0x55cb5b0fb0a0 msgr2=0x55cb5b0fb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 -- 172.21.15.175:0/3684821742 shutdown_connections 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.115+0000 7f3076bc4740 1 -- 172.21.15.175:0/3684821742 wait complete. 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.116+0000 7f3076bc4740 1 -- start start 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.116+0000 7f3076bc4740 1 Processor -- start 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b107570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b10c8c0 con 0x55cb5b0feb10 2024-07-29T00:45:03.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b01ebc0 con 0x55cb5b109cf0 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b10ca40 con 0x55cb5b0ff430 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3076bc4740 10 client.0 _ll_register_callbacks cb 0x55cb5b00ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3071600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:36204/0 (socket says 172.21.15.175:36204) 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 -- 172.21.15.175:0/656993310 learned_addr learned my addr 172.21.15.175:0/656993310 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:03.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3072000640 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 -- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 msgr2=0x55cb5b107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 -- 172.21.15.175:0/656993310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 msgr2=0x55cb5b106af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b106af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.117+0000 7f3070c00640 1 -- 172.21.15.175:0/656993310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5b115d30 con 0x55cb5b0ff430 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f3070c00640 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b107030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f305c034040 tx=0x7f305c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2c1bffb5179dd2 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f304ea00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:03.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f304ea00640 1 -- 172.21.15.175:0/656993310 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c049070 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f304ea00640 1 -- 172.21.15.175:0/656993310 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f305c0c4680 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3060005500 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f304ea00640 1 -- 172.21.15.175:0/656993310 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f305c0bbd20 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f3076bc4740 10 client.27094 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cb5b10c920 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.118+0000 7f304ea00640 1 -- 172.21.15.175:0/656993310 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060005500 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f304ea00640 1 -- 172.21.15.175:0/656993310 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f305c0bb3a0 con 0x55cb5b0ff430 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3076bc4740 10 client.27094 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3076bc4740 10 client.27094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3076bc4740 20 client.27094 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3076bc4740 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b119a90 0x55cb5b11bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3076bc4740 4 client.27094 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb5b071040 con 0x55cb5b119a90 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.119+0000 7f3071600640 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b119a90 0x55cb5b11bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.120+0000 7f3071600640 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b119a90 0x55cb5b11bf00 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb26938f56af7a4 server_cookie=ecd9621234095449 in_seq=0 out_seq=0 2024-07-29T00:45:03.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.120+0000 7f304ea00640 10 client.27094 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f304ea00640 1 -- 172.21.15.175:0/656993310 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cb5b071040 con 0x55cb5b119a90 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f304ea00640 10 client.27094 handle_command_reply: tid=0 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 client.27094 shutdown 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 msgr2=0x55cb5b107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b107030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f305c034040 tx=0x7f305c005d50 comp rx=0 tx=0).stop 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 shutdown_connections 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b119a90 0x55cb5b11bf00 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b109cf0 0x55cb5b107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b0ff430 0x55cb5b107030 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 --2- 172.21.15.175:0/656993310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f304ea00640 0 client.27094 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:03.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 >> 172.21.15.175:0/656993310 conn(0x55cb5b0fb0a0 msgr2=0x55cb5b100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:03.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 shutdown_connections 2024-07-29T00:45:03.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 1 -- 172.21.15.175:0/656993310 wait complete. 2024-07-29T00:45:03.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.121+0000 7f3076bc4740 20 client.27094 trim_cache size 0 max 16384 2024-07-29T00:45:03.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 Processor -- start 2024-07-29T00:45:03.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 -- start start 2024-07-29T00:45:03.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b1073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b0fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b121810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1b3a80 con 0x55cb5b0feb10 2024-07-29T00:45:03.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1b55c0 con 0x55cb5b0ff430 2024-07-29T00:45:03.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3076bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1b4150 con 0x55cb5b109cf0 2024-07-29T00:45:03.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.140+0000 7f3071600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b1073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3072000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b1073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47288/0 (socket says 172.21.15.175:47288) 2024-07-29T00:45:03.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 -- 172.21.15.175:0/1846110439 learned_addr learned my addr 172.21.15.175:0/1846110439 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:03.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3070c00640 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b0fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 -- 172.21.15.175:0/1846110439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 msgr2=0x55cb5b121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 -- 172.21.15.175:0/1846110439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 msgr2=0x55cb5b0fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b0fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3071600640 1 -- 172.21.15.175:0/1846110439 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5b0faed0 con 0x55cb5b0feb10 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3072000640 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.141+0000 7f3070c00640 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b0fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.142+0000 7f3071600640 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b1073c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3060014040 tx=0x7f3060009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b946e57555ec3fb4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.142+0000 7f3059c00640 1 -- 172.21.15.175:0/1846110439 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600044d0 con 0x55cb5b0feb10 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.142+0000 7f3059c00640 1 -- 172.21.15.175:0/1846110439 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060003ad0 con 0x55cb5b0feb10 2024-07-29T00:45:03.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.142+0000 7f3059c00640 1 -- 172.21.15.175:0/1846110439 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306002e8c0 con 0x55cb5b0feb10 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 -- 172.21.15.175:0/1846110439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 msgr2=0x55cb5b1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b1073c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3060014040 tx=0x7f3060009620 comp rx=0 tx=0).stop 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 -- 172.21.15.175:0/1846110439 shutdown_connections 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b0fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 --2- 172.21.15.175:0/1846110439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b1073c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 -- 172.21.15.175:0/1846110439 >> 172.21.15.175:0/1846110439 conn(0x55cb5b0fb0a0 msgr2=0x55cb5b0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 -- 172.21.15.175:0/1846110439 shutdown_connections 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.143+0000 7f3076bc4740 1 -- 172.21.15.175:0/1846110439 wait complete. 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 -- start start 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 Processor -- start 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b1121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b10cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1b4ef0 con 0x55cb5b0feb10 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1b4820 con 0x55cb5b0ff430 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1b1a80 con 0x55cb5b109cf0 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.144+0000 7f3076bc4740 10 client.0 _ll_register_callbacks cb 0x7f305c03b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b1121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3071600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b1121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:48954/0 (socket says 172.21.15.175:48954) 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 -- 172.21.15.175:0/3333513933 learned_addr learned my addr 172.21.15.175:0/3333513933 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:03.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3072000640 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b10cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 -- 172.21.15.175:0/3333513933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 msgr2=0x55cb5b10cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b10cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 -- 172.21.15.175:0/3333513933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 msgr2=0x55cb5b111ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b111ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 -- 172.21.15.175:0/3333513933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5b0fbd30 con 0x55cb5b0ff430 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f3070c00640 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b1121e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9b0 tx=0x7f305c0c48b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bfab66681cfdd8a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f304ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.145+0000 7f304ea00640 1 -- 172.21.15.175:0/3333513933 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c049040 con 0x55cb5b0ff430 2024-07-29T00:45:03.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f304ea00640 1 -- 172.21.15.175:0/3333513933 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f305c031750 con 0x55cb5b0ff430 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3060003ad0 con 0x55cb5b0ff430 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f304ea00640 1 -- 172.21.15.175:0/3333513933 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f305c0bb8b0 con 0x55cb5b0ff430 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 10 client.27152 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cb5b184bd0 con 0x55cb5b0ff430 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f304ea00640 1 -- 172.21.15.175:0/3333513933 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060003ad0 con 0x55cb5b0ff430 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f304ea00640 1 -- 172.21.15.175:0/3333513933 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f305c0a50d0 con 0x55cb5b0ff430 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 10 client.27152 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:03.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 10 client.27152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 20 client.27152 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b1851c0 0x55cb5b1875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 4 client.27152 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.146+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb5b06b550 con 0x55cb5b1851c0 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f3071600640 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b1851c0 0x55cb5b1875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f3071600640 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b1851c0 0x55cb5b1875e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26df9d87aa408ad7 server_cookie=73ef581afd4707f8 in_seq=0 out_seq=0 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f304ea00640 10 client.27152 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f304ea00640 1 -- 172.21.15.175:0/3333513933 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cb5b06b550 con 0x55cb5b1851c0 2024-07-29T00:45:03.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f304ea00640 10 client.27152 handle_command_reply: tid=0 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f3076bc4740 1 client.27152 shutdown 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 msgr2=0x55cb5b1121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.147+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b1121e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9b0 tx=0x7f305c0c48b0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 shutdown_connections 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55cb5b1851c0 0x55cb5b1875e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb5b109cf0 0x55cb5b10cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb5b0ff430 0x55cb5b1121e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 --2- 172.21.15.175:0/3333513933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb5b0feb10 0x55cb5b111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f304ea00640 0 client.27152 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:03.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 >> 172.21.15.175:0/3333513933 conn(0x55cb5b0fb0a0 msgr2=0x55cb5b100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:03.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 shutdown_connections 2024-07-29T00:45:03.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 1 -- 172.21.15.175:0/3333513933 wait complete. 2024-07-29T00:45:03.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.148+0000 7f3076bc4740 20 client.27152 trim_cache size 0 max 16384 2024-07-29T00:45:03.155 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "dc418360-7e40-4ff7-9f62-15d35f767578": { 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "dc418360-7e40-4ff7-9f62-15d35f767578", 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:45:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:45:03.157 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:45:03.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f30540a2d70 msgr2=0x7f30540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f30540a2d70 0x7f30540a5230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3064003f80 tx=0x7f3064009500 comp rx=0 tx=0).stop 2024-07-29T00:45:03.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 msgr2=0x7f306c19ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c19ea70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9e0 tx=0x7f305c003c10 comp rx=0 tx=0).stop 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 shutdown_connections 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306c10dd50 0x7f306c1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f30540a2d70 0x7f30540a5230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f306c108670 0x7f306c19ea70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 --2- 172.21.15.175:0/2496184195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f306c107d10 0x7f306c19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:03.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 >> 172.21.15.175:0/2496184195 conn(0x7f306c077670 msgr2=0x7f306c078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:03.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 shutdown_connections 2024-07-29T00:45:03.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:03.149+0000 7f3072a00640 1 -- 172.21.15.175:0/2496184195 wait complete. 2024-07-29T00:45:03.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:02 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:03.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:02 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:02 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:03.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:02 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:02 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:03.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:02 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:02 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:03.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:02 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:02 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:03.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:02 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:03.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:02 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:03.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:02 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:04.040 INFO:tasks.cephfs.filesystem:scrub status for tag:dc418360-7e40-4ff7-9f62-15d35f767578 - {'path': '/', 'tag': 'dc418360-7e40-4ff7-9f62-15d35f767578', 'options': 'recursive,force'} 2024-07-29T00:45:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:03 smithi181 ceph-mon[36706]: pgmap v263: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:45:04.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:03 smithi175 ceph-mon[32138]: pgmap v263: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:45:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:03 smithi204 ceph-mon[37368]: pgmap v263: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:45:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:05 smithi181 ceph-mon[36706]: pgmap v264: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:45:06.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:05 smithi175 ceph-mon[32138]: pgmap v264: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:45:06.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:05 smithi204 ceph-mon[37368]: pgmap v264: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:45:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:07 smithi181 ceph-mon[36706]: pgmap v265: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:45:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:07 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:08.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:07 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:08.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:07 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:08.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:07 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:08.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:07 smithi175 ceph-mon[32138]: pgmap v265: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:45:08.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:07 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:08.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:07 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:08.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:07 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:08.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:07 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:07 smithi204 ceph-mon[37368]: pgmap v265: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:45:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:07 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:07 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:07 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:08.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:07 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:09.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:08 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:09.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:08 smithi181 ceph-mon[36706]: osdmap e97: 12 total, 12 up, 12 in 2024-07-29T00:45:09.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:08 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:09.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:08 smithi175 ceph-mon[32138]: osdmap e97: 12 total, 12 up, 12 in 2024-07-29T00:45:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:08 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:08 smithi204 ceph-mon[37368]: osdmap e97: 12 total, 12 up, 12 in 2024-07-29T00:45:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:09 smithi181 ceph-mon[36706]: pgmap v267: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:45:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:09 smithi175 ceph-mon[32138]: pgmap v267: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:45:10.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:09 smithi204 ceph-mon[37368]: pgmap v267: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:45:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:11 smithi181 ceph-mon[36706]: pgmap v268: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:45:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:11 smithi175 ceph-mon[32138]: pgmap v268: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:45:12.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:11 smithi204 ceph-mon[37368]: pgmap v268: 97 pgs: 97 active+clean; 128 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:45:14.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:13 smithi181 ceph-mon[36706]: pgmap v269: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:14.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:13 smithi175 ceph-mon[32138]: pgmap v269: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:14.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:13 smithi204 ceph-mon[37368]: pgmap v269: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:16.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:15 smithi181 ceph-mon[36706]: pgmap v270: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:16.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:15 smithi181 ceph-mon[36706]: osdmap e98: 12 total, 12 up, 12 in 2024-07-29T00:45:16.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:15 smithi175 ceph-mon[32138]: pgmap v270: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:16.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:15 smithi175 ceph-mon[32138]: osdmap e98: 12 total, 12 up, 12 in 2024-07-29T00:45:16.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:15 smithi204 ceph-mon[37368]: pgmap v270: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:16.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:15 smithi204 ceph-mon[37368]: osdmap e98: 12 total, 12 up, 12 in 2024-07-29T00:45:18.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:17 smithi181 ceph-mon[36706]: pgmap v272: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 246 B/s wr, 12 op/s 2024-07-29T00:45:18.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:17 smithi175 ceph-mon[32138]: pgmap v272: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 246 B/s wr, 12 op/s 2024-07-29T00:45:18.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:17 smithi204 ceph-mon[37368]: pgmap v272: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 246 B/s wr, 12 op/s 2024-07-29T00:45:20.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:19 smithi181 ceph-mon[36706]: pgmap v273: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:20.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:19 smithi175 ceph-mon[32138]: pgmap v273: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:20.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:19 smithi204 ceph-mon[37368]: pgmap v273: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:22.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:21 smithi181 ceph-mon[36706]: pgmap v274: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:22.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:21 smithi175 ceph-mon[32138]: pgmap v274: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:22.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:21 smithi204 ceph-mon[37368]: pgmap v274: 97 pgs: 97 active+clean; 127 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:45:24.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:23 smithi181 ceph-mon[36706]: pgmap v275: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:45:24.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:23 smithi175 ceph-mon[32138]: pgmap v275: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:45:24.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:23 smithi204 ceph-mon[37368]: pgmap v275: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:45:26.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:25 smithi181 ceph-mon[36706]: pgmap v276: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:45:26.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:25 smithi175 ceph-mon[32138]: pgmap v276: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:45:26.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:25 smithi204 ceph-mon[37368]: pgmap v276: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:45:28.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:27 smithi181 ceph-mon[36706]: pgmap v277: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:45:28.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:27 smithi175 ceph-mon[32138]: pgmap v277: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:45:28.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:27 smithi204 ceph-mon[37368]: pgmap v277: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:45:30.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:29 smithi181 ceph-mon[36706]: pgmap v278: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:30.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:29 smithi175 ceph-mon[32138]: pgmap v278: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:30.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:29 smithi204 ceph-mon[37368]: pgmap v278: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:32.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:31 smithi181 ceph-mon[36706]: pgmap v279: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:32.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:31 smithi175 ceph-mon[32138]: pgmap v279: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:32.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:31 smithi204 ceph-mon[37368]: pgmap v279: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:34.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:45:34.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:33 smithi181 ceph-mon[36706]: pgmap v280: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:34.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:33 smithi175 ceph-mon[32138]: pgmap v280: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:34.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:33 smithi204 ceph-mon[37368]: pgmap v280: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:34.517 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:35.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 -- 172.21.15.175:0/729281563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548109a80 msgr2=0x7f6548109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 --2- 172.21.15.175:0/729281563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548109a80 0x7f6548109e60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6538009930 tx=0x7f653802a420 comp rx=0 tx=0).stop 2024-07-29T00:45:35.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 -- 172.21.15.175:0/729281563 shutdown_connections 2024-07-29T00:45:35.177 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 --2- 172.21.15.175:0/729281563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6548104420 0x7f6548110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 --2- 172.21.15.175:0/729281563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548103a80 0x7f6548103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 --2- 172.21.15.175:0/729281563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548109a80 0x7f6548109e60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 -- 172.21.15.175:0/729281563 >> 172.21.15.175:0/729281563 conn(0x7f65480ff780 msgr2=0x7f6548101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:35.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 -- 172.21.15.175:0/729281563 shutdown_connections 2024-07-29T00:45:35.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.176+0000 7f654d200640 1 -- 172.21.15.175:0/729281563 wait complete. 2024-07-29T00:45:35.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.178+0000 7f654d200640 1 Processor -- start 2024-07-29T00:45:35.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.178+0000 7f654d200640 1 -- start start 2024-07-29T00:45:35.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.178+0000 7f654d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 0x7f654819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.178+0000 7f654d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548104420 0x7f654819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f654d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6548109a80 0x7f654819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f654d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6548113ed0 con 0x7f6548103a80 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f654d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6548113d50 con 0x7f6548104420 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f654d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6548114050 con 0x7f6548109a80 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 0x7f654819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 0x7f654819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59114/0 (socket says 172.21.15.175:59114) 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545e00640 1 -- 172.21.15.175:0/1295975923 learned_addr learned my addr 172.21.15.175:0/1295975923 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545400640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548104420 0x7f654819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6546800640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6548109a80 0x7f654819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545400640 1 -- 172.21.15.175:0/1295975923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6548109a80 msgr2=0x7f654819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545400640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6548109a80 0x7f654819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545400640 1 -- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 msgr2=0x7f654819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545400640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 0x7f654819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545400640 1 -- 172.21.15.175:0/1295975923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654819f2e0 con 0x7f6548104420 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.179+0000 7f6545e00640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 0x7f654819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.180+0000 7f6545400640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548104420 0x7f654819a870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f653000eb00 tx=0x7f653000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:35.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.180+0000 7f653ea00640 1 -- 172.21.15.175:0/1295975923 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530010430 con 0x7f6548104420 2024-07-29T00:45:35.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.180+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654819f5d0 con 0x7f6548104420 2024-07-29T00:45:35.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.180+0000 7f653ea00640 1 -- 172.21.15.175:0/1295975923 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6530010ab0 con 0x7f6548104420 2024-07-29T00:45:35.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.180+0000 7f653ea00640 1 -- 172.21.15.175:0/1295975923 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530017bb0 con 0x7f6548104420 2024-07-29T00:45:35.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.180+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65481a6eb0 con 0x7f6548104420 2024-07-29T00:45:35.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.183+0000 7f654fc36740 1 Processor -- start 2024-07-29T00:45:35.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.183+0000 7f653ea00640 1 -- 172.21.15.175:0/1295975923 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6530010c60 con 0x7f6548104420 2024-07-29T00:45:35.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 -- start start 2024-07-29T00:45:35.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a09d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a16f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56401996ead0 con 0x564019a0a250 2024-07-29T00:45:35.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564019906af0 con 0x564019a09930 2024-07-29T00:45:35.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f654fc36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56401996aa90 con 0x564019a14b10 2024-07-29T00:45:35.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f6545e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f6545e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a145d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59124/0 (socket says 172.21.15.175:59124) 2024-07-29T00:45:35.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f6545e00640 1 -- 172.21.15.175:0/1884752031 learned_addr learned my addr 172.21.15.175:0/1884752031 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f6545400640 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a16f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.184+0000 7f6546800640 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a09d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f6545e00640 1 -- 172.21.15.175:0/1884752031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 msgr2=0x564019a16f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f6545e00640 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a16f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f6545e00640 1 -- 172.21.15.175:0/1884752031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 msgr2=0x564019a09d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f6545e00640 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a09d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f6545e00640 1 -- 172.21.15.175:0/1884752031 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640196e03c0 con 0x564019a0a250 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f653ea00640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f652c0a2cf0 0x7f652c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f6545e00640 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a145d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6538005e10 tx=0x7f6538005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546b5ab9d676c4f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.185+0000 7f653ea00640 1 -- 172.21.15.175:0/1295975923 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f65300cfeb0 con 0x7f6548104420 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.186+0000 7f6546800640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f652c0a2cf0 0x7f652c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.186+0000 7f6537400640 1 -- 172.21.15.175:0/1884752031 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538040070 con 0x564019a0a250 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.186+0000 7f6537400640 1 -- 172.21.15.175:0/1884752031 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6538004760 con 0x564019a0a250 2024-07-29T00:45:35.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.186+0000 7f6537400640 1 -- 172.21.15.175:0/1884752031 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538035040 con 0x564019a0a250 2024-07-29T00:45:35.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 -- 172.21.15.175:0/1884752031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 msgr2=0x564019a145d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a145d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6538005e10 tx=0x7f6538005e40 comp rx=0 tx=0).stop 2024-07-29T00:45:35.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f6546800640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f652c0a2cf0 0x7f652c0a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6540009d20 tx=0x7f6540005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 -- 172.21.15.175:0/1884752031 shutdown_connections 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a16f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a145d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 --2- 172.21.15.175:0/1884752031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a09d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 -- 172.21.15.175:0/1884752031 >> 172.21.15.175:0/1884752031 conn(0x564019a05ec0 msgr2=0x564019a062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 -- 172.21.15.175:0/1884752031 shutdown_connections 2024-07-29T00:45:35.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 -- 172.21.15.175:0/1884752031 wait complete. 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.187+0000 7f654fc36740 1 -- start start 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 Processor -- start 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a09930 0x564019a11980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a12400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564019a176e0 con 0x564019a0a250 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564019929bc0 con 0x564019a14b10 2024-07-29T00:45:35.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564019a17860 con 0x564019a09930 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f654fc36740 10 client.0 _ll_register_callbacks cb 0x564019915d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.188+0000 7f6545e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a09930 0x564019a11980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6545e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a09930 0x564019a11980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:37884/0 (socket says 172.21.15.175:37884) 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6545e00640 1 -- 172.21.15.175:0/3953625542 learned_addr learned my addr 172.21.15.175:0/3953625542 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a12400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6545400640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 -- 172.21.15.175:0/3953625542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a09930 msgr2=0x564019a11980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a09930 0x564019a11980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 -- 172.21.15.175:0/3953625542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 msgr2=0x564019a11ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a11ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 -- 172.21.15.175:0/3953625542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564019a20b80 con 0x564019a14b10 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f6546800640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a12400 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6540004580 tx=0x7f6540004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc8a784cdb9a0270 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.189+0000 7f652be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f652be00640 1 -- 172.21.15.175:0/3953625542 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540009e10 con 0x564019a14b10 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f652be00640 1 -- 172.21.15.175:0/3953625542 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f6540005020 con 0x564019a14b10 2024-07-29T00:45:35.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6538004760 con 0x564019a14b10 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f652be00640 1 -- 172.21.15.175:0/3953625542 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6540005260 con 0x564019a14b10 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f654fc36740 10 client.27164 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564019a23570 con 0x564019a14b10 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.190+0000 7f652be00640 1 -- 172.21.15.175:0/3953625542 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6538004760 con 0x564019a14b10 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f652be00640 1 -- 172.21.15.175:0/3953625542 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6540027030 con 0x564019a14b10 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f654fc36740 10 client.27164 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f654fc36740 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:35.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f654fc36740 20 client.27164 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:35.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f654fc36740 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a24a30 0x564019a26e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f654fc36740 4 client.27164 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:35.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56401997c040 con 0x564019a24a30 2024-07-29T00:45:35.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.191+0000 7f6545e00640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a24a30 0x564019a26e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.192+0000 7f6545e00640 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a24a30 0x564019a26e80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6206fc0968ff2265 server_cookie=f9c5e1cef5c4052d in_seq=0 out_seq=0 2024-07-29T00:45:35.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.192+0000 7f652be00640 10 client.27164 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:35.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.192+0000 7f652be00640 1 -- 172.21.15.175:0/3953625542 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56401997c040 con 0x564019a24a30 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.192+0000 7f652be00640 10 client.27164 handle_command_reply: tid=0 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 client.27164 shutdown 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 msgr2=0x564019a12400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a12400 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6540004580 tx=0x7f6540004730 comp rx=0 tx=0).stop 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 shutdown_connections 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a24a30 0x564019a26e80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a12400 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 --2- 172.21.15.175:0/3953625542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a09930 0x564019a11980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f652be00640 0 client.27164 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:35.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 >> 172.21.15.175:0/3953625542 conn(0x564019a05ec0 msgr2=0x564019a0bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:35.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 shutdown_connections 2024-07-29T00:45:35.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 1 -- 172.21.15.175:0/3953625542 wait complete. 2024-07-29T00:45:35.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.193+0000 7f654fc36740 20 client.27164 trim_cache size 0 max 16384 2024-07-29T00:45:35.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.212+0000 7f654fc36740 1 Processor -- start 2024-07-29T00:45:35.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.212+0000 7f654fc36740 1 -- start start 2024-07-29T00:45:35.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f654fc36740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a09930 0x564019a00b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f654fc36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 0x564019a12250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f654fc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a04de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f654fc36740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65481af950 con 0x564019a09930 2024-07-29T00:45:35.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f654fc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b1490 con 0x564019a14b10 2024-07-29T00:45:35.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f654fc36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b0020 con 0x564019a0a250 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f6545e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a09930 0x564019a00b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f6545400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 0x564019a12250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f6545400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 0x564019a12250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:37896/0 (socket says 172.21.15.175:37896) 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f6545400640 1 -- 172.21.15.175:0/3826862370 learned_addr learned my addr 172.21.15.175:0/3826862370 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.213+0000 7f6546800640 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a04de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6545400640 1 -- 172.21.15.175:0/3826862370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 msgr2=0x564019a04de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6545400640 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a04de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6545400640 1 -- 172.21.15.175:0/3826862370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a09930 msgr2=0x564019a00b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6545400640 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a09930 0x564019a00b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6545400640 1 -- 172.21.15.175:0/3826862370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564019a10930 con 0x564019a0a250 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6545400640 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 0x564019a12250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6530098cf0 tx=0x7f65300d3960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71dda0995369bdb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6536a00640 1 -- 172.21.15.175:0/3826862370 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653009be60 con 0x564019a0a250 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6536a00640 1 -- 172.21.15.175:0/3826862370 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653009ca20 con 0x564019a0a250 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.214+0000 7f6536a00640 1 -- 172.21.15.175:0/3826862370 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653009bc80 con 0x564019a0a250 2024-07-29T00:45:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 -- 172.21.15.175:0/3826862370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 msgr2=0x564019a12250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 0x564019a12250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6530098cf0 tx=0x7f65300d3960 comp rx=0 tx=0).stop 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 -- 172.21.15.175:0/3826862370 shutdown_connections 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a14b10 0x564019a04de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a0a250 0x564019a12250 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 --2- 172.21.15.175:0/3826862370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a09930 0x564019a00b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 -- 172.21.15.175:0/3826862370 >> 172.21.15.175:0/3826862370 conn(0x564019a05ec0 msgr2=0x564019a046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:35.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 -- 172.21.15.175:0/3826862370 shutdown_connections 2024-07-29T00:45:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 -- 172.21.15.175:0/3826862370 wait complete. 2024-07-29T00:45:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.215+0000 7f654fc36740 1 -- start start 2024-07-29T00:45:35.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 Processor -- start 2024-07-29T00:45:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a18bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a8e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b0dc0 con 0x564019a0a250 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b06f0 con 0x564019a09930 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65481ad950 con 0x564019a14b10 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f654fc36740 10 client.0 _ll_register_callbacks cb 0x7f6540004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.216+0000 7f6545400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59156/0 (socket says 172.21.15.175:59156) 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a18bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545400640 1 -- 172.21.15.175:0/2785738922 learned_addr learned my addr 172.21.15.175:0/2785738922 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6546800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a8e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 -- 172.21.15.175:0/2785738922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 msgr2=0x564019a8e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a8e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 -- 172.21.15.175:0/2785738922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 msgr2=0x564019a190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 -- 172.21.15.175:0/2785738922 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564019a0a080 con 0x564019a09930 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545400640 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:35.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f6545e00640 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a18bb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6538005de0 tx=0x7f6538004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9b824c4ca226d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.217+0000 7f652be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f652be00640 1 -- 172.21.15.175:0/2785738922 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538059060 con 0x564019a09930 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f652be00640 1 -- 172.21.15.175:0/2785738922 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f65380051c0 con 0x564019a09930 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653009ca20 con 0x564019a09930 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f652be00640 1 -- 172.21.15.175:0/2785738922 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6538040040 con 0x564019a09930 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f654fc36740 10 client.27170 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:35.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564019a8fc10 con 0x564019a09930 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f652be00640 1 -- 172.21.15.175:0/2785738922 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653009ca20 con 0x564019a09930 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.218+0000 7f652be00640 1 -- 172.21.15.175:0/2785738922 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6538003c00 con 0x564019a09930 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f654fc36740 10 client.27170 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f654fc36740 10 client.27170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f654fc36740 20 client.27170 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f654fc36740 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a8ff00 0x564019a922f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f654fc36740 4 client.27170 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:45:35.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564019976550 con 0x564019a8ff00 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f6545400640 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a8ff00 0x564019a922f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f6545400640 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a8ff00 0x564019a922f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ea5443feddef72 server_cookie=c81986ace74ed2c0 in_seq=0 out_seq=0 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.219+0000 7f652be00640 10 client.27170 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f652be00640 1 -- 172.21.15.175:0/2785738922 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564019976550 con 0x564019a8ff00 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f652be00640 10 client.27170 handle_command_reply: tid=0 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 client.27170 shutdown 2024-07-29T00:45:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 msgr2=0x564019a18bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a18bb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6538005de0 tx=0x7f6538004da0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 shutdown_connections 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564019a8ff00 0x564019a922f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564019a14b10 0x564019a8e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564019a0a250 0x564019a190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 --2- 172.21.15.175:0/2785738922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564019a09930 0x564019a18bb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f652be00640 0 client.27170 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 >> 172.21.15.175:0/2785738922 conn(0x564019a05ec0 msgr2=0x564019a04ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:35.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 shutdown_connections 2024-07-29T00:45:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 1 -- 172.21.15.175:0/2785738922 wait complete. 2024-07-29T00:45:35.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.220+0000 7f654fc36740 20 client.27170 trim_cache size 0 max 16384 2024-07-29T00:45:35.229 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:45:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:45:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:45:35.230 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:45:35.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f652c0a2cf0 msgr2=0x7f652c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f652c0a2cf0 0x7f652c0a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6540009d20 tx=0x7f6540005f90 comp rx=0 tx=0).stop 2024-07-29T00:45:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548104420 msgr2=0x7f654819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548104420 0x7f654819a870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f653000eb00 tx=0x7f653000c2d0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 shutdown_connections 2024-07-29T00:45:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6548109a80 0x7f654819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6548104420 0x7f654819a870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f652c0a2cf0 0x7f652c0a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 --2- 172.21.15.175:0/1295975923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6548103a80 0x7f654819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 >> 172.21.15.175:0/1295975923 conn(0x7f65480ff780 msgr2=0x7f6548100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 shutdown_connections 2024-07-29T00:45:35.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:35.221+0000 7f654d200640 1 -- 172.21.15.175:0/1295975923 wait complete. 2024-07-29T00:45:36.209 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:35 smithi175 ceph-mon[32138]: pgmap v281: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:36.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:45:36.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:45:36.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:35 smithi181 ceph-mon[36706]: pgmap v281: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:36.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:35 smithi204 ceph-mon[37368]: pgmap v281: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:36.670 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:37.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 -- 172.21.15.175:0/894708774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 msgr2=0x7fa154110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 --2- 172.21.15.175:0/894708774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa154110770 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009cd0 tx=0x7fa14c02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 -- 172.21.15.175:0/894708774 shutdown_connections 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 --2- 172.21.15.175:0/894708774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa154110770 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 --2- 172.21.15.175:0/894708774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1541038a0 0x7fa154103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 --2- 172.21.15.175:0/894708774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 0x7fa154109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 -- 172.21.15.175:0/894708774 >> 172.21.15.175:0/894708774 conn(0x7fa1540ff560 msgr2=0x7fa154101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:37.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 -- 172.21.15.175:0/894708774 shutdown_connections 2024-07-29T00:45:37.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.346+0000 7fa15a200640 1 -- 172.21.15.175:0/894708774 wait complete. 2024-07-29T00:45:37.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 Processor -- start 2024-07-29T00:45:37.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 -- start start 2024-07-29T00:45:37.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1541038a0 0x7fa15419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa15419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 0x7fa15419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa154113cf0 con 0x7fa154104240 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa154113b70 con 0x7fa1541038a0 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.347+0000 7fa15a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa154113e70 con 0x7fa1541098a0 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa15419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:37.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 0x7fa15419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa153e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa15419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59176/0 (socket says 172.21.15.175:59176) 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa153e00640 1 -- 172.21.15.175:0/2607275249 learned_addr learned my addr 172.21.15.175:0/2607275249 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa158e00640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1541038a0 0x7fa15419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 -- 172.21.15.175:0/2607275249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1541038a0 msgr2=0x7fa15419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1541038a0 0x7fa15419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 -- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 msgr2=0x7fa15419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa15419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 -- 172.21.15.175:0/2607275249 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15419f110 con 0x7fa1541098a0 2024-07-29T00:45:37.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.348+0000 7fa159800640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 0x7fa15419ea30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009ca0 tx=0x7fa14c031e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:37.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.349+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c004040 con 0x7fa1541098a0 2024-07-29T00:45:37.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.349+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14c0046c0 con 0x7fa1541098a0 2024-07-29T00:45:37.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.349+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c030e40 con 0x7fa1541098a0 2024-07-29T00:45:37.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.349+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15419f3a0 con 0x7fa1541098a0 2024-07-29T00:45:37.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.349+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541a6ce0 con 0x7fa1541098a0 2024-07-29T00:45:37.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.350+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa154104fe0 con 0x7fa1541098a0 2024-07-29T00:45:37.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.352+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa14c0305a0 con 0x7fa1541098a0 2024-07-29T00:45:37.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.353+0000 7fa151600640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa13c0a2d70 0x7fa13c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:37.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.353+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fa14c02e030 con 0x7fa1541098a0 2024-07-29T00:45:37.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.354+0000 7fa158e00640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa13c0a2d70 0x7fa13c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:37.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.354+0000 7fa158e00640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa13c0a2d70 0x7fa13c0a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa148009b50 tx=0x7fa148005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:37.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.358+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa14c04b060 con 0x7fa1541098a0 2024-07-29T00:45:37.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.586+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa154103d00 con 0x7fa1541098a0 2024-07-29T00:45:37.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.587+0000 7fa151600640 1 -- 172.21.15.175:0/2607275249 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11994 (secure 0 0 0) 0x7fa154103d00 con 0x7fa1541098a0 2024-07-29T00:45:37.587 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:45:37.588 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":43,"btime":"2024-07-29T00:43:57:876956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:57.442709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:45:37.588 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 43 2024-07-29T00:45:37.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa13c0a2d70 msgr2=0x7fa13c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa13c0a2d70 0x7fa13c0a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa148009b50 tx=0x7fa148005f90 comp rx=0 tx=0).stop 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 msgr2=0x7fa15419ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 0x7fa15419ea30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009ca0 tx=0x7fa14c031e60 comp rx=0 tx=0).stop 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 shutdown_connections 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1541098a0 0x7fa15419ea30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa13c0a2d70 0x7fa13c0a5230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa154104240 0x7fa15419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 --2- 172.21.15.175:0/2607275249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1541038a0 0x7fa15419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:37.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.590+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 >> 172.21.15.175:0/2607275249 conn(0x7fa1540ff560 msgr2=0x7fa1540ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:37.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.591+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 shutdown_connections 2024-07-29T00:45:37.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:37.591+0000 7fa15a200640 1 -- 172.21.15.175:0/2607275249 wait complete. 2024-07-29T00:45:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:37 smithi181 ceph-mon[36706]: pgmap v282: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:37 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2607275249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:45:38.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:37 smithi175 ceph-mon[32138]: pgmap v282: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:38.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:37 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2607275249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:45:38.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:37 smithi204 ceph-mon[37368]: pgmap v282: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:38.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:37 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2607275249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:45:38.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:45:38.986 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:39.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 -- 172.21.15.175:0/3921793159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf0104240 msgr2=0x7fabf0110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 --2- 172.21.15.175:0/3921793159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf0104240 0x7fabf0110770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009cd0 tx=0x7fabe802a8f0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 -- 172.21.15.175:0/3921793159 shutdown_connections 2024-07-29T00:45:39.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 --2- 172.21.15.175:0/3921793159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf0104240 0x7fabf0110770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 --2- 172.21.15.175:0/3921793159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabf01038a0 0x7fabf0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 --2- 172.21.15.175:0/3921793159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf01098a0 0x7fabf0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.693+0000 7fabf4e00640 1 -- 172.21.15.175:0/3921793159 >> 172.21.15.175:0/3921793159 conn(0x7fabf00ff560 msgr2=0x7fabf0101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:39.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.694+0000 7fabf4e00640 1 -- 172.21.15.175:0/3921793159 shutdown_connections 2024-07-29T00:45:39.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.694+0000 7fabf4e00640 1 -- 172.21.15.175:0/3921793159 wait complete. 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 Processor -- start 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 -- start start 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabf01038a0 0x7fabf019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf0104240 0x7fabf019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 0x7fabf019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0113cf0 con 0x7fabf0104240 2024-07-29T00:45:39.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0113b70 con 0x7fabf01038a0 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.695+0000 7fabf4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0113e70 con 0x7fabf01098a0 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 0x7fabf019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 0x7fabf019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53054/0 (socket says 172.21.15.175:53054) 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 -- 172.21.15.175:0/2623486735 learned_addr learned my addr 172.21.15.175:0/2623486735 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabeea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf0104240 0x7fabf019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabef400640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabf01038a0 0x7fabf019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 -- 172.21.15.175:0/2623486735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabf01038a0 msgr2=0x7fabf019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabf01038a0 0x7fabf019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 -- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf0104240 msgr2=0x7fabf019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf0104240 0x7fabf019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 -- 172.21.15.175:0/2623486735 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf019f0a0 con 0x7fabf01098a0 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.696+0000 7fabefe00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 0x7fabf019e9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fabe8005e30 tx=0x7fabe8003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.697+0000 7fabe7e00640 1 -- 172.21.15.175:0/2623486735 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8030dc0 con 0x7fabf01098a0 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.697+0000 7fabe7e00640 1 -- 172.21.15.175:0/2623486735 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe8040070 con 0x7fabf01098a0 2024-07-29T00:45:39.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.697+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf019f330 con 0x7fabf01098a0 2024-07-29T00:45:39.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.697+0000 7fabe7e00640 1 -- 172.21.15.175:0/2623486735 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8045ad0 con 0x7fabf01098a0 2024-07-29T00:45:39.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.697+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf01a6c70 con 0x7fabf01098a0 2024-07-29T00:45:39.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.700+0000 7fabe7e00640 1 -- 172.21.15.175:0/2623486735 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fabe8004700 con 0x7fabf01098a0 2024-07-29T00:45:39.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.700+0000 7fabf8e43740 1 Processor -- start 2024-07-29T00:45:39.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.700+0000 7fabf8e43740 1 -- start start 2024-07-29T00:45:39.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4419630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4419fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4426d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabf8e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5572b437fab0 con 0x5572b4419b70 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabf8e43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5572b4317a80 con 0x5572b4424910 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabf8e43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5572b437ba70 con 0x5572b441b0e0 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4419fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4419fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37598/0 (socket says 172.21.15.175:37598) 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabef400640 1 -- 172.21.15.175:0/2372939508 learned_addr learned my addr 172.21.15.175:0/2372939508 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:39.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabeea00640 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4419630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.701+0000 7fabeea00640 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4426d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabef400640 1 -- 172.21.15.175:0/2372939508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 msgr2=0x5572b4419630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabef400640 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4419630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabef400640 1 -- 172.21.15.175:0/2372939508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 msgr2=0x5572b4426d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabef400640 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4426d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabef400640 1 -- 172.21.15.175:0/2372939508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b40f13c0 con 0x5572b4419b70 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabeea00640 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4426d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:45:39.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabe7e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fabd40a2d70 0x7fabd40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabef400640 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4419fb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009c70 tx=0x7fabe0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff14702b3188611 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:39.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabe7e00640 1 -- 172.21.15.175:0/2623486735 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fabe80f0320 con 0x7fabf01098a0 2024-07-29T00:45:39.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabeea00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fabd40a2d70 0x7fabd40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabe4c00640 1 -- 172.21.15.175:0/2372939508 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0004230 con 0x5572b4419b70 2024-07-29T00:45:39.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabe4c00640 1 -- 172.21.15.175:0/2372939508 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe00043d0 con 0x5572b4419b70 2024-07-29T00:45:39.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.702+0000 7fabe4c00640 1 -- 172.21.15.175:0/2372939508 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0019820 con 0x5572b4419b70 2024-07-29T00:45:39.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 -- 172.21.15.175:0/2372939508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 msgr2=0x5572b4419fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabeea00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fabd40a2d70 0x7fabd40a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5572b441a610 tx=0x7fabdc013040 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:39.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4419fb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009c70 tx=0x7fabe0011810 comp rx=0 tx=0).stop 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 -- 172.21.15.175:0/2372939508 shutdown_connections 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4426d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4419fb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 --2- 172.21.15.175:0/2372939508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4419630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.703+0000 7fabf8e43740 1 -- 172.21.15.175:0/2372939508 >> 172.21.15.175:0/2372939508 conn(0x5572b43e2950 msgr2=0x5572b4416200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.704+0000 7fabf8e43740 1 -- 172.21.15.175:0/2372939508 shutdown_connections 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.704+0000 7fabf8e43740 1 -- 172.21.15.175:0/2372939508 wait complete. 2024-07-29T00:45:39.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.704+0000 7fabf8e43740 1 -- start start 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.704+0000 7fabf8e43740 1 Processor -- start 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4421880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4421dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4422300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5572b44276b0 con 0x5572b4419b70 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5572b433ac10 con 0x5572b4424910 2024-07-29T00:45:39.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5572b4427830 con 0x5572b441b0e0 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabf8e43740 10 client.0 _ll_register_callbacks cb 0x5572b4303a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4421880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4421880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37608/0 (socket says 172.21.15.175:37608) 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabef400640 1 -- 172.21.15.175:0/4252990403 learned_addr learned my addr 172.21.15.175:0/4252990403 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabeea00640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4421dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.705+0000 7fabefe00640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4422300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabef400640 1 -- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 msgr2=0x5572b4421dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabef400640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4421dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabef400640 1 -- 172.21.15.175:0/4252990403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 msgr2=0x5572b4422300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabef400640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4422300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabef400640 1 -- 172.21.15.175:0/4252990403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b433e1e0 con 0x5572b4419b70 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabef400640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4421880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009560 tx=0x7fabe00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1fc21737829c5e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.706+0000 7fabd9600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabd9600640 1 -- 172.21.15.175:0/4252990403 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0027070 con 0x5572b4419b70 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabd9600640 1 -- 172.21.15.175:0/4252990403 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fabe0003800 con 0x5572b4419b70 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe00043d0 con 0x5572b4419b70 2024-07-29T00:45:39.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabd9600640 1 -- 172.21.15.175:0/4252990403 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fabe0033050 con 0x5572b4419b70 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabf8e43740 10 client.17040 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5572b4433420 con 0x5572b4419b70 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabd9600640 1 -- 172.21.15.175:0/4252990403 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe00043d0 con 0x5572b4419b70 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.707+0000 7fabd9600640 1 -- 172.21.15.175:0/4252990403 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fabe0018070 con 0x5572b4419b70 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabf8e43740 10 client.17040 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabf8e43740 10 client.17040 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabf8e43740 20 client.17040 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:39.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabf8e43740 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44347d0 0x5572b4436bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabf8e43740 4 client.17040 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:39.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b438d020 con 0x5572b44347d0 2024-07-29T00:45:39.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.708+0000 7fabeea00640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44347d0 0x5572b4436bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.709+0000 7fabeea00640 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44347d0 0x5572b4436bc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd860c6b611d692b server_cookie=a15ebc091f9da851 in_seq=0 out_seq=0 2024-07-29T00:45:39.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.709+0000 7fabd9600640 10 client.17040 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:45:39.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabd9600640 1 -- 172.21.15.175:0/4252990403 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5572b438d020 con 0x5572b44347d0 2024-07-29T00:45:39.715 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabd9600640 10 client.17040 handle_command_reply: tid=0 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 client.17040 shutdown 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 msgr2=0x5572b4421880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4421880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009560 tx=0x7fabe00056d0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 shutdown_connections 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44347d0 0x5572b4436bc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b4422300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b4421dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 --2- 172.21.15.175:0/4252990403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b4421880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 >> 172.21.15.175:0/4252990403 conn(0x5572b43e2950 msgr2=0x5572b441bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabd9600640 0 client.17040 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 shutdown_connections 2024-07-29T00:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 1 -- 172.21.15.175:0/4252990403 wait complete. 2024-07-29T00:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.710+0000 7fabf8e43740 20 client.17040 trim_cache size 0 max 16384 2024-07-29T00:45:39.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 Processor -- start 2024-07-29T00:45:39.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 -- start start 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b4419b70 0x5572b443c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 0x5572b443c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4424910 0x5572b4414b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01af630 con 0x5572b4424910 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b1170 con 0x5572b441b0e0 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.725+0000 7fabf8e43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01afd00 con 0x5572b4419b70 2024-07-29T00:45:39.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 0x5572b443c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 0x5572b443c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35700/0 (socket says 172.21.15.175:35700) 2024-07-29T00:45:39.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 -- 172.21.15.175:0/3091390045 learned_addr learned my addr 172.21.15.175:0/3091390045 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:39.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabef400640 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b4419b70 0x5572b443c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabefe00640 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4424910 0x5572b4414b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 -- 172.21.15.175:0/3091390045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b4419b70 msgr2=0x5572b443c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b4419b70 0x5572b443c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 -- 172.21.15.175:0/3091390045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4424910 msgr2=0x5572b4414b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4424910 0x5572b4414b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 -- 172.21.15.175:0/3091390045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b4416f00 con 0x5572b441b0e0 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabef400640 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b4419b70 0x5572b443c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.726+0000 7fabeea00640 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 0x5572b443c930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0097e0 tx=0x7fabdc009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d6f3141888eca4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.727+0000 7fabdbe00640 1 -- 172.21.15.175:0/3091390045 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc032070 con 0x5572b441b0e0 2024-07-29T00:45:39.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.727+0000 7fabdbe00640 1 -- 172.21.15.175:0/3091390045 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc0044c0 con 0x5572b441b0e0 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.727+0000 7fabdbe00640 1 -- 172.21.15.175:0/3091390045 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0208a0 con 0x5572b441b0e0 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 -- 172.21.15.175:0/3091390045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 msgr2=0x5572b443c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 0x5572b443c930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0097e0 tx=0x7fabdc009770 comp rx=0 tx=0).stop 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 -- 172.21.15.175:0/3091390045 shutdown_connections 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4424910 0x5572b4414b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b441b0e0 0x5572b443c930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 --2- 172.21.15.175:0/3091390045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b4419b70 0x5572b443c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 -- 172.21.15.175:0/3091390045 >> 172.21.15.175:0/3091390045 conn(0x5572b43e2950 msgr2=0x5572b4423900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:39.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 -- 172.21.15.175:0/3091390045 shutdown_connections 2024-07-29T00:45:39.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 -- 172.21.15.175:0/3091390045 wait complete. 2024-07-29T00:45:39.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.728+0000 7fabf8e43740 1 -- start start 2024-07-29T00:45:39.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 Processor -- start 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b442cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b442d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b441e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b0aa0 con 0x5572b4419b70 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b03d0 con 0x5572b4424910 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01ad630 con 0x5572b441b0e0 2024-07-29T00:45:39.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabf8e43740 10 client.0 _ll_register_callbacks cb 0x7fabdc009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b442cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabeea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b442d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b442cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37630/0 (socket says 172.21.15.175:37630) 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabef400640 1 -- 172.21.15.175:0/1489292832 learned_addr learned my addr 172.21.15.175:0/1489292832 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.729+0000 7fabefe00640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b441e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabeea00640 1 -- 172.21.15.175:0/1489292832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 msgr2=0x5572b441e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabeea00640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b441e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabeea00640 1 -- 172.21.15.175:0/1489292832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 msgr2=0x5572b442cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabeea00640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b442cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabeea00640 1 -- 172.21.15.175:0/1489292832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b441e750 con 0x5572b441b0e0 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabef400640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b442cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabeea00640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b442d1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabdc004700 tx=0x7fabdc0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8091bbfabd53d555 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabd9600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabd9600640 1 -- 172.21.15.175:0/1489292832 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc03b060 con 0x5572b441b0e0 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572b4387530 con 0x5572b441b0e0 2024-07-29T00:45:39.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.730+0000 7fabd9600640 1 -- 172.21.15.175:0/1489292832 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fabdc009040 con 0x5572b441b0e0 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabd9600640 1 -- 172.21.15.175:0/1489292832 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fabdc032040 con 0x5572b441b0e0 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 10 client.27142 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5572b44a7550 con 0x5572b441b0e0 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabd9600640 1 -- 172.21.15.175:0/1489292832 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5572b4387530 con 0x5572b441b0e0 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabd9600640 1 -- 172.21.15.175:0/1489292832 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fabdc02c940 con 0x5572b441b0e0 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 10 client.27142 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 10 client.27142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 20 client.27142 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:39.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44a7b80 0x5572b44a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 4 client.27142 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572b4304010 con 0x5572b44a7b80 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.731+0000 7fabef400640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44a7b80 0x5572b44a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabef400640 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44a7b80 0x5572b44a9fa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63d235f2ec605e3b server_cookie=b2fecb199f9786aa in_seq=0 out_seq=0 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabd9600640 10 client.27142 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabd9600640 1 -- 172.21.15.175:0/1489292832 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572b4304010 con 0x5572b44a7b80 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabd9600640 10 client.27142 handle_command_reply: tid=0 2024-07-29T00:45:39.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 client.27142 shutdown 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 msgr2=0x5572b442d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b442d1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabdc004700 tx=0x7fabdc0093b0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 shutdown_connections 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x5572b44a7b80 0x5572b44a9fa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572b4424910 0x5572b441e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572b441b0e0 0x5572b442d1e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 --2- 172.21.15.175:0/1489292832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572b4419b70 0x5572b442cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 >> 172.21.15.175:0/1489292832 conn(0x5572b43e2950 msgr2=0x5572b4420040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:39.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.732+0000 7fabd9600640 0 client.27142 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:45:39.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 shutdown_connections 2024-07-29T00:45:39.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf8e43740 1 -- 172.21.15.175:0/1489292832 wait complete. 2024-07-29T00:45:39.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf8e43740 20 client.27142 trim_cache size 0 max 16384 2024-07-29T00:45:39.740 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:45:39.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fabd40a2d70 msgr2=0x7fabd40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf4e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fabd40a2d70 0x7fabd40a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5572b441a610 tx=0x7fabdc013040 comp rx=0 tx=0).stop 2024-07-29T00:45:39.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 msgr2=0x7fabf019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:39.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.733+0000 7fabf4e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 0x7fabf019e9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fabe8005e30 tx=0x7fabe8003f10 comp rx=0 tx=0).stop 2024-07-29T00:45:39.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 shutdown_connections 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabf01098a0 0x7fabf019e9c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fabd40a2d70 0x7fabd40a5230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabf0104240 0x7fabf019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 --2- 172.21.15.175:0/2623486735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabf01038a0 0x7fabf019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 >> 172.21.15.175:0/2623486735 conn(0x7fabf00ff560 msgr2=0x7fabf0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 shutdown_connections 2024-07-29T00:45:39.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:39.734+0000 7fabf4e00640 1 -- 172.21.15.175:0/2623486735 wait complete. 2024-07-29T00:45:40.200 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:39 smithi175 ceph-mon[32138]: pgmap v283: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:39 smithi181 ceph-mon[36706]: pgmap v283: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:40.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:39 smithi204 ceph-mon[37368]: pgmap v283: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:40.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:45:41.130 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:41.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.837+0000 7f63bea00640 1 -- 172.21.15.175:0/2452501851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8103240 msgr2=0x7f63b810b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.837+0000 7f63bea00640 1 --2- 172.21.15.175:0/2452501851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8103240 0x7f63b810b730 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009900 tx=0x7f63a402a480 comp rx=0 tx=0).stop 2024-07-29T00:45:41.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.837+0000 7f63bea00640 1 -- 172.21.15.175:0/2452501851 shutdown_connections 2024-07-29T00:45:41.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.838+0000 7f63bea00640 1 --2- 172.21.15.175:0/2452501851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b810bcc0 0x7f63b810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.838+0000 7f63bea00640 1 --2- 172.21.15.175:0/2452501851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8103240 0x7f63b810b730 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.838+0000 7f63bea00640 1 --2- 172.21.15.175:0/2452501851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b8102920 0x7f63b8102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.838+0000 7f63bea00640 1 -- 172.21.15.175:0/2452501851 >> 172.21.15.175:0/2452501851 conn(0x7f63b80fc300 msgr2=0x7f63b80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:41.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.838+0000 7f63bea00640 1 -- 172.21.15.175:0/2452501851 shutdown_connections 2024-07-29T00:45:41.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.838+0000 7f63bea00640 1 -- 172.21.15.175:0/2452501851 wait complete. 2024-07-29T00:45:41.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 Processor -- start 2024-07-29T00:45:41.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 -- start start 2024-07-29T00:45:41.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8102920 0x7f63b819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 0x7f63b819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b810bcc0 0x7f63b819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8111ab0 con 0x7f63b8102920 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8111930 con 0x7f63b810bcc0 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.839+0000 7f63bea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8111c30 con 0x7f63b8103240 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8102920 0x7f63b819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 0x7f63b819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 0x7f63b819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53108/0 (socket says 172.21.15.175:53108) 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 -- 172.21.15.175:0/566780629 learned_addr learned my addr 172.21.15.175:0/566780629 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63be000640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b810bcc0 0x7f63b819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 -- 172.21.15.175:0/566780629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b810bcc0 msgr2=0x7f63b819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b810bcc0 0x7f63b819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 -- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8102920 msgr2=0x7f63b819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8102920 0x7f63b819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 -- 172.21.15.175:0/566780629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b819f160 con 0x7f63b8103240 2024-07-29T00:45:41.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63be000640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b810bcc0 0x7f63b819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:41.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.840+0000 7f63bcc00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 0x7f63b819a6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006010 tx=0x7f63a4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:41.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.841+0000 7f63b6000640 1 -- 172.21.15.175:0/566780629 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a403f030 con 0x7f63b8103240 2024-07-29T00:45:41.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.841+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b819f3f0 con 0x7f63b8103240 2024-07-29T00:45:41.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.841+0000 7f63b6000640 1 -- 172.21.15.175:0/566780629 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a4004440 con 0x7f63b8103240 2024-07-29T00:45:41.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.841+0000 7f63b6000640 1 -- 172.21.15.175:0/566780629 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4035070 con 0x7f63b8103240 2024-07-29T00:45:41.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.841+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81a6d30 con 0x7f63b8103240 2024-07-29T00:45:41.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.844+0000 7f63c2c79740 1 Processor -- start 2024-07-29T00:45:41.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.844+0000 7f63c2c79740 1 -- start start 2024-07-29T00:45:41.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63b6000640 1 -- 172.21.15.175:0/566780629 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f63a4042030 con 0x7f63b8103240 2024-07-29T00:45:41.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 0x56458f576630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f576fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f583d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63c2c79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56458f4dcab0 con 0x56458f5780e0 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63c2c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56458f474a80 con 0x56458f576b70 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63c2c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56458f4d8a70 con 0x56458f581910 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 0x56458f576630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 0x56458f576630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37648/0 (socket says 172.21.15.175:37648) 2024-07-29T00:45:41.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63be000640 1 -- 172.21.15.175:0/3944977935 learned_addr learned my addr 172.21.15.175:0/3944977935 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:41.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63bd600640 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f576fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.845+0000 7f63bcc00640 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f583d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.848 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63be000640 1 -- 172.21.15.175:0/3944977935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 msgr2=0x56458f583d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63be000640 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f583d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63be000640 1 -- 172.21.15.175:0/3944977935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 msgr2=0x56458f576fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63be000640 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f576fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63be000640 1 -- 172.21.15.175:0/3944977935 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458f24e3c0 con 0x56458f5780e0 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63be000640 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 0x56458f576630 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63b0011440 tx=0x7f63b0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44f8137ce1dc855 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63aa600640 1 -- 172.21.15.175:0/3944977935 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0004020 con 0x56458f5780e0 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63aa600640 1 -- 172.21.15.175:0/3944977935 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b00041c0 con 0x56458f5780e0 2024-07-29T00:45:41.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.846+0000 7f63aa600640 1 -- 172.21.15.175:0/3944977935 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000f940 con 0x56458f5780e0 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63b6000640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63a00a2d70 0x7f63a00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63b6000640 1 -- 172.21.15.175:0/566780629 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f63a40ef870 con 0x7f63b8103240 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63bd600640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63a00a2d70 0x7f63a00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 -- 172.21.15.175:0/3944977935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 msgr2=0x56458f576630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 0x56458f576630 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63b0011440 tx=0x7f63b0011900 comp rx=0 tx=0).stop 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 -- 172.21.15.175:0/3944977935 shutdown_connections 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f583d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f576fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3944977935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f5780e0 0x56458f576630 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.847+0000 7f63c2c79740 1 -- 172.21.15.175:0/3944977935 >> 172.21.15.175:0/3944977935 conn(0x56458f53f950 msgr2=0x56458f573200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.848+0000 7f63c2c79740 1 -- 172.21.15.175:0/3944977935 shutdown_connections 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.848+0000 7f63c2c79740 1 -- 172.21.15.175:0/3944977935 wait complete. 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.848+0000 7f63bd600640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63a00a2d70 0x7f63a00a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x56458f574060 tx=0x7f63ac009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.848+0000 7f63c2c79740 1 -- start start 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.848+0000 7f63c2c79740 1 Processor -- start 2024-07-29T00:45:41.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f57e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f57ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f57f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56458f5846b0 con 0x56458f576b70 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56458f497c10 con 0x56458f5780e0 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56458f584830 con 0x56458f581910 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63c2c79740 10 client.0 _ll_register_callbacks cb 0x56458f460a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63bd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f57e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63bd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f57e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37662/0 (socket says 172.21.15.175:37662) 2024-07-29T00:45:41.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63bd600640 1 -- 172.21.15.175:0/631128306 learned_addr learned my addr 172.21.15.175:0/631128306 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:41.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63bcc00640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f57ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.849+0000 7f63be000640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f57f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63bcc00640 1 -- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 msgr2=0x56458f57f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63bcc00640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f57f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63bcc00640 1 -- 172.21.15.175:0/631128306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 msgr2=0x56458f57e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63bcc00640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f57e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63bcc00640 1 -- 172.21.15.175:0/631128306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458f49b1e0 con 0x56458f5780e0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63bcc00640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f57ee20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f63a40ecf10 tx=0x7f63a40b80f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02906f24a6d9005 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f639ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f639ea00640 1 -- 172.21.15.175:0/631128306 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4049070 con 0x56458f5780e0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f639ea00640 1 -- 172.21.15.175:0/631128306 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f63a4031bd0 con 0x56458f5780e0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b00041c0 con 0x56458f5780e0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f639ea00640 1 -- 172.21.15.175:0/631128306 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f63a40bb7b0 con 0x56458f5780e0 2024-07-29T00:45:41.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63c2c79740 10 client.27206 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.850+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56458f590470 con 0x56458f5780e0 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f639ea00640 1 -- 172.21.15.175:0/631128306 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b00041c0 con 0x56458f5780e0 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f639ea00640 1 -- 172.21.15.175:0/631128306 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f63a40c4580 con 0x56458f5780e0 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63c2c79740 10 client.27206 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63c2c79740 10 client.27206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63c2c79740 20 client.27206 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63c2c79740 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f591860 0x56458f593c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.857 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63c2c79740 4 client.27206 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56458f4ea020 con 0x56458f591860 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.851+0000 7f63bd600640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f591860 0x56458f593c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.852+0000 7f63bd600640 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f591860 0x56458f593c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c41bd428eda7dd4 server_cookie=5b7496e518ec7340 in_seq=0 out_seq=0 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.852+0000 7f639ea00640 10 client.27206 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f639ea00640 1 -- 172.21.15.175:0/631128306 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56458f4ea020 con 0x56458f591860 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f639ea00640 10 client.27206 handle_command_reply: tid=0 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 client.27206 shutdown 2024-07-29T00:45:41.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 msgr2=0x56458f57ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f57ee20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f63a40ecf10 tx=0x7f63a40b80f0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 shutdown_connections 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f591860 0x56458f593c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f57f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f57ee20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 --2- 172.21.15.175:0/631128306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f57e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f639ea00640 0 client.27206 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 >> 172.21.15.175:0/631128306 conn(0x56458f53f950 msgr2=0x56458f578ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 shutdown_connections 2024-07-29T00:45:41.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 1 -- 172.21.15.175:0/631128306 wait complete. 2024-07-29T00:45:41.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.853+0000 7f63c2c79740 20 client.27206 trim_cache size 0 max 16384 2024-07-29T00:45:41.866 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 Processor -- start 2024-07-29T00:45:41.867 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 -- start start 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f580850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f599560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f599aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81af780 con 0x56458f576b70 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81b12c0 con 0x56458f5780e0 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63c2c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81afe50 con 0x56458f581910 2024-07-29T00:45:41.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63bd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f580850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63bd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f580850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37678/0 (socket says 172.21.15.175:37678) 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63bd600640 1 -- 172.21.15.175:0/3104149617 learned_addr learned my addr 172.21.15.175:0/3104149617 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63bcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f599560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.868+0000 7f63be000640 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f599aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.869+0000 7f63bcc00640 1 -- 172.21.15.175:0/3104149617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 msgr2=0x56458f599aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.869+0000 7f63bcc00640 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f599aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.869+0000 7f63bcc00640 1 -- 172.21.15.175:0/3104149617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 msgr2=0x56458f580850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.869+0000 7f63bcc00640 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f580850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.869+0000 7f63bcc00640 1 -- 172.21.15.175:0/3104149617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458f574160 con 0x56458f5780e0 2024-07-29T00:45:41.870 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.869+0000 7f63bd600640 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f580850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:41.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.882+0000 7f63bcc00640 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f599560 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f63a4003cd0 tx=0x7f63a40b86d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9285c8a1d05946ab server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:41.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.882+0000 7f63a9c00640 1 -- 172.21.15.175:0/3104149617 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4049070 con 0x56458f5780e0 2024-07-29T00:45:41.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.882+0000 7f63a9c00640 1 -- 172.21.15.175:0/3104149617 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a40f2e50 con 0x56458f5780e0 2024-07-29T00:45:41.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.882+0000 7f63a9c00640 1 -- 172.21.15.175:0/3104149617 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4031bd0 con 0x56458f5780e0 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.883+0000 7f63c2c79740 1 -- 172.21.15.175:0/3104149617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 msgr2=0x56458f599560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.883+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f599560 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f63a4003cd0 tx=0x7f63a40b86d0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 -- 172.21.15.175:0/3104149617 shutdown_connections 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f581910 0x56458f599aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f5780e0 0x56458f599560 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 --2- 172.21.15.175:0/3104149617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f576b70 0x56458f580850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 -- 172.21.15.175:0/3104149617 >> 172.21.15.175:0/3104149617 conn(0x56458f53f950 msgr2=0x56458f580140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:41.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 -- 172.21.15.175:0/3104149617 shutdown_connections 2024-07-29T00:45:41.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 -- 172.21.15.175:0/3104149617 wait complete. 2024-07-29T00:45:41.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 -- start start 2024-07-29T00:45:41.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.884+0000 7f63c2c79740 1 Processor -- start 2024-07-29T00:45:41.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f58a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f5780e0 0x56458f5856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f581910 0x56458f585cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81b0bf0 con 0x56458f581910 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81b0520 con 0x56458f576b70 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81ad780 con 0x56458f5780e0 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63c2c79740 10 client.0 _ll_register_callbacks cb 0x7f63a40b79d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f581910 0x56458f585cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63bd600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f58a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63bcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f5780e0 0x56458f5856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63bd600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f58a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35760/0 (socket says 172.21.15.175:35760) 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f581910 0x56458f585cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37692/0 (socket says 172.21.15.175:37692) 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.885+0000 7f63bd600640 1 -- 172.21.15.175:0/2492786487 learned_addr learned my addr 172.21.15.175:0/2492786487 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63bd600640 1 -- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f5780e0 msgr2=0x56458f5856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63bd600640 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f5780e0 0x56458f5856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63bd600640 1 -- 172.21.15.175:0/2492786487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f581910 msgr2=0x56458f585cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63bd600640 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f581910 0x56458f585cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63bd600640 1 -- 172.21.15.175:0/2492786487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458f5864e0 con 0x56458f576b70 2024-07-29T00:45:41.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63bd600640 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f58a2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f63ac012010 tx=0x7f63ac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9da011f479a3d6f server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:41.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f639ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f639ea00640 1 -- 172.21.15.175:0/2492786487 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac004b20 con 0x56458f576b70 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.886+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a40f2e50 con 0x56458f576b70 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f639ea00640 1 -- 172.21.15.175:0/2492786487 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f63ac004cc0 con 0x56458f576b70 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f639ea00640 1 -- 172.21.15.175:0/2492786487 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f63ac02a380 con 0x56458f576b70 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 10 client.27212 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56458f5fc670 con 0x56458f576b70 2024-07-29T00:45:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f639ea00640 1 -- 172.21.15.175:0/2492786487 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a40f2e50 con 0x56458f576b70 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f639ea00640 1 -- 172.21.15.175:0/2492786487 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f63ac00f7e0 con 0x56458f576b70 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 10 client.27212 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 10 client.27212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 20 client.27212 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f5fcda0 0x56458f5ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 4 client.27212 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:45:41.891 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.887+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56458f4e4530 con 0x56458f5fcda0 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.888+0000 7f63bcc00640 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f5fcda0 0x56458f5ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.888+0000 7f63bcc00640 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f5fcda0 0x56458f5ff1c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd4e77477fbb7440 server_cookie=cfcea1924e6c117f in_seq=0 out_seq=0 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.888+0000 7f639ea00640 10 client.27212 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f639ea00640 1 -- 172.21.15.175:0/2492786487 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56458f4e4530 con 0x56458f5fcda0 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f639ea00640 10 client.27212 handle_command_reply: tid=0 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 client.27212 shutdown 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 msgr2=0x56458f58a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f58a2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f63ac012010 tx=0x7f63ac004a40 comp rx=0 tx=0).stop 2024-07-29T00:45:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 shutdown_connections 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56458f5fcda0 0x56458f5ff1c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56458f581910 0x56458f585cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458f5780e0 0x56458f5856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f639ea00640 0 client.27212 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 --2- 172.21.15.175:0/2492786487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56458f576b70 0x56458f58a2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 >> 172.21.15.175:0/2492786487 conn(0x56458f53f950 msgr2=0x56458f572360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 shutdown_connections 2024-07-29T00:45:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 1 -- 172.21.15.175:0/2492786487 wait complete. 2024-07-29T00:45:41.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.889+0000 7f63c2c79740 20 client.27212 trim_cache size 0 max 16384 2024-07-29T00:45:41.894 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63a00a2d70 msgr2=0x7f63a00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63a00a2d70 0x7f63a00a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x56458f574060 tx=0x7f63ac009500 comp rx=0 tx=0).stop 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 msgr2=0x7f63b819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 0x7f63b819a6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006010 tx=0x7f63a4004040 comp rx=0 tx=0).stop 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 shutdown_connections 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b810bcc0 0x7f63b819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63b8103240 0x7f63b819a6f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f63a00a2d70 0x7f63a00a5230 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 --2- 172.21.15.175:0/566780629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b8102920 0x7f63b819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:41.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 >> 172.21.15.175:0/566780629 conn(0x7f63b80fc300 msgr2=0x7f63b8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:41.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 shutdown_connections 2024-07-29T00:45:41.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:41.890+0000 7f63bea00640 1 -- 172.21.15.175:0/566780629 wait complete. 2024-07-29T00:45:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:41 smithi181 ceph-mon[36706]: pgmap v284: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:42.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:41 smithi175 ceph-mon[32138]: pgmap v284: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:41 smithi204 ceph-mon[37368]: pgmap v284: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:42.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:45:43.345 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:43.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 -- 172.21.15.175:0/1727902785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 msgr2=0x7f78d810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:43.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 --2- 172.21.15.175:0/1727902785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 0x7f78d810e130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009930 tx=0x7f78cc02a440 comp rx=0 tx=0).stop 2024-07-29T00:45:43.996 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 -- 172.21.15.175:0/1727902785 shutdown_connections 2024-07-29T00:45:43.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 --2- 172.21.15.175:0/1727902785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 0x7f78d8114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:43.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 --2- 172.21.15.175:0/1727902785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 0x7f78d8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:43.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 --2- 172.21.15.175:0/1727902785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 0x7f78d810e130 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:43.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 -- 172.21.15.175:0/1727902785 >> 172.21.15.175:0/1727902785 conn(0x7f78d8077670 msgr2=0x7f78d8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:43.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.996+0000 7f78df200640 1 -- 172.21.15.175:0/1727902785 shutdown_connections 2024-07-29T00:45:43.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.997+0000 7f78df200640 1 -- 172.21.15.175:0/1727902785 wait complete. 2024-07-29T00:45:43.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 Processor -- start 2024-07-29T00:45:43.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 -- start start 2024-07-29T00:45:43.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 0x7f78d819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:43.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 0x7f78d819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:43.999 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 0x7f78d81a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8118070 con 0x7f78d810dd50 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8117ef0 con 0x7f78d8107d10 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.998+0000 7f78df200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81181f0 con 0x7f78d8108670 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 0x7f78d819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 0x7f78d819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35768/0 (socket says 172.21.15.175:35768) 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dde00640 1 -- 172.21.15.175:0/579738309 learned_addr learned my addr 172.21.15.175:0/579738309 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 0x7f78d819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78de800640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 0x7f78d81a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.000 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dd400640 1 -- 172.21.15.175:0/579738309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 msgr2=0x7f78d819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dd400640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 0x7f78d819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dd400640 1 -- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 msgr2=0x7f78d81a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dd400640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 0x7f78d81a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:43.999+0000 7f78dd400640 1 -- 172.21.15.175:0/579738309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81a3550 con 0x7f78d8108670 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.000+0000 7f78dd400640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 0x7f78d819eae0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e9f0 tx=0x7f78c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.000+0000 7f78d2a00640 1 -- 172.21.15.175:0/579738309 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c800cd80 con 0x7f78d8108670 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.000+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81a3840 con 0x7f78d8108670 2024-07-29T00:45:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.000+0000 7f78d2a00640 1 -- 172.21.15.175:0/579738309 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c80035f0 con 0x7f78d8108670 2024-07-29T00:45:44.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.000+0000 7f78d2a00640 1 -- 172.21.15.175:0/579738309 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8010670 con 0x7f78d8108670 2024-07-29T00:45:44.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.000+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81ab120 con 0x7f78d8108670 2024-07-29T00:45:44.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.003+0000 7f78d2a00640 1 -- 172.21.15.175:0/579738309 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f78c80037a0 con 0x7f78d8108670 2024-07-29T00:45:44.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 Processor -- start 2024-07-29T00:45:44.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 -- start start 2024-07-29T00:45:44.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c0010e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0010efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0011bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c00074ab0 con 0x559c00119910 2024-07-29T00:45:44.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559c0000ca80 con 0x559c001100e0 2024-07-29T00:45:44.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.004+0000 7f78e33ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559c00070a70 con 0x559c0010eb70 2024-07-29T00:45:44.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78de800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c0010e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0010efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0011bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0010efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53188/0 (socket says 172.21.15.175:53188) 2024-07-29T00:45:44.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78de800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c0010e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35776/0 (socket says 172.21.15.175:35776) 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 -- 172.21.15.175:0/1123571783 learned_addr learned my addr 172.21.15.175:0/1123571783 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78d2a00640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f78c00a2cf0 0x7f78c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 -- 172.21.15.175:0/1123571783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 msgr2=0x559c0010e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c0010e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 -- 172.21.15.175:0/1123571783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 msgr2=0x559c0011bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0011bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78dde00640 1 -- 172.21.15.175:0/1123571783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bffde63c0 con 0x559c0010eb70 2024-07-29T00:45:44.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.005+0000 7f78d2a00640 1 -- 172.21.15.175:0/579738309 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f78c80cf7e0 con 0x7f78d8108670 2024-07-29T00:45:44.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.006+0000 7f78de800640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f78c00a2cf0 0x7f78c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.006+0000 7f78dde00640 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0010efb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f78cc006010 tx=0x7f78cc005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be28baa7cf4fd0a server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:44.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.006+0000 7f78c7000640 1 -- 172.21.15.175:0/1123571783 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc040070 con 0x559c0010eb70 2024-07-29T00:45:44.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.006+0000 7f78c7000640 1 -- 172.21.15.175:0/1123571783 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc0318f0 con 0x559c0010eb70 2024-07-29T00:45:44.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.006+0000 7f78c7000640 1 -- 172.21.15.175:0/1123571783 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc035040 con 0x559c0010eb70 2024-07-29T00:45:44.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 -- 172.21.15.175:0/1123571783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 msgr2=0x559c0010efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0010efb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f78cc006010 tx=0x7f78cc005dc0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78de800640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f78c00a2cf0 0x7f78c00a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78d4005c40 tx=0x7f78d4004a40 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 -- 172.21.15.175:0/1123571783 shutdown_connections 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0011bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0010efb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1123571783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c0010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 -- 172.21.15.175:0/1123571783 >> 172.21.15.175:0/1123571783 conn(0x559c000d7950 msgr2=0x559c0010b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 -- 172.21.15.175:0/1123571783 shutdown_connections 2024-07-29T00:45:44.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.007+0000 7f78e33ca740 1 -- 172.21.15.175:0/1123571783 wait complete. 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 -- start start 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 Processor -- start 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c00116840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 0x559c00116d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c001172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c0011c650 con 0x559c001100e0 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559c0002fc10 con 0x559c00119910 2024-07-29T00:45:44.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.008+0000 7f78e33ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559c0011c7d0 con 0x559c0010eb70 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78e33ca740 10 client.0 _ll_register_callbacks cb 0x559bffff8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 0x559c00116d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 0x559c00116d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37726/0 (socket says 172.21.15.175:37726) 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 -- 172.21.15.175:0/1651163942 learned_addr learned my addr 172.21.15.175:0/1651163942 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dde00640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c00116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78de800640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c001172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 -- 172.21.15.175:0/1651163942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 msgr2=0x559c00116840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c00116840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 -- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 msgr2=0x559c001172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c001172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.009+0000 7f78dd400640 1 -- 172.21.15.175:0/1651163942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c000331e0 con 0x559c001100e0 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78dd400640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 0x559c00116d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78c809bbe0 tx=0x7f78c809b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d231d45a42310d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78bb400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78bb400640 1 -- 172.21.15.175:0/1651163942 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c80b1020 con 0x559c001100e0 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78bb400640 1 -- 172.21.15.175:0/1651163942 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f78c80982c0 con 0x559c001100e0 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc0318f0 con 0x559c001100e0 2024-07-29T00:45:44.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78bb400640 1 -- 172.21.15.175:0/1651163942 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f78c80b9050 con 0x559c001100e0 2024-07-29T00:45:44.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78e33ca740 10 client.17076 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:44.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.010+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559c00128390 con 0x559c001100e0 2024-07-29T00:45:44.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78bb400640 1 -- 172.21.15.175:0/1651163942 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc0318f0 con 0x559c001100e0 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78bb400640 1 -- 172.21.15.175:0/1651163942 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f78c80cfbd0 con 0x559c001100e0 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78e33ca740 10 client.17076 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78e33ca740 10 client.17076 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78e33ca740 20 client.17076 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c001297a0 0x559c0012bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78e33ca740 4 client.17076 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.011+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c00082020 con 0x559c001297a0 2024-07-29T00:45:44.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.012+0000 7f78dde00640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c001297a0 0x559c0012bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.012+0000 7f78dde00640 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c001297a0 0x559c0012bbe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7409c86a0c63ebe9 server_cookie=eeadb365610692cd in_seq=0 out_seq=0 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.012+0000 7f78bb400640 10 client.17076 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78bb400640 1 -- 172.21.15.175:0/1651163942 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c00082020 con 0x559c001297a0 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78bb400640 10 client.17076 handle_command_reply: tid=0 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 client.17076 shutdown 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 msgr2=0x559c00116d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 0x559c00116d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78c809bbe0 tx=0x7f78c809b6a0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 shutdown_connections 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c001297a0 0x559c0012bbe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c001172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c001100e0 0x559c00116d80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 --2- 172.21.15.175:0/1651163942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c00116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78bb400640 0 client.17076 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.013+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 >> 172.21.15.175:0/1651163942 conn(0x559c000d7950 msgr2=0x559c00110f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.014+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 shutdown_connections 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.014+0000 7f78e33ca740 1 -- 172.21.15.175:0/1651163942 wait complete. 2024-07-29T00:45:44.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.014+0000 7f78e33ca740 20 client.17076 trim_cache size 0 max 16384 2024-07-29T00:45:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 Processor -- start 2024-07-29T00:45:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 -- start start 2024-07-29T00:45:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0011b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c001187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0010ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81b3bf0 con 0x559c00119910 2024-07-29T00:45:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81b5730 con 0x559c001100e0 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.028+0000 7f78e33ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81b42c0 con 0x559c0010eb70 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78de800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0010ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0011b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0011b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53202/0 (socket says 172.21.15.175:53202) 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78de800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0010ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37730/0 (socket says 172.21.15.175:37730) 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 -- 172.21.15.175:0/2500866145 learned_addr learned my addr 172.21.15.175:0/2500866145 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dd400640 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c001187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 -- 172.21.15.175:0/2500866145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 msgr2=0x559c001187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c001187b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 -- 172.21.15.175:0/2500866145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 msgr2=0x559c0010ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0010ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.029+0000 7f78dde00640 1 -- 172.21.15.175:0/2500866145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c001157f0 con 0x559c0010eb70 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.030+0000 7f78dde00640 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0011b9b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005500 tx=0x7f78cc02a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6644546406f1420d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.030+0000 7f78c6600640 1 -- 172.21.15.175:0/2500866145 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc040040 con 0x559c0010eb70 2024-07-29T00:45:44.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.030+0000 7f78c6600640 1 -- 172.21.15.175:0/2500866145 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc0035f0 con 0x559c0010eb70 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.030+0000 7f78c6600640 1 -- 172.21.15.175:0/2500866145 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc035040 con 0x559c0010eb70 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.030+0000 7f78e33ca740 1 -- 172.21.15.175:0/2500866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 msgr2=0x559c0011b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.030+0000 7f78e33ca740 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0011b9b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005500 tx=0x7f78cc02a440 comp rx=0 tx=0).stop 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 -- 172.21.15.175:0/2500866145 shutdown_connections 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c00119910 0x559c0010ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c001100e0 0x559c001187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 --2- 172.21.15.175:0/2500866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c0010eb70 0x559c0011b9b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 -- 172.21.15.175:0/2500866145 >> 172.21.15.175:0/2500866145 conn(0x559c000d7950 msgr2=0x559c001180a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 -- 172.21.15.175:0/2500866145 shutdown_connections 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 -- 172.21.15.175:0/2500866145 wait complete. 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 -- start start 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.031+0000 7f78e33ca740 1 Processor -- start 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 0x559c00122190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c001100e0 0x559c0011a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c0011aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81b5060 con 0x559c0010eb70 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81b4990 con 0x559c00119910 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81b1bf0 con 0x559c001100e0 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78e33ca740 10 client.0 _ll_register_callbacks cb 0x7f78c8003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 0x559c00122190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 0x559c00122190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37746/0 (socket says 172.21.15.175:37746) 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78dde00640 1 -- 172.21.15.175:0/167938529 learned_addr learned my addr 172.21.15.175:0/167938529 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78dd400640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c001100e0 0x559c0011a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.032+0000 7f78de800640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c0011aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78dde00640 1 -- 172.21.15.175:0/167938529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c001100e0 msgr2=0x559c0011a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78dde00640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c001100e0 0x559c0011a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78dde00640 1 -- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 msgr2=0x559c0011aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78dde00640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c0011aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78dde00640 1 -- 172.21.15.175:0/167938529 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0011b6b0 con 0x559c0010eb70 2024-07-29T00:45:44.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78dde00640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 0x559c00122190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005c40 tx=0x7f78cc0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0ca585945dd659 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:44.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78bb400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:44.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78bb400640 1 -- 172.21.15.175:0/167938529 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc05e020 con 0x559c0010eb70 2024-07-29T00:45:44.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78bb400640 1 -- 172.21.15.175:0/167938529 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f78cc004200 con 0x559c0010eb70 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.033+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc0035f0 con 0x559c0010eb70 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78bb400640 1 -- 172.21.15.175:0/167938529 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f78cc069050 con 0x559c0010eb70 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 10 client.17082 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559c00194480 con 0x559c0010eb70 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78bb400640 1 -- 172.21.15.175:0/167938529 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc0035f0 con 0x559c0010eb70 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78bb400640 1 -- 172.21.15.175:0/167938529 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f78cc050560 con 0x559c0010eb70 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 10 client.17082 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 10 client.17082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:45:44.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 20 client.17082 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c00194540 0x559c00196930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 4 client.17082 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.034+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c0011c090 con 0x559c00194540 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78dd400640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c00194540 0x559c00196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78dd400640 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c00194540 0x559c00196930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2377ff93a87902d1 server_cookie=5b9ebee6379486b5 in_seq=0 out_seq=0 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78bb400640 10 client.17082 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78bb400640 1 -- 172.21.15.175:0/167938529 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c0011c090 con 0x559c00194540 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78bb400640 10 client.17082 handle_command_reply: tid=0 2024-07-29T00:45:44.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78e33ca740 1 client.17082 shutdown 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 msgr2=0x559c00122190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.035+0000 7f78e33ca740 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 0x559c00122190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005c40 tx=0x7f78cc0316f0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 shutdown_connections 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559c00194540 0x559c00196930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559c00119910 0x559c0011aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c001100e0 0x559c0011a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 --2- 172.21.15.175:0/167938529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c0010eb70 0x559c00122190 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 >> 172.21.15.175:0/167938529 conn(0x559c000d7950 msgr2=0x559c00117020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:44.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78bb400640 0 client.17082 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:45:44.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 shutdown_connections 2024-07-29T00:45:44.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 1 -- 172.21.15.175:0/167938529 wait complete. 2024-07-29T00:45:44.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.036+0000 7f78e33ca740 20 client.17082 trim_cache size 0 max 16384 2024-07-29T00:45:44.042 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:45:44.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f78c00a2cf0 msgr2=0x7f78c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f78c00a2cf0 0x7f78c00a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78d4005c40 tx=0x7f78d4004a40 comp rx=0 tx=0).stop 2024-07-29T00:45:44.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 msgr2=0x7f78d819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:44.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 0x7f78d819eae0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e9f0 tx=0x7f78c800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 shutdown_connections 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f78c00a2cf0 0x7f78c00a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d810dd50 0x7f78d81a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d8108670 0x7f78d819eae0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 --2- 172.21.15.175:0/579738309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d8107d10 0x7f78d819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 >> 172.21.15.175:0/579738309 conn(0x7f78d8077670 msgr2=0x7f78d8078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 shutdown_connections 2024-07-29T00:45:44.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:44.037+0000 7f78df200640 1 -- 172.21.15.175:0/579738309 wait complete. 2024-07-29T00:45:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:43 smithi181 ceph-mon[36706]: pgmap v285: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:44.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:43 smithi175 ceph-mon[32138]: pgmap v285: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:44.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:43 smithi204 ceph-mon[37368]: pgmap v285: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:44.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:45:45.383 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:46.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 -- 172.21.15.175:0/1403461879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef010d220 msgr2=0x7feef0114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 --2- 172.21.15.175:0/1403461879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef010d220 0x7feef0114da0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feee8009900 tx=0x7feee802a410 comp rx=0 tx=0).stop 2024-07-29T00:45:46.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 -- 172.21.15.175:0/1403461879 shutdown_connections 2024-07-29T00:45:46.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 --2- 172.21.15.175:0/1403461879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef010d220 0x7feef0114da0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 --2- 172.21.15.175:0/1403461879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef0108af0 0x7feef010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 --2- 172.21.15.175:0/1403461879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef0108140 0x7feef0108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.042+0000 7feef6000640 1 -- 172.21.15.175:0/1403461879 >> 172.21.15.175:0/1403461879 conn(0x7feef0077890 msgr2=0x7feef0079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.043+0000 7feef6000640 1 -- 172.21.15.175:0/1403461879 shutdown_connections 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.043+0000 7feef6000640 1 -- 172.21.15.175:0/1403461879 wait complete. 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.043+0000 7feef6000640 1 Processor -- start 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 -- start start 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 0x7feef019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef0108af0 0x7feef019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef010d220 0x7feef01a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feef01182a0 con 0x7feef0108140 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feef0118120 con 0x7feef010d220 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feef6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feef0118420 con 0x7feef0108af0 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feeef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 0x7feef019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feeef400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 0x7feef019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37770/0 (socket says 172.21.15.175:37770) 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feeef400640 1 -- 172.21.15.175:0/1581857512 learned_addr learned my addr 172.21.15.175:0/1581857512 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feee6a00640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef0108af0 0x7feef019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.044+0000 7feeefe00640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef010d220 0x7feef01a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeef400640 1 -- 172.21.15.175:0/1581857512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef0108af0 msgr2=0x7feef019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeef400640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef0108af0 0x7feef019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeef400640 1 -- 172.21.15.175:0/1581857512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef010d220 msgr2=0x7feef01a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeef400640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef010d220 0x7feef01a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeef400640 1 -- 172.21.15.175:0/1581857512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef01a3690 con 0x7feef0108140 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeef400640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 0x7feef019e6e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feee000ebc0 tx=0x7feee000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeecc00640 1 -- 172.21.15.175:0/1581857512 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0010630 con 0x7feef0108140 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.045+0000 7feeecc00640 1 -- 172.21.15.175:0/1581857512 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee0010cb0 con 0x7feef0108140 2024-07-29T00:45:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.046+0000 7feeecc00640 1 -- 172.21.15.175:0/1581857512 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0018bd0 con 0x7feef0108140 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.046+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef01a3980 con 0x7feef0108140 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.046+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01ab260 con 0x7feef0108140 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feeecc00640 1 -- 172.21.15.175:0/1581857512 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7feee00107d0 con 0x7feef0108140 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 Processor -- start 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 -- start start 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d2234e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d2234edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d2235bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564d222b4ab0 con 0x564d22359720 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564d2224ca80 con 0x564d2234fef0 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.049+0000 7feef8bdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564d222b0a70 con 0x564d2234e980 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d2234edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeefe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d2234e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feee6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d2235bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d2234edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53242/0 (socket says 172.21.15.175:53242) 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 -- 172.21.15.175:0/3895266310 learned_addr learned my addr 172.21.15.175:0/3895266310 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 -- 172.21.15.175:0/3895266310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 msgr2=0x564d2234e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d2234e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 -- 172.21.15.175:0/3895266310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 msgr2=0x564d2235bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d2235bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.050+0000 7feeef400640 1 -- 172.21.15.175:0/3895266310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d220263c0 con 0x564d2234e980 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feeef400640 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d2234edc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feee00cd080 tx=0x7feee00d8190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8350f6dbe3bc0030 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feeecc00640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feed40a2d70 0x7feed40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feedfe00640 1 -- 172.21.15.175:0/3895266310 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00d6070 con 0x564d2234e980 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feedfe00640 1 -- 172.21.15.175:0/3895266310 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00d8dd0 con 0x564d2234e980 2024-07-29T00:45:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feedfe00640 1 -- 172.21.15.175:0/3895266310 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00dc7b0 con 0x564d2234e980 2024-07-29T00:45:46.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feeecc00640 1 -- 172.21.15.175:0/1581857512 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7feee00cfaa0 con 0x7feef0108140 2024-07-29T00:45:46.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.051+0000 7feeefe00640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feed40a2d70 0x7feed40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.052+0000 7feef8bdf740 1 -- 172.21.15.175:0/3895266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 msgr2=0x564d2234edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.052+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d2234edc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feee00cd080 tx=0x7feee00d8190 comp rx=0 tx=0).stop 2024-07-29T00:45:46.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 -- 172.21.15.175:0/3895266310 shutdown_connections 2024-07-29T00:45:46.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d2235bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d2234edc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3895266310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d2234e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 -- 172.21.15.175:0/3895266310 >> 172.21.15.175:0/3895266310 conn(0x564d22317950 msgr2=0x564d2234b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 -- 172.21.15.175:0/3895266310 shutdown_connections 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feeefe00640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feed40a2d70 0x7feed40a5230 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feee8031550 tx=0x7feee80046c0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.053+0000 7feef8bdf740 1 -- 172.21.15.175:0/3895266310 wait complete. 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 -- start start 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 Processor -- start 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22356730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22356c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d223571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564d2235c4c0 con 0x564d2234fef0 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564d2226fc10 con 0x564d22359720 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.054+0000 7feef8bdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564d2235c640 con 0x564d2234e980 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feef8bdf740 10 client.0 _ll_register_callbacks cb 0x564d22238a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22356730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeefe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d223571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feee6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22356c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22356730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53254/0 (socket says 172.21.15.175:53254) 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 -- 172.21.15.175:0/3536808854 learned_addr learned my addr 172.21.15.175:0/3536808854 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 -- 172.21.15.175:0/3536808854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 msgr2=0x564d223571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d223571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 -- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 msgr2=0x564d22356c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22356c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.055+0000 7feeef400640 1 -- 172.21.15.175:0/3536808854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d222731e0 con 0x564d2234e980 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feeef400640 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22356730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feee0098420 tx=0x7feee00ddc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53758274c1baf107 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:46.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feedcc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feedcc00640 1 -- 172.21.15.175:0/3536808854 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00d6070 con 0x564d2234e980 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feedcc00640 1 -- 172.21.15.175:0/3536808854 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7feee009a2c0 con 0x564d2234e980 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee00d8dd0 con 0x564d2234e980 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feedcc00640 1 -- 172.21.15.175:0/3536808854 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feee0010e60 con 0x564d2234e980 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feef8bdf740 10 client.27190 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.056+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564d2235c4c0 con 0x564d2234e980 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feedcc00640 1 -- 172.21.15.175:0/3536808854 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00d8dd0 con 0x564d2234e980 2024-07-29T00:45:46.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feedcc00640 1 -- 172.21.15.175:0/3536808854 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7feee00cd3c0 con 0x564d2234e980 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feef8bdf740 10 client.27190 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feef8bdf740 10 client.27190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feef8bdf740 20 client.27190 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d22369450 0x564d2236b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feef8bdf740 4 client.27190 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d222c2020 con 0x564d22369450 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.057+0000 7feee6a00640 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d22369450 0x564d2236b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.058+0000 7feee6a00640 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d22369450 0x564d2236b840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd7459ce4d9970b7 server_cookie=d74e28d93b8288b6 in_seq=0 out_seq=0 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.058+0000 7feedcc00640 10 client.27190 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.058+0000 7feedcc00640 1 -- 172.21.15.175:0/3536808854 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564d222c2020 con 0x564d22369450 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.058+0000 7feedcc00640 10 client.27190 handle_command_reply: tid=0 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 client.27190 shutdown 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 msgr2=0x564d22356730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22356730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feee0098420 tx=0x7feee00ddc80 comp rx=0 tx=0).stop 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 shutdown_connections 2024-07-29T00:45:46.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d22369450 0x564d2236b840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d223571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22356c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 --2- 172.21.15.175:0/3536808854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22356730 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feedcc00640 0 client.27190 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 >> 172.21.15.175:0/3536808854 conn(0x564d22317950 msgr2=0x564d22350d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 shutdown_connections 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 1 -- 172.21.15.175:0/3536808854 wait complete. 2024-07-29T00:45:46.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.059+0000 7feef8bdf740 20 client.27190 trim_cache size 0 max 16384 2024-07-29T00:45:46.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 Processor -- start 2024-07-29T00:45:46.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 -- start start 2024-07-29T00:45:46.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22370e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d223713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d223566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b3d80 con 0x564d22359720 2024-07-29T00:45:46.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b58c0 con 0x564d2234fef0 2024-07-29T00:45:46.074 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feef8bdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b4450 con 0x564d2234e980 2024-07-29T00:45:46.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feeefe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d223566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feeefe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d223566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37800/0 (socket says 172.21.15.175:37800) 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feeefe00640 1 -- 172.21.15.175:0/1881634511 learned_addr learned my addr 172.21.15.175:0/1881634511 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.074+0000 7feeef400640 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22370e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feee6a00640 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d223713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feeef400640 1 -- 172.21.15.175:0/1881634511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 msgr2=0x564d223713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feeef400640 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d223713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feeef400640 1 -- 172.21.15.175:0/1881634511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 msgr2=0x564d223566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feeef400640 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d223566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feeef400640 1 -- 172.21.15.175:0/1881634511 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d22354c80 con 0x564d2234e980 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feeef400640 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22370e70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7feee00a4650 tx=0x7feee0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb95cdef01b23567 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feedf400640 1 -- 172.21.15.175:0/1881634511 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00d6070 con 0x564d2234e980 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feedf400640 1 -- 172.21.15.175:0/1881634511 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00d3790 con 0x564d2234e980 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.075+0000 7feedf400640 1 -- 172.21.15.175:0/1881634511 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee009bfc0 con 0x564d2234e980 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 -- 172.21.15.175:0/1881634511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 msgr2=0x564d22370e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22370e70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7feee00a4650 tx=0x7feee0000f40 comp rx=0 tx=0).stop 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 -- 172.21.15.175:0/1881634511 shutdown_connections 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d22359720 0x564d223566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d2234fef0 0x564d223713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 --2- 172.21.15.175:0/1881634511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22370e70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 -- 172.21.15.175:0/1881634511 >> 172.21.15.175:0/1881634511 conn(0x564d22317950 msgr2=0x564d22355820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 -- 172.21.15.175:0/1881634511 shutdown_connections 2024-07-29T00:45:46.078 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.076+0000 7feef8bdf740 1 -- 172.21.15.175:0/1881634511 wait complete. 2024-07-29T00:45:46.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.077+0000 7feef8bdf740 1 -- start start 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.077+0000 7feef8bdf740 1 Processor -- start 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22361b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22362060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d22352f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b51f0 con 0x564d2234fef0 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b4b20 con 0x564d22359720 2024-07-29T00:45:46.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b1d80 con 0x564d2234e980 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feef8bdf740 10 client.0 _ll_register_callbacks cb 0x7feed8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22362060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22362060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37802/0 (socket says 172.21.15.175:37802) 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 -- 172.21.15.175:0/2532599326 learned_addr learned my addr 172.21.15.175:0/2532599326 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feeef400640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22361b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feeefe00640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d22352f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 -- 172.21.15.175:0/2532599326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 msgr2=0x564d22361b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22361b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 -- 172.21.15.175:0/2532599326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 msgr2=0x564d22352f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d22352f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feee6a00640 1 -- 172.21.15.175:0/2532599326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d22353780 con 0x564d2234fef0 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.078+0000 7feeef400640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22361b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feee6a00640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22362060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feed8009dd0 tx=0x7feed80109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac72c4c313df6772 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feedcc00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feedcc00640 1 -- 172.21.15.175:0/2532599326 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8005220 con 0x564d2234fef0 2024-07-29T00:45:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feedcc00640 1 -- 172.21.15.175:0/2532599326 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7feed80053c0 con 0x564d2234fef0 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d222bc530 con 0x564d2234fef0 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feedcc00640 1 -- 172.21.15.175:0/2532599326 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feed80055c0 con 0x564d2234fef0 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feef8bdf740 10 client.17100 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.079+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564d223d39d0 con 0x564d2234fef0 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feedcc00640 1 -- 172.21.15.175:0/2532599326 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564d222bc530 con 0x564d2234fef0 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feedcc00640 1 -- 172.21.15.175:0/2532599326 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7feed8025070 con 0x564d2234fef0 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feef8bdf740 10 client.17100 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:46.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feef8bdf740 10 client.17100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feef8bdf740 20 client.17100 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feef8bdf740 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d223d4180 0x564d223d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feef8bdf740 4 client.17100 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d22239010 con 0x564d223d4180 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.080+0000 7feeef400640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d223d4180 0x564d223d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feeef400640 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d223d4180 0x564d223d65a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8eb65aa8111c8a7e server_cookie=4e7335bca1ce550e in_seq=0 out_seq=0 2024-07-29T00:45:46.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feedcc00640 10 client.17100 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feedcc00640 1 -- 172.21.15.175:0/2532599326 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d22239010 con 0x564d223d4180 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feedcc00640 10 client.17100 handle_command_reply: tid=0 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feef8bdf740 1 client.17100 shutdown 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 msgr2=0x564d22362060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feef8bdf740 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22362060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feed8009dd0 tx=0x7feed80109b0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 shutdown_connections 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feef8bdf740 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x564d223d4180 0x564d223d65a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.081+0000 7feef8bdf740 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d22359720 0x564d22352f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feef8bdf740 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d2234fef0 0x564d22362060 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feef8bdf740 1 --2- 172.21.15.175:0/2532599326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d2234e980 0x564d22361b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 >> 172.21.15.175:0/2532599326 conn(0x564d22317950 msgr2=0x564d22349950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feedcc00640 0 client.17100 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 shutdown_connections 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feef8bdf740 1 -- 172.21.15.175:0/2532599326 wait complete. 2024-07-29T00:45:46.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.082+0000 7feef8bdf740 20 client.17100 trim_cache size 0 max 16384 2024-07-29T00:45:46.089 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:45:46.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feed40a2d70 msgr2=0x7feed40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feed40a2d70 0x7feed40a5230 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feee8031550 tx=0x7feee80046c0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 msgr2=0x7feef019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 0x7feef019e6e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feee000ebc0 tx=0x7feee000c6a0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 shutdown_connections 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef010d220 0x7feef01a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef0108af0 0x7feef019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feed40a2d70 0x7feed40a5230 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 --2- 172.21.15.175:0/1581857512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef0108140 0x7feef019e6e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:46.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 >> 172.21.15.175:0/1581857512 conn(0x7feef0077890 msgr2=0x7feef0079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:46.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 shutdown_connections 2024-07-29T00:45:46.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:46.083+0000 7feef6000640 1 -- 172.21.15.175:0/1581857512 wait complete. 2024-07-29T00:45:46.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:45 smithi181 ceph-mon[36706]: pgmap v286: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:46.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:45 smithi175 ceph-mon[32138]: pgmap v286: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:46.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:45 smithi204 ceph-mon[37368]: pgmap v286: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:46.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:45:47.440 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:48.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 -- 172.21.15.175:0/2660849774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081038a0 msgr2=0x7f5808103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 --2- 172.21.15.175:0/2660849774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081038a0 0x7f5808103d00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009860 tx=0x7f57f402a340 comp rx=0 tx=0).stop 2024-07-29T00:45:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 -- 172.21.15.175:0/2660849774 shutdown_connections 2024-07-29T00:45:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 --2- 172.21.15.175:0/2660849774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f5808110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 --2- 172.21.15.175:0/2660849774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081038a0 0x7f5808103d00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 --2- 172.21.15.175:0/2660849774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081098a0 0x7f5808109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 -- 172.21.15.175:0/2660849774 >> 172.21.15.175:0/2660849774 conn(0x7f58080ff560 msgr2=0x7f5808101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:48.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 -- 172.21.15.175:0/2660849774 shutdown_connections 2024-07-29T00:45:48.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.085+0000 7f580e400640 1 -- 172.21.15.175:0/2660849774 wait complete. 2024-07-29T00:45:48.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.087+0000 7f580e400640 1 Processor -- start 2024-07-29T00:45:48.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.087+0000 7f580e400640 1 -- start start 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 0x7f580819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f580819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081098a0 0x7f580819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5808113ca0 con 0x7f5808104240 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5808113b20 con 0x7f58081098a0 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5808113e20 con 0x7f58081038a0 2024-07-29T00:45:48.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f580819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 0x7f580819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 0x7f580819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:53276/0 (socket says 172.21.15.175:53276) 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580d000640 1 -- 172.21.15.175:0/1087945482 learned_addr learned my addr 172.21.15.175:0/1087945482 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f580819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37830/0 (socket says 172.21.15.175:37830) 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.088+0000 7f580da00640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081098a0 0x7f580819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5807e00640 1 -- 172.21.15.175:0/1087945482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 msgr2=0x7f580819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5807e00640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 0x7f580819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5807e00640 1 -- 172.21.15.175:0/1087945482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081098a0 msgr2=0x7f580819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5807e00640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081098a0 0x7f580819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5807e00640 1 -- 172.21.15.175:0/1087945482 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580819f170 con 0x7f5808104240 2024-07-29T00:45:48.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f580d000640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 0x7f580819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:48.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5807e00640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f580819a700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f57f402a860 tx=0x7f57f4033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:48.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5805600640 1 -- 172.21.15.175:0/1087945482 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4037040 con 0x7f5808104240 2024-07-29T00:45:48.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5805600640 1 -- 172.21.15.175:0/1087945482 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4004050 con 0x7f5808104240 2024-07-29T00:45:48.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580819f400 con 0x7f5808104240 2024-07-29T00:45:48.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.089+0000 7f5805600640 1 -- 172.21.15.175:0/1087945482 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003ab0 con 0x7f5808104240 2024-07-29T00:45:48.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.090+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58081a6d40 con 0x7f5808104240 2024-07-29T00:45:48.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.093+0000 7f5805600640 1 -- 172.21.15.175:0/1087945482 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f57f4042020 con 0x7f5808104240 2024-07-29T00:45:48.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.093+0000 7f581263c740 1 Processor -- start 2024-07-29T00:45:48.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.093+0000 7f581263c740 1 -- start start 2024-07-29T00:45:48.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f581263c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f581263c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af3efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f581263c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af4bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f581263c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56044aea4ab0 con 0x56044af400e0 2024-07-29T00:45:48.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f581263c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56044ae3ca80 con 0x56044af3eb70 2024-07-29T00:45:48.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f581263c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56044aea0a70 con 0x56044af49910 2024-07-29T00:45:48.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f580da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f580da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af3e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37832/0 (socket says 172.21.15.175:37832) 2024-07-29T00:45:48.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f580da00640 1 -- 172.21.15.175:0/1870084098 learned_addr learned my addr 172.21.15.175:0/1870084098 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f5807e00640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af4bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.094+0000 7f580d000640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af3efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580da00640 1 -- 172.21.15.175:0/1870084098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 msgr2=0x56044af4bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580da00640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af4bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580da00640 1 -- 172.21.15.175:0/1870084098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 msgr2=0x56044af3efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580da00640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af3efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580da00640 1 -- 172.21.15.175:0/1870084098 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044ac163c0 con 0x56044af400e0 2024-07-29T00:45:48.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f5807e00640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af4bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:48.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f5805600640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f57ec0a2d70 0x7f57ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580d000640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af3efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:48.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f5805600640 1 -- 172.21.15.175:0/1087945482 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f57f40eea10 con 0x7f5808104240 2024-07-29T00:45:48.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580da00640 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af3e630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f580000e450 tx=0x7f58000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2e1df0d2a861de server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:48.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f580d000640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f57ec0a2d70 0x7f57ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f57fe000640 1 -- 172.21.15.175:0/1870084098 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580000d910 con 0x56044af400e0 2024-07-29T00:45:48.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f57fe000640 1 -- 172.21.15.175:0/1870084098 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580001c070 con 0x56044af400e0 2024-07-29T00:45:48.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.095+0000 7f57fe000640 1 -- 172.21.15.175:0/1870084098 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800016380 con 0x56044af400e0 2024-07-29T00:45:48.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 -- 172.21.15.175:0/1870084098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 msgr2=0x56044af3e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f580d000640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f57ec0a2d70 0x7f57ec0a5230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x56044af3c060 tx=0x7f57f800b480 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:48.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af3e630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f580000e450 tx=0x7f58000095c0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 -- 172.21.15.175:0/1870084098 shutdown_connections 2024-07-29T00:45:48.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af4bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af3efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 --2- 172.21.15.175:0/1870084098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af3e630 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.096+0000 7f581263c740 1 -- 172.21.15.175:0/1870084098 >> 172.21.15.175:0/1870084098 conn(0x56044af07950 msgr2=0x56044af3b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:48.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.097+0000 7f581263c740 1 -- 172.21.15.175:0/1870084098 shutdown_connections 2024-07-29T00:45:48.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.097+0000 7f581263c740 1 -- 172.21.15.175:0/1870084098 wait complete. 2024-07-29T00:45:48.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.097+0000 7f581263c740 1 -- start start 2024-07-29T00:45:48.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.097+0000 7f581263c740 1 Processor -- start 2024-07-29T00:45:48.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af46de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af47320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56044af4c6b0 con 0x56044af400e0 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56044ae5fc10 con 0x56044af3eb70 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56044af4c830 con 0x56044af49910 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f581263c740 10 client.0 _ll_register_callbacks cb 0x56044ae28a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af46de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af46de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37842/0 (socket says 172.21.15.175:37842) 2024-07-29T00:45:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f5807e00640 1 -- 172.21.15.175:0/1912372146 learned_addr learned my addr 172.21.15.175:0/1912372146 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f580d000640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.098+0000 7f580da00640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af47320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f5807e00640 1 -- 172.21.15.175:0/1912372146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 msgr2=0x56044af47320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f5807e00640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af47320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f5807e00640 1 -- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 msgr2=0x56044af468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f5807e00640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f5807e00640 1 -- 172.21.15.175:0/1912372146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044ae631e0 con 0x56044af400e0 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f5807e00640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af46de0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f57f402a930 tx=0x7f57f4009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fa7aa7692defb2 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f57f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f57f2a00640 1 -- 172.21.15.175:0/1912372146 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f40bad60 con 0x56044af400e0 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f57f2a00640 1 -- 172.21.15.175:0/1912372146 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f57f40ec330 con 0x56044af400e0 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.099+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580001c070 con 0x56044af400e0 2024-07-29T00:45:48.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f57f2a00640 1 -- 172.21.15.175:0/1912372146 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f57f40ba8f0 con 0x56044af400e0 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f581263c740 10 client.17112 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56044af58440 con 0x56044af400e0 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f57f2a00640 1 -- 172.21.15.175:0/1912372146 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580001c070 con 0x56044af400e0 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f57f2a00640 1 -- 172.21.15.175:0/1912372146 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f57f403a070 con 0x56044af400e0 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f581263c740 10 client.17112 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f581263c740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:48.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f581263c740 20 client.17112 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.100+0000 7f581263c740 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044af59790 0x56044af5bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.101+0000 7f581263c740 4 client.17112 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.101+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56044aeb2020 con 0x56044af59790 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.101+0000 7f580d000640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044af59790 0x56044af5bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.101+0000 7f580d000640 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044af59790 0x56044af5bbc0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df7c18e11ab5617 server_cookie=d6ecec91dd8e8527 in_seq=0 out_seq=0 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.101+0000 7f57f2a00640 10 client.17112 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:48.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.102+0000 7f57f2a00640 1 -- 172.21.15.175:0/1912372146 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56044aeb2020 con 0x56044af59790 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.102+0000 7f57f2a00640 10 client.17112 handle_command_reply: tid=0 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.102+0000 7f581263c740 1 client.17112 shutdown 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 msgr2=0x56044af46de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af46de0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f57f402a930 tx=0x7f57f4009490 comp rx=0 tx=0).stop 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 shutdown_connections 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044af59790 0x56044af5bbc0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af47320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af46de0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.111 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 --2- 172.21.15.175:0/1912372146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f57f2a00640 0 client.17112 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:48.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 >> 172.21.15.175:0/1912372146 conn(0x56044af07950 msgr2=0x56044af40ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:48.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 shutdown_connections 2024-07-29T00:45:48.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 1 -- 172.21.15.175:0/1912372146 wait complete. 2024-07-29T00:45:48.112 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.103+0000 7f581263c740 20 client.17112 trim_cache size 0 max 16384 2024-07-29T00:45:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.117+0000 7f581263c740 1 Processor -- start 2024-07-29T00:45:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.117+0000 7f581263c740 1 -- start start 2024-07-29T00:45:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f581263c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f581263c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f581263c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af61a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f581263c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58081af770 con 0x56044af400e0 2024-07-29T00:45:48.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f581263c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f58081b12b0 con 0x56044af3eb70 2024-07-29T00:45:48.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f581263c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58081afe40 con 0x56044af49910 2024-07-29T00:45:48.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f580d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f580d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af48810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35922/0 (socket says 172.21.15.175:35922) 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f580d000640 1 -- 172.21.15.175:0/4201900971 learned_addr learned my addr 172.21.15.175:0/4201900971 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37856/0 (socket says 172.21.15.175:37856) 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.118+0000 7f580da00640 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af61a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f580d000640 1 -- 172.21.15.175:0/4201900971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 msgr2=0x56044af61a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f580d000640 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af61a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f580d000640 1 -- 172.21.15.175:0/4201900971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 msgr2=0x56044af614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f580d000640 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f580d000640 1 -- 172.21.15.175:0/4201900971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044af5c240 con 0x56044af3eb70 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f580d000640 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af48810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c010 tx=0x7f57f8016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93db5bc9fd634999 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f57fd600640 1 -- 172.21.15.175:0/4201900971 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8004400 con 0x56044af3eb70 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f57fd600640 1 -- 172.21.15.175:0/4201900971 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8005500 con 0x56044af3eb70 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.119+0000 7f57fd600640 1 -- 172.21.15.175:0/4201900971 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80035f0 con 0x56044af3eb70 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 -- 172.21.15.175:0/4201900971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 msgr2=0x56044af48810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af48810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c010 tx=0x7f57f8016000 comp rx=0 tx=0).stop 2024-07-29T00:45:48.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 -- 172.21.15.175:0/4201900971 shutdown_connections 2024-07-29T00:45:48.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af49910 0x56044af61a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 --2- 172.21.15.175:0/4201900971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af3eb70 0x56044af48810 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 -- 172.21.15.175:0/4201900971 >> 172.21.15.175:0/4201900971 conn(0x56044af07950 msgr2=0x56044af48100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:48.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 -- 172.21.15.175:0/4201900971 shutdown_connections 2024-07-29T00:45:48.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.120+0000 7f581263c740 1 -- 172.21.15.175:0/4201900971 wait complete. 2024-07-29T00:45:48.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 -- start start 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 Processor -- start 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af3eb70 0x56044af51ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af52000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af49910 0x56044af4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58081b0be0 con 0x56044af400e0 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f58081b0510 con 0x56044af49910 2024-07-29T00:45:48.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ad770 con 0x56044af3eb70 2024-07-29T00:45:48.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.121+0000 7f581263c740 10 client.0 _ll_register_callbacks cb 0x7f57f40f21b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:48.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af52000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f580d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af3eb70 0x56044af51ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af52000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37858/0 (socket says 172.21.15.175:37858) 2024-07-29T00:45:48.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 -- 172.21.15.175:0/4250375651 learned_addr learned my addr 172.21.15.175:0/4250375651 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:48.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f580da00640 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af49910 0x56044af4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 -- 172.21.15.175:0/4250375651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af3eb70 msgr2=0x56044af51ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af3eb70 0x56044af51ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 -- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af49910 msgr2=0x56044af4ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:48.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af49910 0x56044af4ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 -- 172.21.15.175:0/4250375651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044af525f0 con 0x56044af400e0 2024-07-29T00:45:48.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f5807e00640 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af52000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f57f4032860 tx=0x7f57f40c39a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a2364862ad3fe6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.122+0000 7f57f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f57f2a00640 1 -- 172.21.15.175:0/4250375651 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f40baad0 con 0x56044af400e0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f57f2a00640 1 -- 172.21.15.175:0/4250375651 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f57f40b5b00 con 0x56044af400e0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f8005500 con 0x56044af400e0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f57f2a00640 1 -- 172.21.15.175:0/4250375651 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f57f40bad60 con 0x56044af400e0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f581263c740 10 client.17118 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56044afc49c0 con 0x56044af400e0 2024-07-29T00:45:48.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f57f2a00640 1 -- 172.21.15.175:0/4250375651 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8005500 con 0x56044af400e0 2024-07-29T00:45:48.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f57f2a00640 1 -- 172.21.15.175:0/4250375651 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f57f403a070 con 0x56044af400e0 2024-07-29T00:45:48.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f581263c740 10 client.17118 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:48.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f581263c740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:48.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.123+0000 7f581263c740 20 client.17118 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:48.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.124+0000 7f581263c740 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044afc4fa0 0x56044afc7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:48.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.124+0000 7f581263c740 4 client.17118 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.124+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56044aeac530 con 0x56044afc4fa0 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.124+0000 7f580d000640 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044afc4fa0 0x56044afc7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f580d000640 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044afc4fa0 0x56044afc7390 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bd9e9d5efcd20a server_cookie=40211a647a141548 in_seq=0 out_seq=0 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f57f2a00640 10 client.17118 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f57f2a00640 1 -- 172.21.15.175:0/4250375651 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56044aeac530 con 0x56044afc4fa0 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f57f2a00640 10 client.17118 handle_command_reply: tid=0 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 client.17118 shutdown 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 msgr2=0x56044af52000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af52000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f57f4032860 tx=0x7f57f40c39a0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 shutdown_connections 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56044afc4fa0 0x56044afc7390 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56044af49910 0x56044af4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56044af400e0 0x56044af52000 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 --2- 172.21.15.175:0/4250375651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56044af3eb70 0x56044af51ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 >> 172.21.15.175:0/4250375651 conn(0x56044af07950 msgr2=0x56044af47080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f57f2a00640 0 client.17118 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:48.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 shutdown_connections 2024-07-29T00:45:48.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 1 -- 172.21.15.175:0/4250375651 wait complete. 2024-07-29T00:45:48.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.125+0000 7f581263c740 20 client.17118 trim_cache size 0 max 16384 2024-07-29T00:45:48.134 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:45:48.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f57ec0a2d70 msgr2=0x7f57ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f57ec0a2d70 0x7f57ec0a5230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x56044af3c060 tx=0x7f57f800b480 comp rx=0 tx=0).stop 2024-07-29T00:45:48.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 msgr2=0x7f580819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:48.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f580819a700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f57f402a860 tx=0x7f57f4033d90 comp rx=0 tx=0).stop 2024-07-29T00:45:48.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 shutdown_connections 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58081098a0 0x7f580819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f57ec0a2d70 0x7f57ec0a5230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5808104240 0x7f580819a700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 --2- 172.21.15.175:0/1087945482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58081038a0 0x7f580819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.126+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 >> 172.21.15.175:0/1087945482 conn(0x7f58080ff560 msgr2=0x7f58080ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.127+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 shutdown_connections 2024-07-29T00:45:48.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:48.127+0000 7f580e400640 1 -- 172.21.15.175:0/1087945482 wait complete. 2024-07-29T00:45:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:47 smithi181 ceph-mon[36706]: pgmap v287: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:47 smithi175 ceph-mon[32138]: pgmap v287: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:47 smithi204 ceph-mon[37368]: pgmap v287: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:50.040 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:45:50.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:45:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:49 smithi181 ceph-mon[36706]: pgmap v288: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:50.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:49 smithi175 ceph-mon[32138]: pgmap v288: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:50.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:49 smithi204 ceph-mon[37368]: pgmap v288: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:50.503 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:51.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 -- 172.21.15.175:0/2141574704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074200 msgr2=0x7f67240745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 --2- 172.21.15.175:0/2141574704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074200 0x7f67240745e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6718009900 tx=0x7f671802a480 comp rx=0 tx=0).stop 2024-07-29T00:45:51.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 -- 172.21.15.175:0/2141574704 shutdown_connections 2024-07-29T00:45:51.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 --2- 172.21.15.175:0/2141574704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672410e130 0x7f6724110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 --2- 172.21.15.175:0/2141574704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6724074b20 0x7f672410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 --2- 172.21.15.175:0/2141574704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074200 0x7f67240745e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 -- 172.21.15.175:0/2141574704 >> 172.21.15.175:0/2141574704 conn(0x7f67240fd5b0 msgr2=0x7f67240ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:51.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 -- 172.21.15.175:0/2141574704 shutdown_connections 2024-07-29T00:45:51.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.179+0000 7f672b800640 1 -- 172.21.15.175:0/2141574704 wait complete. 2024-07-29T00:45:51.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.180+0000 7f672b800640 1 Processor -- start 2024-07-29T00:45:51.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.180+0000 7f672b800640 1 -- start start 2024-07-29T00:45:51.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f672b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 0x7f672419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f672b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 0x7f672419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f672b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f672410e130 0x7f672419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f672b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6724113e60 con 0x7f6724074b20 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f672b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6724113ce0 con 0x7f672410e130 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f672b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6724113fe0 con 0x7f6724074200 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f6728e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 0x7f672419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f6723e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 0x7f672419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f6728e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 0x7f672419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52686/0 (socket says 172.21.15.175:52686) 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f6728e00640 1 -- 172.21.15.175:0/3682976152 learned_addr learned my addr 172.21.15.175:0/3682976152 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f6723e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 0x7f672419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51692/0 (socket says 172.21.15.175:51692) 2024-07-29T00:45:51.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.181+0000 7f6729800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f672410e130 0x7f672419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6728e00640 1 -- 172.21.15.175:0/3682976152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f672410e130 msgr2=0x7f672419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6728e00640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f672410e130 0x7f672419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6728e00640 1 -- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 msgr2=0x7f672419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6728e00640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 0x7f672419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6728e00640 1 -- 172.21.15.175:0/3682976152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672419f3d0 con 0x7f6724074200 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6723e00640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 0x7f672419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6728e00640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 0x7f672419a420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6718006010 tx=0x7f67180042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6721600640 1 -- 172.21.15.175:0/3682976152 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671803f030 con 0x7f6724074200 2024-07-29T00:45:51.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6721600640 1 -- 172.21.15.175:0/3682976152 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6718003ce0 con 0x7f6724074200 2024-07-29T00:45:51.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672419f660 con 0x7f6724074200 2024-07-29T00:45:51.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.182+0000 7f6721600640 1 -- 172.21.15.175:0/3682976152 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718035070 con 0x7f6724074200 2024-07-29T00:45:51.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.183+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67241a6fa0 con 0x7f6724074200 2024-07-29T00:45:51.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f672e3c7740 1 Processor -- start 2024-07-29T00:45:51.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f6721600640 1 -- 172.21.15.175:0/3682976152 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6718042030 con 0x7f6724074200 2024-07-29T00:45:51.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f672e3c7740 1 -- start start 2024-07-29T00:45:51.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 0x56308b0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0d0460 0x56308b0da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.187+0000 7f672e3c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56308b0358a0 con 0x56308b0dad20 2024-07-29T00:45:51.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f672e3c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56308afcd710 con 0x56308b0cfb40 2024-07-29T00:45:51.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f672e3c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56308b031860 con 0x56308b0d0460 2024-07-29T00:45:51.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6729800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 0x56308b0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6728e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0d0460 0x56308b0da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6729800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 0x56308b0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:43856/0 (socket says 172.21.15.175:43856) 2024-07-29T00:45:51.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6729800640 1 -- 172.21.15.175:0/1142586959 learned_addr learned my addr 172.21.15.175:0/1142586959 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:51.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6723e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6728e00640 1 -- 172.21.15.175:0/1142586959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 msgr2=0x56308b0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6728e00640 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 0x56308b0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6728e00640 1 -- 172.21.15.175:0/1142586959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 msgr2=0x56308b0dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6728e00640 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6728e00640 1 -- 172.21.15.175:0/1142586959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308ada76a0 con 0x56308b0d0460 2024-07-29T00:45:51.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.188+0000 7f6729800640 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 0x56308b0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6728e00640 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0d0460 0x56308b0da7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67180ed050 tx=0x7f67180f5bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d75052525777646 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6721600640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f670c0a2d70 0x7f670c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6721600640 1 -- 172.21.15.175:0/3682976152 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f67180ef950 con 0x7f6724074200 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6729800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f670c0a2d70 0x7f670c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6716000640 1 -- 172.21.15.175:0/1142586959 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180fe070 con 0x56308b0d0460 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6716000640 1 -- 172.21.15.175:0/1142586959 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67180bbf90 con 0x56308b0d0460 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.189+0000 7f6716000640 1 -- 172.21.15.175:0/1142586959 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180b8230 con 0x56308b0d0460 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 -- 172.21.15.175:0/1142586959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0d0460 msgr2=0x56308b0da7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0d0460 0x56308b0da7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67180ed050 tx=0x7f67180f5bc0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f6729800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f670c0a2d70 0x7f670c0a5230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f671c009fd0 tx=0x7f671c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 -- 172.21.15.175:0/1142586959 shutdown_connections 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0d0460 0x56308b0da7e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1142586959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0cfb40 0x56308b0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 -- 172.21.15.175:0/1142586959 >> 172.21.15.175:0/1142586959 conn(0x56308b0cc0d0 msgr2=0x56308b0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 -- 172.21.15.175:0/1142586959 shutdown_connections 2024-07-29T00:45:51.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.190+0000 7f672e3c7740 1 -- 172.21.15.175:0/1142586959 wait complete. 2024-07-29T00:45:51.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.191+0000 7f672e3c7740 1 -- start start 2024-07-29T00:45:51.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.191+0000 7f672e3c7740 1 Processor -- start 2024-07-29T00:45:51.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0d7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0d8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56308b0dd6b0 con 0x56308b0dad20 2024-07-29T00:45:51.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56308aff08c0 con 0x56308b0d0460 2024-07-29T00:45:51.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56308b0dd830 con 0x56308b0cfb40 2024-07-29T00:45:51.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f672e3c7740 10 client.0 _ll_register_callbacks cb 0x56308afdca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0d7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6729800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0d7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52704/0 (socket says 172.21.15.175:52704) 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 -- 172.21.15.175:0/2199500554 learned_addr learned my addr 172.21.15.175:0/2199500554 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6723e00640 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 -- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 msgr2=0x56308b0d8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0d8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 -- 172.21.15.175:0/2199500554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 msgr2=0x56308b0d8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0d8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.192+0000 7f6728e00640 1 -- 172.21.15.175:0/2199500554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308afd3bc0 con 0x56308b0cfb40 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f6728e00640 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0d7bf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67180f51b0 tx=0x7f6718005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=793ca24a9294cda3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f670aa00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f670aa00640 1 -- 172.21.15.175:0/2199500554 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718049070 con 0x56308b0cfb40 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f670aa00640 1 -- 172.21.15.175:0/2199500554 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f67180ba290 con 0x56308b0cfb40 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67180bbf90 con 0x56308b0cfb40 2024-07-29T00:45:51.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f670aa00640 1 -- 172.21.15.175:0/2199500554 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f67180fe030 con 0x56308b0cfb40 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f672e3c7740 10 client.27226 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.193+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56308b0dd6b0 con 0x56308b0cfb40 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f670aa00640 1 -- 172.21.15.175:0/2199500554 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67180bbf90 con 0x56308b0cfb40 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f670aa00640 1 -- 172.21.15.175:0/2199500554 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f67180b8600 con 0x56308b0cfb40 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f672e3c7740 10 client.27226 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f672e3c7740 10 client.27226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f672e3c7740 20 client.27226 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f672e3c7740 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b0ea880 0x56308b0ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f672e3c7740 4 client.27226 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56308aff5c90 con 0x56308b0ea880 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.194+0000 7f6723e00640 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b0ea880 0x56308b0ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.195+0000 7f6723e00640 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b0ea880 0x56308b0ecc70 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f219d86bb53b5c0b server_cookie=692ac22cc7ba9a7b in_seq=0 out_seq=0 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.195+0000 7f670aa00640 10 client.27226 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f670aa00640 1 -- 172.21.15.175:0/2199500554 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56308aff5c90 con 0x56308b0ea880 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f670aa00640 10 client.27226 handle_command_reply: tid=0 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 client.27226 shutdown 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 msgr2=0x56308b0d7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.201 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0d7bf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67180f51b0 tx=0x7f6718005e50 comp rx=0 tx=0).stop 2024-07-29T00:45:51.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 shutdown_connections 2024-07-29T00:45:51.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b0ea880 0x56308b0ecc70 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0dad20 0x56308b0d8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 --2- 172.21.15.175:0/2199500554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0d7bf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f670aa00640 0 client.27226 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:51.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 >> 172.21.15.175:0/2199500554 conn(0x56308b0cc0d0 msgr2=0x56308b0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:51.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 shutdown_connections 2024-07-29T00:45:51.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 1 -- 172.21.15.175:0/2199500554 wait complete. 2024-07-29T00:45:51.203 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.196+0000 7f672e3c7740 20 client.27226 trim_cache size 0 max 16384 2024-07-29T00:45:51.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.215+0000 7f672e3c7740 1 Processor -- start 2024-07-29T00:45:51.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.215+0000 7f672e3c7740 1 -- start start 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 0x56308b0d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0dad20 0x56308b0f2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f672e3c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67241af9a0 con 0x56308b0cfb40 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f672e3c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67241b14e0 con 0x56308b0d0460 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f672e3c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67241b0070 con 0x56308b0dad20 2024-07-29T00:45:51.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 0x56308b0d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 0x56308b0d84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51710/0 (socket says 172.21.15.175:51710) 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 -- 172.21.15.175:0/418436268 learned_addr learned my addr 172.21.15.175:0/418436268 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6729800640 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0dad20 0x56308b0f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6723e00640 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 -- 172.21.15.175:0/418436268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0dad20 msgr2=0x56308b0f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0dad20 0x56308b0f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 -- 172.21.15.175:0/418436268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 msgr2=0x56308b0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.216+0000 7f6728e00640 1 -- 172.21.15.175:0/418436268 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308b0cbb70 con 0x56308b0cfb40 2024-07-29T00:45:51.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.217+0000 7f6728e00640 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 0x56308b0d84c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67180b8960 tx=0x7f671803b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee1fb2211105a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:51.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.217+0000 7f6715600640 1 -- 172.21.15.175:0/418436268 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718049070 con 0x56308b0cfb40 2024-07-29T00:45:51.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.217+0000 7f6715600640 1 -- 172.21.15.175:0/418436268 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67180c0290 con 0x56308b0cfb40 2024-07-29T00:45:51.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.217+0000 7f6715600640 1 -- 172.21.15.175:0/418436268 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180044e0 con 0x56308b0cfb40 2024-07-29T00:45:51.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 -- 172.21.15.175:0/418436268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 msgr2=0x56308b0d84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 0x56308b0d84c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67180b8960 tx=0x7f671803b5d0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 -- 172.21.15.175:0/418436268 shutdown_connections 2024-07-29T00:45:51.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0dad20 0x56308b0f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0d0460 0x56308b0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 --2- 172.21.15.175:0/418436268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0cfb40 0x56308b0d84c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 -- 172.21.15.175:0/418436268 >> 172.21.15.175:0/418436268 conn(0x56308b0cc0d0 msgr2=0x56308b0caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 -- 172.21.15.175:0/418436268 shutdown_connections 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 -- 172.21.15.175:0/418436268 wait complete. 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.218+0000 7f672e3c7740 1 -- start start 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 Processor -- start 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 0x56308b0dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0dad20 0x56308b0df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67241b0e10 con 0x56308b0d0460 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67241b0740 con 0x56308b0dad20 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67241ad9a0 con 0x56308b0cfb40 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f672e3c7740 10 client.0 _ll_register_callbacks cb 0x7f67180f5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f6723e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 0x56308b0dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f6723e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 0x56308b0dee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51718/0 (socket says 172.21.15.175:51718) 2024-07-29T00:45:51.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f6723e00640 1 -- 172.21.15.175:0/1216754229 learned_addr learned my addr 172.21.15.175:0/1216754229 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f6728e00640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.219+0000 7f6729800640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0dad20 0x56308b0df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f6723e00640 1 -- 172.21.15.175:0/1216754229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 msgr2=0x56308b0de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f6723e00640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0de8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f6723e00640 1 -- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0dad20 msgr2=0x56308b0df370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f6723e00640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0dad20 0x56308b0df370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f6723e00640 1 -- 172.21.15.175:0/1216754229 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308b1546b0 con 0x56308b0d0460 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f6723e00640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 0x56308b0dee30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x56308b0cd450 tx=0x7f6710011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a29f0a0e6444ace server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f670aa00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.220+0000 7f670aa00640 1 -- 172.21.15.175:0/1216754229 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710010cf0 con 0x56308b0d0460 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f670aa00640 1 -- 172.21.15.175:0/1216754229 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f6710005020 con 0x56308b0d0460 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67180c0290 con 0x56308b0d0460 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f670aa00640 1 -- 172.21.15.175:0/1216754229 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6710010e90 con 0x56308b0d0460 2024-07-29T00:45:51.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 10 client.17136 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56308b155a60 con 0x56308b0d0460 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f670aa00640 1 -- 172.21.15.175:0/1216754229 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67180c0290 con 0x56308b0d0460 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f670aa00640 1 -- 172.21.15.175:0/1216754229 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6710025400 con 0x56308b0d0460 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 10 client.17136 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 20 client.17136 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b156260 0x56308b158680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 4 client.17136 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:45:51.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.221+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56308b042e10 con 0x56308b156260 2024-07-29T00:45:51.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.222+0000 7f6728e00640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b156260 0x56308b158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.222+0000 7f6728e00640 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b156260 0x56308b158680 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad70df1de4cf97c5 server_cookie=1ec6289f4c11bdd3 in_seq=0 out_seq=0 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.222+0000 7f670aa00640 10 client.17136 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f670aa00640 1 -- 172.21.15.175:0/1216754229 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56308b042e10 con 0x56308b156260 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f670aa00640 10 client.17136 handle_command_reply: tid=0 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 client.17136 shutdown 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 msgr2=0x56308b0dee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 0x56308b0dee30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x56308b0cd450 tx=0x7f6710011c60 comp rx=0 tx=0).stop 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 shutdown_connections 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x56308b156260 0x56308b158680 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56308b0dad20 0x56308b0df370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56308b0d0460 0x56308b0dee30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 --2- 172.21.15.175:0/1216754229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56308b0cfb40 0x56308b0de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f670aa00640 0 client.17136 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 >> 172.21.15.175:0/1216754229 conn(0x56308b0cc0d0 msgr2=0x56308b0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:51.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 shutdown_connections 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 1 -- 172.21.15.175:0/1216754229 wait complete. 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.224+0000 7f672e3c7740 20 client.17136 trim_cache size 0 max 16384 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "b75c0cec-bfaf-4f5e-9408-7e23c3bb9823", 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:45:51.231 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:45:51.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f670c0a2d70 msgr2=0x7f670c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f670c0a2d70 0x7f670c0a5230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f671c009fd0 tx=0x7f671c009500 comp rx=0 tx=0).stop 2024-07-29T00:45:51.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 msgr2=0x7f672419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:51.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 0x7f672419a420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6718006010 tx=0x7f67180042f0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 shutdown_connections 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f672410e130 0x7f672419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f670c0a2d70 0x7f670c0a5230 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6724074b20 0x7f672419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 --2- 172.21.15.175:0/3682976152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6724074200 0x7f672419a420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.225+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 >> 172.21.15.175:0/3682976152 conn(0x7f67240fd5b0 msgr2=0x7f67240fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.226+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 shutdown_connections 2024-07-29T00:45:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:51.226+0000 7f672b800640 1 -- 172.21.15.175:0/3682976152 wait complete. 2024-07-29T00:45:52.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:45:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: pgmap v289: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:51 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: pgmap v289: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:52.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:51 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: pgmap v289: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:52.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:45:52.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:51 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:52.573 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:45:53.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 -- 172.21.15.175:0/2963921263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac076410 msgr2=0x7fcfac0767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2963921263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac076410 0x7fcfac0767f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009900 tx=0x7fcf9c02a400 comp rx=0 tx=0).stop 2024-07-29T00:45:53.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 -- 172.21.15.175:0/2963921263 shutdown_connections 2024-07-29T00:45:53.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2963921263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac112740 0x7fcfac114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2963921263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfac076d30 0x7fcfac112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2963921263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac076410 0x7fcfac0767f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 -- 172.21.15.175:0/2963921263 >> 172.21.15.175:0/2963921263 conn(0x7fcfac0ff740 msgr2=0x7fcfac101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:53.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.213+0000 7fcfb3200640 1 -- 172.21.15.175:0/2963921263 shutdown_connections 2024-07-29T00:45:53.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.215+0000 7fcfb3200640 1 -- 172.21.15.175:0/2963921263 wait complete. 2024-07-29T00:45:53.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 Processor -- start 2024-07-29T00:45:53.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 -- start start 2024-07-29T00:45:53.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 0x7fcfac1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.216 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfac076d30 0x7fcfac1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac112740 0x7fcfac19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac118320 con 0x7fcfac112740 2024-07-29T00:45:53.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1181a0 con 0x7fcfac076d30 2024-07-29T00:45:53.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.216+0000 7fcfb3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1184a0 con 0x7fcfac076410 2024-07-29T00:45:53.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 0x7fcfac1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfac076d30 0x7fcfac1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 0x7fcfac1a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52734/0 (socket says 172.21.15.175:52734) 2024-07-29T00:45:53.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 -- 172.21.15.175:0/2864252696 learned_addr learned my addr 172.21.15.175:0/2864252696 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:53.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfb1200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac112740 0x7fcfac19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 -- 172.21.15.175:0/2864252696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfac076d30 msgr2=0x7fcfac1a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfac076d30 0x7fcfac1a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 -- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac112740 msgr2=0x7fcfac19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac112740 0x7fcfac19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.217+0000 7fcfabe00640 1 -- 172.21.15.175:0/2864252696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac19ef00 con 0x7fcfac076410 2024-07-29T00:45:53.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.218+0000 7fcfabe00640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 0x7fcfac1a4570 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c004770 tx=0x7fcf9c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:53.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.218+0000 7fcfa9600640 1 -- 172.21.15.175:0/2864252696 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c03f070 con 0x7fcfac076410 2024-07-29T00:45:53.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.218+0000 7fcfa9600640 1 -- 172.21.15.175:0/2864252696 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf9c031ca0 con 0x7fcfac076410 2024-07-29T00:45:53.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.218+0000 7fcfa9600640 1 -- 172.21.15.175:0/2864252696 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c035030 con 0x7fcfac076410 2024-07-29T00:45:53.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.218+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac19f190 con 0x7fcfac076410 2024-07-29T00:45:53.220 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.218+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac1ab390 con 0x7fcfac076410 2024-07-29T00:45:53.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.221+0000 7fcfb5e01740 1 Processor -- start 2024-07-29T00:45:53.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.221+0000 7fcfa9600640 1 -- 172.21.15.175:0/2864252696 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcf9c042070 con 0x7fcfac076410 2024-07-29T00:45:53.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 -- start start 2024-07-29T00:45:53.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f07d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f12b10 0x555c60f14f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555c60e6cad0 con 0x555c60f12b10 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555c60e04af0 con 0x555c60f07930 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555c60e68a90 con 0x555c60f08250 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f07d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f07d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:43906/0 (socket says 172.21.15.175:43906) 2024-07-29T00:45:53.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfb1200640 1 -- 172.21.15.175:0/12346040 learned_addr learned my addr 172.21.15.175:0/12346040 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:53.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.222+0000 7fcfabe00640 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f12b10 0x555c60f14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfb1200640 1 -- 172.21.15.175:0/12346040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 msgr2=0x555c60f125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfb1200640 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfb1200640 1 -- 172.21.15.175:0/12346040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f12b10 msgr2=0x555c60f14f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfb1200640 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f12b10 0x555c60f14f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfb1200640 1 -- 172.21.15.175:0/12346040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c60bde3c0 con 0x555c60f07930 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.223+0000 7fcfb1200640 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f07d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009f30 tx=0x7fcfa40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154fc7a5fc3ac48 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.224+0000 7fcfa1600640 1 -- 172.21.15.175:0/12346040 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004d20 con 0x555c60f07930 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.224+0000 7fcfa1600640 1 -- 172.21.15.175:0/12346040 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4004050 con 0x555c60f07930 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.224+0000 7fcfa1600640 1 -- 172.21.15.175:0/12346040 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4005280 con 0x555c60f07930 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.224+0000 7fcfa9600640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcf900a2d70 0x7fcf900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.224+0000 7fcfa9600640 1 -- 172.21.15.175:0/2864252696 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fcf9c0ef6d0 con 0x7fcfac076410 2024-07-29T00:45:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.224+0000 7fcfabe00640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcf900a2d70 0x7fcf900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfabe00640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcf900a2d70 0x7fcf900a5230 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x555c60f04d20 tx=0x7fcf9c009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 -- 172.21.15.175:0/12346040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 msgr2=0x555c60f07d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f07d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009f30 tx=0x7fcfa40118d0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 -- 172.21.15.175:0/12346040 shutdown_connections 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f12b10 0x555c60f14f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/12346040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f07d10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 -- 172.21.15.175:0/12346040 >> 172.21.15.175:0/12346040 conn(0x555c60f03ec0 msgr2=0x555c60f042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 -- 172.21.15.175:0/12346040 shutdown_connections 2024-07-29T00:45:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.225+0000 7fcfb5e01740 1 -- 172.21.15.175:0/12346040 wait complete. 2024-07-29T00:45:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 -- start start 2024-07-29T00:45:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 Processor -- start 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f07930 0x555c60f0f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f0ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f10470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555c60f156e0 con 0x555c60f08250 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555c60e27bc0 con 0x555c60f12b10 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555c60f15860 con 0x555c60f07930 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.226+0000 7fcfb5e01740 10 client.0 _ll_register_callbacks cb 0x555c60e13d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f0ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f10470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f0ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51742/0 (socket says 172.21.15.175:51742) 2024-07-29T00:45:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f10470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:43920/0 (socket says 172.21.15.175:43920) 2024-07-29T00:45:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 -- 172.21.15.175:0/1667365249 learned_addr learned my addr 172.21.15.175:0/1667365249 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f07930 0x555c60f0f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 -- 172.21.15.175:0/1667365249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f07930 msgr2=0x555c60f0f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f07930 0x555c60f0f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 -- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 msgr2=0x555c60f10470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f10470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.227+0000 7fcfa3e00640 1 -- 172.21.15.175:0/1667365249 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c60f1ebd0 con 0x555c60f08250 2024-07-29T00:45:53.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f0ff30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf94012730 tx=0x7fcf94012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3d39e19251c200 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:53.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcf9a000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcf9a000640 1 -- 172.21.15.175:0/1667365249 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94004050 con 0x555c60f08250 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa4004050 con 0x555c60f08250 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcf9a000640 1 -- 172.21.15.175:0/1667365249 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fcf940041f0 con 0x555c60f08250 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcf9a000640 1 -- 172.21.15.175:0/1667365249 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcf940043f0 con 0x555c60f08250 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcfb5e01740 10 client.17148 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.228+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c60f21610 con 0x555c60f08250 2024-07-29T00:45:53.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcf9a000640 1 -- 172.21.15.175:0/1667365249 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4004050 con 0x555c60f08250 2024-07-29T00:45:53.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcf9a000640 1 -- 172.21.15.175:0/1667365249 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fcf94003b20 con 0x555c60f08250 2024-07-29T00:45:53.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcfb5e01740 10 client.17148 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:53.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcfb5e01740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:53.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcfb5e01740 20 client.17148 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:53.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f22ac0 0x555c60f24eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcfb5e01740 4 client.17148 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:45:53.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.229+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c60e7a040 con 0x555c60f22ac0 2024-07-29T00:45:53.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.230+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f22ac0 0x555c60f24eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.230+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f22ac0 0x555c60f24eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7975b670586412da server_cookie=5ae4b6f1cfbc1bc9 in_seq=0 out_seq=0 2024-07-29T00:45:53.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.230+0000 7fcf9a000640 10 client.17148 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:53.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.231+0000 7fcf9a000640 1 -- 172.21.15.175:0/1667365249 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555c60e7a040 con 0x555c60f22ac0 2024-07-29T00:45:53.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.231+0000 7fcf9a000640 10 client.17148 handle_command_reply: tid=0 2024-07-29T00:45:53.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.231+0000 7fcfb5e01740 1 client.17148 shutdown 2024-07-29T00:45:53.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.231+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 msgr2=0x555c60f0ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.231+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f0ff30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf94012730 tx=0x7fcf94012bf0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 shutdown_connections 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f22ac0 0x555c60f24eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f10470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f0ff30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1667365249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f07930 0x555c60f0f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 >> 172.21.15.175:0/1667365249 conn(0x555c60f03ec0 msgr2=0x555c60f09ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcf9a000640 0 client.17148 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:53.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 shutdown_connections 2024-07-29T00:45:53.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1667365249 wait complete. 2024-07-29T00:45:53.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.232+0000 7fcfb5e01740 20 client.17148 trim_cache size 0 max 16384 2024-07-29T00:45:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:52 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:52 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:52 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:52 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:52 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:45:53.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:52 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 Processor -- start 2024-07-29T00:45:53.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 -- start start 2024-07-29T00:45:53.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:52 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:53.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:52 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:52 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:53.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:52 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:52 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:45:53.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:52 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f0adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f12b10 0x555c60f02e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1b3d70 con 0x555c60f08250 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1b58b0 con 0x555c60f07930 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.253+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1b4440 con 0x555c60f12b10 2024-07-29T00:45:53.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f0adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f0adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:43932/0 (socket says 172.21.15.175:43932) 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 -- 172.21.15.175:0/1404056756 learned_addr learned my addr 172.21.15.175:0/1404056756 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f12b10 0x555c60f02e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 -- 172.21.15.175:0/1404056756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f12b10 msgr2=0x555c60f02e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f12b10 0x555c60f02e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.254+0000 7fcfb1200640 1 -- 172.21.15.175:0/1404056756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 msgr2=0x555c60f102c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.255+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.255+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.255+0000 7fcfb1200640 1 -- 172.21.15.175:0/1404056756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c60f0e9e0 con 0x555c60f07930 2024-07-29T00:45:53.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.255+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:45:53.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.255+0000 7fcfb1200640 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f0adf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4011dd0 tx=0x7fcfa40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f8b5fda706a640 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:53.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.255+0000 7fcfa0c00640 1 -- 172.21.15.175:0/1404056756 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004900 con 0x555c60f07930 2024-07-29T00:45:53.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.256+0000 7fcfa0c00640 1 -- 172.21.15.175:0/1404056756 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4005030 con 0x555c60f07930 2024-07-29T00:45:53.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.256+0000 7fcfa0c00640 1 -- 172.21.15.175:0/1404056756 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40286b0 con 0x555c60f07930 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1404056756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 msgr2=0x555c60f0adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f0adf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4011dd0 tx=0x7fcfa40118d0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1404056756 shutdown_connections 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f12b10 0x555c60f02e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f08250 0x555c60f102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/1404056756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f07930 0x555c60f0adf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.257+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1404056756 >> 172.21.15.175:0/1404056756 conn(0x555c60f03ec0 msgr2=0x555c60f026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.258+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1404056756 shutdown_connections 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.258+0000 7fcfb5e01740 1 -- 172.21.15.175:0/1404056756 wait complete. 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.258+0000 7fcfb5e01740 1 -- start start 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.258+0000 7fcfb5e01740 1 Processor -- start 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f07930 0x555c60f1ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f1b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f0c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1b51e0 con 0x555c60f07930 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1b4b10 con 0x555c60f12b10 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1b1d70 con 0x555c60f08250 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb5e01740 10 client.0 _ll_register_callbacks cb 0x7fcf94009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f1b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfb1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f07930 0x555c60f1ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f1b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52742/0 (socket says 172.21.15.175:52742) 2024-07-29T00:45:53.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.259+0000 7fcfa3e00640 1 -- 172.21.15.175:0/4121061155 learned_addr learned my addr 172.21.15.175:0/4121061155 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:45:53.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfb1200640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f0c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfa3e00640 1 -- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 msgr2=0x555c60f0c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f0c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfa3e00640 1 -- 172.21.15.175:0/4121061155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f07930 msgr2=0x555c60f1ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f07930 0x555c60f1ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfa3e00640 1 -- 172.21.15.175:0/4121061155 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c60f0a8e0 con 0x555c60f08250 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfb1200640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f07930 0x555c60f1ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:45:53.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcfa3e00640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f1b240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x555c60f0f280 tx=0x7fcf940042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb0e1d24ab01dd4b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.260+0000 7fcf9a000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.261+0000 7fcf9a000640 1 -- 172.21.15.175:0/4121061155 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94028040 con 0x555c60f08250 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.261+0000 7fcf9a000640 1 -- 172.21.15.175:0/4121061155 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7fcf94010970 con 0x555c60f08250 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.261+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa4005030 con 0x555c60f08250 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.261+0000 7fcf9a000640 1 -- 172.21.15.175:0/4121061155 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcf94010b70 con 0x555c60f08250 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.261+0000 7fcfb5e01740 10 client.27250 fetch_fsmap learned FSMap version 43 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.261+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c60f8cfa0 con 0x555c60f08250 2024-07-29T00:45:53.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcf9a000640 1 -- 172.21.15.175:0/4121061155 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4005030 con 0x555c60f08250 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcf9a000640 1 -- 172.21.15.175:0/4121061155 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fcf94005310 con 0x555c60f08250 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb5e01740 10 client.27250 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb5e01740 10 client.27250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb5e01740 20 client.27250 populate_metadata read hostname 'smithi175' 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f8d750 0x555c60f8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb5e01740 4 client.27250 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c60e74550 con 0x555c60f8d750 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.262+0000 7fcfb1200640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f8d750 0x555c60f8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:45:53.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.263+0000 7fcfb1200640 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f8d750 0x555c60f8fb70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8f084c909891c1 server_cookie=83e4fbc24d075ee9 in_seq=0 out_seq=0 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcf9a000640 10 client.27250 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcf9a000640 1 -- 172.21.15.175:0/4121061155 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555c60e74550 con 0x555c60f8d750 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcf9a000640 10 client.27250 handle_command_reply: tid=0 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 client.27250 shutdown 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 msgr2=0x555c60f1b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f1b240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x555c60f0f280 tx=0x7fcf940042f0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 shutdown_connections 2024-07-29T00:45:53.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555c60f8d750 0x555c60f8fb70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c60f12b10 0x555c60f0c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c60f08250 0x555c60f1b240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 --2- 172.21.15.175:0/4121061155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c60f07930 0x555c60f1ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 >> 172.21.15.175:0/4121061155 conn(0x555c60f03ec0 msgr2=0x555c60f11a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.264+0000 7fcf9a000640 0 client.27250 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.265+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 shutdown_connections 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.265+0000 7fcfb5e01740 1 -- 172.21.15.175:0/4121061155 wait complete. 2024-07-29T00:45:53.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.265+0000 7fcfb5e01740 20 client.27250 trim_cache size 0 max 16384 2024-07-29T00:45:53.270 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:45:53.270 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:45:53.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:45:53.270 INFO:teuthology.orchestra.run.smithi175.stdout: "b75c0cec-bfaf-4f5e-9408-7e23c3bb9823": { 2024-07-29T00:45:53.271 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:45:53.271 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "b75c0cec-bfaf-4f5e-9408-7e23c3bb9823", 2024-07-29T00:45:53.271 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:45:53.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:45:53.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:45:53.271 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.265+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcf900a2d70 msgr2=0x7fcf900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcf900a2d70 0x7fcf900a5230 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x555c60f04d20 tx=0x7fcf9c009260 comp rx=0 tx=0).stop 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 msgr2=0x7fcfac1a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 0x7fcfac1a4570 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c004770 tx=0x7fcf9c003c10 comp rx=0 tx=0).stop 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 shutdown_connections 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fcf900a2d70 0x7fcf900a5230 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfac112740 0x7fcfac19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfac076d30 0x7fcfac1a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 --2- 172.21.15.175:0/2864252696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfac076410 0x7fcfac1a4570 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:45:53.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 >> 172.21.15.175:0/2864252696 conn(0x7fcfac0ff740 msgr2=0x7fcfac101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:45:53.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 shutdown_connections 2024-07-29T00:45:53.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:45:53.266+0000 7fcfb3200640 1 -- 172.21.15.175:0/2864252696 wait complete. 2024-07-29T00:45:53.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:52 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:53.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:52 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:52 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:53.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:52 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:53.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:52 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:45:53.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:52 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:45:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:53 smithi181 ceph-mon[36706]: pgmap v290: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:53 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:45:54.281 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:53 smithi175 ceph-mon[32138]: pgmap v290: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:54.282 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:53 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:45:54.284 INFO:tasks.cephfs.filesystem:scrub status for tag:b75c0cec-bfaf-4f5e-9408-7e23c3bb9823 - {'path': '/', 'tag': 'b75c0cec-bfaf-4f5e-9408-7e23c3bb9823', 'options': 'recursive,force'} 2024-07-29T00:45:54.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:53 smithi204 ceph-mon[37368]: pgmap v290: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:45:54.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:53 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:45:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:55 smithi175 ceph-mon[32138]: pgmap v291: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:55 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:45:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:55 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:45:55.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:55 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:45:55.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:55 smithi204 ceph-mon[37368]: pgmap v291: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:55.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:55 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:45:55.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:55 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:45:55.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:55 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:45:55.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:55 smithi181 ceph-mon[36706]: pgmap v291: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:45:55.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:55 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:45:55.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:55 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:45:55.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:55 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:45:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:57 smithi181 ceph-mon[36706]: pgmap v292: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-07-29T00:45:57.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:57 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:58.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:57 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:58.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:57 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:58.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:57 smithi175 ceph-mon[32138]: pgmap v292: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-07-29T00:45:58.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:57 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:58.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:57 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:58.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:57 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:57 smithi204 ceph-mon[37368]: pgmap v292: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-07-29T00:45:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:57 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:57 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:58.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:57 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:59.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:58 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:59.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:58 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:45:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:58 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:59.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:58 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:45:59.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:58 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:45:59.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:58 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:46:00.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:45:59 smithi175 ceph-mon[32138]: pgmap v293: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:00.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:45:59 smithi204 ceph-mon[37368]: pgmap v293: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:00.151 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:45:59 smithi181 ceph-mon[36706]: pgmap v293: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:02.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:01 smithi175 ceph-mon[32138]: pgmap v294: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:02.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:01 smithi204 ceph-mon[37368]: pgmap v294: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:02.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:01 smithi181 ceph-mon[36706]: pgmap v294: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:04.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:03 smithi175 ceph-mon[32138]: pgmap v295: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-07-29T00:46:04.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:03 smithi204 ceph-mon[37368]: pgmap v295: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-07-29T00:46:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:03 smithi181 ceph-mon[36706]: pgmap v295: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-07-29T00:46:06.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:05 smithi175 ceph-mon[32138]: pgmap v296: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-07-29T00:46:06.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:05 smithi204 ceph-mon[37368]: pgmap v296: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-07-29T00:46:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:05 smithi181 ceph-mon[36706]: pgmap v296: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-07-29T00:46:08.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:07 smithi175 ceph-mon[32138]: pgmap v297: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:46:08.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:07 smithi204 ceph-mon[37368]: pgmap v297: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:46:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:07 smithi181 ceph-mon[36706]: pgmap v297: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:46:09.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:08 smithi204 ceph-mon[37368]: osdmap e99: 12 total, 12 up, 12 in 2024-07-29T00:46:09.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:08 smithi181 ceph-mon[36706]: osdmap e99: 12 total, 12 up, 12 in 2024-07-29T00:46:09.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:08 smithi175 ceph-mon[32138]: osdmap e99: 12 total, 12 up, 12 in 2024-07-29T00:46:10.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:09 smithi204 ceph-mon[37368]: pgmap v299: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:46:10.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:09 smithi181 ceph-mon[36706]: pgmap v299: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:46:10.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:09 smithi175 ceph-mon[32138]: pgmap v299: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:46:12.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:11 smithi204 ceph-mon[37368]: pgmap v300: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:46:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:11 smithi181 ceph-mon[36706]: pgmap v300: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:46:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:11 smithi175 ceph-mon[32138]: pgmap v300: 97 pgs: 97 active+clean; 127 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:46:14.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:13 smithi204 ceph-mon[37368]: pgmap v301: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-07-29T00:46:14.087 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:13 smithi175 ceph-mon[32138]: pgmap v301: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-07-29T00:46:14.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:13 smithi181 ceph-mon[36706]: pgmap v301: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-07-29T00:46:16.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:15 smithi175 ceph-mon[32138]: pgmap v302: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-07-29T00:46:16.035 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:15 smithi175 ceph-mon[32138]: osdmap e100: 12 total, 12 up, 12 in 2024-07-29T00:46:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:15 smithi204 ceph-mon[37368]: pgmap v302: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-07-29T00:46:16.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:15 smithi204 ceph-mon[37368]: osdmap e100: 12 total, 12 up, 12 in 2024-07-29T00:46:16.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:15 smithi181 ceph-mon[36706]: pgmap v302: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-07-29T00:46:16.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:15 smithi181 ceph-mon[36706]: osdmap e100: 12 total, 12 up, 12 in 2024-07-29T00:46:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:17 smithi204 ceph-mon[37368]: pgmap v304: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 243 B/s wr, 12 op/s 2024-07-29T00:46:18.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:17 smithi181 ceph-mon[36706]: pgmap v304: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 243 B/s wr, 12 op/s 2024-07-29T00:46:18.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:17 smithi175 ceph-mon[32138]: pgmap v304: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 243 B/s wr, 12 op/s 2024-07-29T00:46:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:19 smithi204 ceph-mon[37368]: pgmap v305: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:46:20.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:19 smithi181 ceph-mon[36706]: pgmap v305: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:46:20.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:19 smithi175 ceph-mon[32138]: pgmap v305: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:46:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:21 smithi204 ceph-mon[37368]: pgmap v306: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:46:22.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:21 smithi181 ceph-mon[36706]: pgmap v306: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:46:22.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:21 smithi175 ceph-mon[32138]: pgmap v306: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:46:24.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:23 smithi181 ceph-mon[36706]: pgmap v307: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:46:24.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:23 smithi175 ceph-mon[32138]: pgmap v307: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:46:24.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:46:24.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:23 smithi204 ceph-mon[37368]: pgmap v307: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:46:24.748 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:25.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 -- 172.21.15.175:0/4218314937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41076c0 msgr2=0x7faab4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 --2- 172.21.15.175:0/4218314937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41076c0 0x7faab4107aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009930 tx=0x7faaa402a420 comp rx=0 tx=0).stop 2024-07-29T00:46:25.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 -- 172.21.15.175:0/4218314937 shutdown_connections 2024-07-29T00:46:25.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 --2- 172.21.15.175:0/4218314937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faab4102060 0x7faab410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 --2- 172.21.15.175:0/4218314937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab41016c0 0x7faab4101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 --2- 172.21.15.175:0/4218314937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41076c0 0x7faab4107aa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 -- 172.21.15.175:0/4218314937 >> 172.21.15.175:0/4218314937 conn(0x7faab40fd400 msgr2=0x7faab40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:25.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 -- 172.21.15.175:0/4218314937 shutdown_connections 2024-07-29T00:46:25.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.451+0000 7faab8e00640 1 -- 172.21.15.175:0/4218314937 wait complete. 2024-07-29T00:46:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.452+0000 7faab8e00640 1 Processor -- start 2024-07-29T00:46:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.452+0000 7faab8e00640 1 -- start start 2024-07-29T00:46:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41016c0 0x7faab419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 0x7faab419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faab41076c0 0x7faab419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111ac0 con 0x7faab41016c0 2024-07-29T00:46:25.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab8e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111940 con 0x7faab4102060 2024-07-29T00:46:25.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab8e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111c40 con 0x7faab41076c0 2024-07-29T00:46:25.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 0x7faab419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41016c0 0x7faab419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 0x7faab419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37736/0 (socket says 172.21.15.175:37736) 2024-07-29T00:46:25.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 -- 172.21.15.175:0/2248766955 learned_addr learned my addr 172.21.15.175:0/2248766955 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:25.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab3e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faab41076c0 0x7faab419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 -- 172.21.15.175:0/2248766955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faab41076c0 msgr2=0x7faab419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faab41076c0 0x7faab419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 -- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41016c0 msgr2=0x7faab419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41016c0 0x7faab419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.453+0000 7faab2a00640 1 -- 172.21.15.175:0/2248766955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab419f150 con 0x7faab4102060 2024-07-29T00:46:25.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.454+0000 7faab2a00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 0x7faab419a6e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faaa000ea60 tx=0x7faaa000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:25.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.454+0000 7faaabe00640 1 -- 172.21.15.175:0/2248766955 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa000cd20 con 0x7faab4102060 2024-07-29T00:46:25.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.454+0000 7faaabe00640 1 -- 172.21.15.175:0/2248766955 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa00035f0 con 0x7faab4102060 2024-07-29T00:46:25.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.454+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab419f440 con 0x7faab4102060 2024-07-29T00:46:25.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.454+0000 7faaabe00640 1 -- 172.21.15.175:0/2248766955 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0010690 con 0x7faab4102060 2024-07-29T00:46:25.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.454+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab41a6d20 con 0x7faab4102060 2024-07-29T00:46:25.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.457+0000 7faabce0e740 1 Processor -- start 2024-07-29T00:46:25.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 -- start start 2024-07-29T00:46:25.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faaabe00640 1 -- 172.21.15.175:0/2248766955 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7faaa00037a0 con 0x7faab4102060 2024-07-29T00:46:25.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122c8430 0x559f122d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122d2cf0 0x559f122d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f1222cad0 con 0x559f122c8430 2024-07-29T00:46:25.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559f121c4af0 con 0x559f122c7b10 2024-07-29T00:46:25.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faabce0e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559f12228a90 con 0x559f122d2cf0 2024-07-29T00:46:25.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faab3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122c8430 0x559f122d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faab3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122c8430 0x559f122d27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37592/0 (socket says 172.21.15.175:37592) 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faab3400640 1 -- 172.21.15.175:0/2447403025 learned_addr learned my addr 172.21.15.175:0/2447403025 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faab3e00640 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.458+0000 7faab2a00640 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122d2cf0 0x559f122d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faab3e00640 1 -- 172.21.15.175:0/2447403025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122d2cf0 msgr2=0x559f122d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faab3e00640 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122d2cf0 0x559f122d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faab3e00640 1 -- 172.21.15.175:0/2447403025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122c8430 msgr2=0x559f122d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faab3e00640 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122c8430 0x559f122d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faab3e00640 1 -- 172.21.15.175:0/2447403025 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f11f9e3c0 con 0x559f122c7b10 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faab3e00640 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122c7ef0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7faaac009f10 tx=0x7faaac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20f10010d8edd4e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.459+0000 7faaabe00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa980a2cf0 0x7faa980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faaabe00640 1 -- 172.21.15.175:0/2248766955 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7faaa00cf7e0 con 0x7faab4102060 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faaa8c00640 1 -- 172.21.15.175:0/2447403025 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac004d20 con 0x559f122c7b10 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faaa8c00640 1 -- 172.21.15.175:0/2447403025 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac004050 con 0x559f122c7b10 2024-07-29T00:46:25.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faaa8c00640 1 -- 172.21.15.175:0/2447403025 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac005280 con 0x559f122c7b10 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faab3400640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa980a2cf0 0x7faa980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faabce0e740 1 -- 172.21.15.175:0/2447403025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 msgr2=0x559f122c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faabce0e740 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122c7ef0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7faaac009f10 tx=0x7faaac011810 comp rx=0 tx=0).stop 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.460+0000 7faab3400640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa980a2cf0 0x7faa980a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faaa40096c0 tx=0x7faaa40058d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 -- 172.21.15.175:0/2447403025 shutdown_connections 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122d2cf0 0x559f122d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122c8430 0x559f122d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 --2- 172.21.15.175:0/2447403025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122c7ef0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 -- 172.21.15.175:0/2447403025 >> 172.21.15.175:0/2447403025 conn(0x559f122c40a0 msgr2=0x559f122c4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:25.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 -- 172.21.15.175:0/2447403025 shutdown_connections 2024-07-29T00:46:25.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 -- 172.21.15.175:0/2447403025 wait complete. 2024-07-29T00:46:25.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.461+0000 7faabce0e740 1 -- start start 2024-07-29T00:46:25.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 Processor -- start 2024-07-29T00:46:25.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f122d58c0 con 0x559f122d2cf0 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559f121e7bc0 con 0x559f122c7b10 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559f122d5a40 con 0x559f122c8430 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faabce0e740 10 client.0 _ll_register_callbacks cb 0x559f121d3d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faab3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faab3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faab2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faab3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37610/0 (socket says 172.21.15.175:37610) 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.462+0000 7faab3e00640 1 -- 172.21.15.175:0/1530279736 learned_addr learned my addr 172.21.15.175:0/1530279736 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faab2a00640 1 -- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 msgr2=0x559f122cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faab2a00640 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faab2a00640 1 -- 172.21.15.175:0/1530279736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 msgr2=0x559f122d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faab2a00640 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faab2a00640 1 -- 172.21.15.175:0/1530279736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f122ded80 con 0x559f122c8430 2024-07-29T00:46:25.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faab2a00640 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d00d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faaa00cce80 tx=0x7faaa0097890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4bc345d2d59fb0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.463+0000 7faa9d600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faa9d600640 1 -- 172.21.15.175:0/1530279736 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa00b1060 con 0x559f122c8430 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaac004050 con 0x559f122c8430 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faa9d600640 1 -- 172.21.15.175:0/1530279736 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7faaa00cd100 con 0x559f122c8430 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faa9d600640 1 -- 172.21.15.175:0/1530279736 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faaa009b790 con 0x559f122c8430 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 10 client.27262 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559f122e1740 con 0x559f122c8430 2024-07-29T00:46:25.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faa9d600640 1 -- 172.21.15.175:0/1530279736 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac004050 con 0x559f122c8430 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faa9d600640 1 -- 172.21.15.175:0/1530279736 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7faaa00cfbd0 con 0x559f122c8430 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 10 client.27262 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 20 client.27262 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.464+0000 7faabce0e740 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f122e2ba0 0x559f122e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.465+0000 7faabce0e740 4 client.27262 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.465+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f1223a040 con 0x559f122e2ba0 2024-07-29T00:46:25.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.465+0000 7faab3400640 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f122e2ba0 0x559f122e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.465+0000 7faab3400640 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f122e2ba0 0x559f122e4fb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60dd6d3e4ee9f986 server_cookie=d4199cb2ea9a4ecb in_seq=0 out_seq=0 2024-07-29T00:46:25.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.465+0000 7faa9d600640 10 client.27262 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:25.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.466+0000 7faa9d600640 1 -- 172.21.15.175:0/1530279736 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f1223a040 con 0x559f122e2ba0 2024-07-29T00:46:25.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.466+0000 7faa9d600640 10 client.27262 handle_command_reply: tid=0 2024-07-29T00:46:25.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.466+0000 7faabce0e740 1 client.27262 shutdown 2024-07-29T00:46:25.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.466+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 msgr2=0x559f122d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.466+0000 7faabce0e740 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d00d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faaa00cce80 tx=0x7faaa0097890 comp rx=0 tx=0).stop 2024-07-29T00:46:25.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 shutdown_connections 2024-07-29T00:46:25.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f122e2ba0 0x559f122e4fb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d00d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 --2- 172.21.15.175:0/1530279736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 >> 172.21.15.175:0/1530279736 conn(0x559f122c40a0 msgr2=0x559f122c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:25.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faa9d600640 0 client.27262 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:25.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 shutdown_connections 2024-07-29T00:46:25.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 1 -- 172.21.15.175:0/1530279736 wait complete. 2024-07-29T00:46:25.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.467+0000 7faabce0e740 20 client.27262 trim_cache size 0 max 16384 2024-07-29T00:46:25.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 Processor -- start 2024-07-29T00:46:25.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 -- start start 2024-07-29T00:46:25.486 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 0x559f122d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c8430 0x559f122c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faab41af800 con 0x559f122d2cf0 2024-07-29T00:46:25.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faab41b1340 con 0x559f122c8430 2024-07-29T00:46:25.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faabce0e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faab41afed0 con 0x559f122c7b10 2024-07-29T00:46:25.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faab3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faab3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 0x559f122d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faab3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 0x559f122d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:52828/0 (socket says 172.21.15.175:52828) 2024-07-29T00:46:25.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.486+0000 7faab3400640 1 -- 172.21.15.175:0/1411222844 learned_addr learned my addr 172.21.15.175:0/1411222844 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:25.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab2a00640 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c8430 0x559f122c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.490 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab3400640 1 -- 172.21.15.175:0/1411222844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c8430 msgr2=0x559f122c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab3400640 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c8430 0x559f122c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab3400640 1 -- 172.21.15.175:0/1411222844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 msgr2=0x559f122ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab3400640 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab3400640 1 -- 172.21.15.175:0/1411222844 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f122c4d30 con 0x559f122c7b10 2024-07-29T00:46:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faab3400640 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 0x559f122d0460 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faaa4004770 tx=0x7faaa403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abaad8e90c64aba1 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.487+0000 7faa9fe00640 1 -- 172.21.15.175:0/1411222844 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4035040 con 0x559f122c7b10 2024-07-29T00:46:25.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faa9fe00640 1 -- 172.21.15.175:0/1411222844 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa405a070 con 0x559f122c7b10 2024-07-29T00:46:25.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faa9fe00640 1 -- 172.21.15.175:0/1411222844 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa403d450 con 0x559f122c7b10 2024-07-29T00:46:25.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 -- 172.21.15.175:0/1411222844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 msgr2=0x559f122d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 0x559f122d0460 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faaa4004770 tx=0x7faaa403f000 comp rx=0 tx=0).stop 2024-07-29T00:46:25.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 -- 172.21.15.175:0/1411222844 shutdown_connections 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c8430 0x559f122c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 --2- 172.21.15.175:0/1411222844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c7b10 0x559f122d0460 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 -- 172.21.15.175:0/1411222844 >> 172.21.15.175:0/1411222844 conn(0x559f122c40a0 msgr2=0x559f122c29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 -- 172.21.15.175:0/1411222844 shutdown_connections 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.488+0000 7faabce0e740 1 -- 172.21.15.175:0/1411222844 wait complete. 2024-07-29T00:46:25.493 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 -- start start 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 Processor -- start 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faab41b0c70 con 0x559f122d2cf0 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faab41b05a0 con 0x559f122c7b10 2024-07-29T00:46:25.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.489+0000 7faabce0e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faab41ad800 con 0x559f122c8430 2024-07-29T00:46:25.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faabce0e740 10 client.0 _ll_register_callbacks cb 0x7faaa0009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:25.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37624/0 (socket says 172.21.15.175:37624) 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3e00640 1 -- 172.21.15.175:0/945695669 learned_addr learned my addr 172.21.15.175:0/945695669 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab2a00640 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3400640 1 -- 172.21.15.175:0/945695669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 msgr2=0x559f122d71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3400640 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3400640 1 -- 172.21.15.175:0/945695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 msgr2=0x559f122d76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3400640 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.490+0000 7faab3400640 1 -- 172.21.15.175:0/945695669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f122ca980 con 0x559f122c7b10 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faab3400640 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122d6c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faaa4031a90 tx=0x7faaa4031910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8157dc0558fe78a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faa9d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faa9d600640 1 -- 172.21.15.175:0/945695669 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4003970 con 0x559f122c7b10 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faa9d600640 1 -- 172.21.15.175:0/945695669 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7faaa4003b10 con 0x559f122c7b10 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa405a070 con 0x559f122c7b10 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faa9d600640 1 -- 172.21.15.175:0/945695669 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faaa4003d50 con 0x559f122c7b10 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faabce0e740 10 client.27320 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:25.499 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559f1234dc00 con 0x559f122c7b10 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.491+0000 7faa9d600640 1 -- 172.21.15.175:0/945695669 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa405a070 con 0x559f122c7b10 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faa9d600640 1 -- 172.21.15.175:0/945695669 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7faaa404b430 con 0x559f122c7b10 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faabce0e740 10 client.27320 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faabce0e740 10 client.27320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faabce0e740 20 client.27320 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faabce0e740 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f1234e2c0 0x559f123506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faabce0e740 4 client.27320 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:46:25.500 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f12234550 con 0x559f1234e2c0 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.492+0000 7faab2a00640 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f1234e2c0 0x559f123506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faab2a00640 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f1234e2c0 0x559f123506e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc08d90161bfa9f server_cookie=66cd088822a3c101 in_seq=0 out_seq=0 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faa9d600640 10 client.27320 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faa9d600640 1 -- 172.21.15.175:0/945695669 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559f12234550 con 0x559f1234e2c0 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faa9d600640 10 client.27320 handle_command_reply: tid=0 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 client.27320 shutdown 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 msgr2=0x559f122d6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.501 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122d6c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faaa4031a90 tx=0x7faaa4031910 comp rx=0 tx=0).stop 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 shutdown_connections 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x559f1234e2c0 0x559f123506e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f122d2cf0 0x559f122d76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f122c8430 0x559f122d71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 --2- 172.21.15.175:0/945695669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f122c7b10 0x559f122d6c70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faa9d600640 0 client.27320 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 >> 172.21.15.175:0/945695669 conn(0x559f122c40a0 msgr2=0x559f122c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:25.502 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 shutdown_connections 2024-07-29T00:46:25.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.493+0000 7faabce0e740 1 -- 172.21.15.175:0/945695669 wait complete. 2024-07-29T00:46:25.503 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faabce0e740 20 client.27320 trim_cache size 0 max 16384 2024-07-29T00:46:25.503 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:46:25.503 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:46:25.504 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:46:25.504 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:46:25.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa980a2cf0 msgr2=0x7faa980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa980a2cf0 0x7faa980a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faaa40096c0 tx=0x7faaa40058d0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 msgr2=0x7faab419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 0x7faab419a6e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faaa000ea60 tx=0x7faaa000ee60 comp rx=0 tx=0).stop 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 shutdown_connections 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faab41076c0 0x7faab419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab4102060 0x7faab419a6e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7faa980a2cf0 0x7faa980a51b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 --2- 172.21.15.175:0/2248766955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faab41016c0 0x7faab419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:25.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.494+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 >> 172.21.15.175:0/2248766955 conn(0x7faab40fd400 msgr2=0x7faab410cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:25.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.495+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 shutdown_connections 2024-07-29T00:46:25.506 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:25.495+0000 7faab8e00640 1 -- 172.21.15.175:0/2248766955 wait complete. 2024-07-29T00:46:26.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:25 smithi181 ceph-mon[36706]: pgmap v308: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:46:26.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:25 smithi175 ceph-mon[32138]: pgmap v308: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:46:26.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:25 smithi204 ceph-mon[37368]: pgmap v308: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:46:26.562 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:46:26.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:46:27.030 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:27.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 -- 172.21.15.175:0/2660134327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80107880 msgr2=0x7f2b80107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:27.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 --2- 172.21.15.175:0/2660134327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80107880 0x7f2b80107c60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009930 tx=0x7f2b7002a440 comp rx=0 tx=0).stop 2024-07-29T00:46:27.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 -- 172.21.15.175:0/2660134327 shutdown_connections 2024-07-29T00:46:27.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 --2- 172.21.15.175:0/2660134327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b80102220 0x7f2b8010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 --2- 172.21.15.175:0/2660134327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80101880 0x7f2b80101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 --2- 172.21.15.175:0/2660134327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80107880 0x7f2b80107c60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 -- 172.21.15.175:0/2660134327 >> 172.21.15.175:0/2660134327 conn(0x7f2b800fd5a0 msgr2=0x7f2b800ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:27.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 -- 172.21.15.175:0/2660134327 shutdown_connections 2024-07-29T00:46:27.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.694+0000 7f2b86000640 1 -- 172.21.15.175:0/2660134327 wait complete. 2024-07-29T00:46:27.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 Processor -- start 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 -- start start 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 0x7f2b8019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80102220 0x7f2b8019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b80107880 0x7f2b8019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80111cd0 con 0x7f2b80101880 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80111b50 con 0x7f2b80107880 2024-07-29T00:46:27.698 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.695+0000 7f2b86000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80111e50 con 0x7f2b80102220 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 0x7f2b8019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 0x7f2b8019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37644/0 (socket says 172.21.15.175:37644) 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7f400640 1 -- 172.21.15.175:0/3168688395 learned_addr learned my addr 172.21.15.175:0/3168688395 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7ea00640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80102220 0x7f2b8019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7fe00640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b80107880 0x7f2b8019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7ea00640 1 -- 172.21.15.175:0/3168688395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b80107880 msgr2=0x7f2b8019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7ea00640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b80107880 0x7f2b8019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7ea00640 1 -- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 msgr2=0x7f2b8019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7ea00640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 0x7f2b8019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7ea00640 1 -- 172.21.15.175:0/3168688395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8019f340 con 0x7f2b80102220 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.696+0000 7f2b7f400640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 0x7f2b8019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.697+0000 7f2b7ea00640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80102220 0x7f2b8019a8d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e9f0 tx=0x7f2b6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.697+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00ccc0 con 0x7f2b80102220 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.697+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c005020 con 0x7f2b80102220 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.697+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8019f630 con 0x7f2b80102220 2024-07-29T00:46:27.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.697+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004a00 con 0x7f2b80102220 2024-07-29T00:46:27.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.697+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b801a6f10 con 0x7f2b80102220 2024-07-29T00:46:27.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.698+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b80101ce0 con 0x7f2b80102220 2024-07-29T00:46:27.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.700+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2b6c0051d0 con 0x7f2b80102220 2024-07-29T00:46:27.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.702+0000 7f2b77e00640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2b640a2d70 0x7f2b640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:27.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.702+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f2b6c0cf3a0 con 0x7f2b80102220 2024-07-29T00:46:27.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.702+0000 7f2b7f400640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2b640a2d70 0x7f2b640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:27.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.703+0000 7f2b7f400640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2b640a2d70 0x7f2b640a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b70000c00 tx=0x7f2b70031510 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:27.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.704+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2b6c09bf10 con 0x7f2b80102220 2024-07-29T00:46:27.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.940+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8019b6f0 con 0x7f2b80102220 2024-07-29T00:46:27.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.942+0000 7f2b77e00640 1 -- 172.21.15.175:0/3168688395 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11994 (secure 0 0 0) 0x7f2b8019b6f0 con 0x7f2b80102220 2024-07-29T00:46:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:46:27.943 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":43,"btime":"2024-07-29T00:43:57:876956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:57.442709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:46:27.943 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 43 2024-07-29T00:46:27.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2b640a2d70 msgr2=0x7f2b640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2b640a2d70 0x7f2b640a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b70000c00 tx=0x7f2b70031510 comp rx=0 tx=0).stop 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80102220 msgr2=0x7f2b8019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80102220 0x7f2b8019a8d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e9f0 tx=0x7f2b6c00edf0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 shutdown_connections 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b80107880 0x7f2b8019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b80102220 0x7f2b8019a8d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f2b640a2d70 0x7f2b640a5230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 --2- 172.21.15.175:0/3168688395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b80101880 0x7f2b8019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:27.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 >> 172.21.15.175:0/3168688395 conn(0x7f2b800fd5a0 msgr2=0x7f2b8010ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:27.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 shutdown_connections 2024-07-29T00:46:27.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:27.945+0000 7f2b86000640 1 -- 172.21.15.175:0/3168688395 wait complete. 2024-07-29T00:46:28.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:27 smithi181 ceph-mon[36706]: pgmap v309: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:46:28.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:27 smithi175 ceph-mon[32138]: pgmap v309: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:46:28.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:27 smithi204 ceph-mon[37368]: pgmap v309: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:46:28.819 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:46:29.013 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:28 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3168688395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:46:29.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:28 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3168688395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:46:29.254 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:29.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:28 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3168688395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:46:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:29 smithi175 ceph-mon[32138]: pgmap v310: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:30.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.095+0000 7f6d59c00640 1 -- 172.21.15.175:0/1168424148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d54074310 msgr2=0x7f6d540746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.095+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1168424148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d54074310 0x7f6d540746f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009900 tx=0x7f6d4402a480 comp rx=0 tx=0).stop 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 -- 172.21.15.175:0/1168424148 shutdown_connections 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1168424148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d5410df00 0x7f6d54110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1168424148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 0x7f6d5410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1168424148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d54074310 0x7f6d540746f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 -- 172.21.15.175:0/1168424148 >> 172.21.15.175:0/1168424148 conn(0x7f6d540fd400 msgr2=0x7f6d540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 -- 172.21.15.175:0/1168424148 shutdown_connections 2024-07-29T00:46:30.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.096+0000 7f6d59c00640 1 -- 172.21.15.175:0/1168424148 wait complete. 2024-07-29T00:46:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.097+0000 7f6d59c00640 1 Processor -- start 2024-07-29T00:46:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.097+0000 7f6d59c00640 1 -- start start 2024-07-29T00:46:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d54074310 0x7f6d5419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 0x7f6d5419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 0x7f6d5419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54113cb0 con 0x7f6d5410df00 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54113b30 con 0x7f6d54074c30 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54113e30 con 0x7f6d54074310 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 0x7f6d5419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d54074310 0x7f6d5419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d4b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 0x7f6d5419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 0x7f6d5419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50952/0 (socket says 172.21.15.175:50952) 2024-07-29T00:46:30.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.098+0000 7f6d59200640 1 -- 172.21.15.175:0/1022024749 learned_addr learned my addr 172.21.15.175:0/1022024749 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d53e00640 1 -- 172.21.15.175:0/1022024749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 msgr2=0x7f6d5419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d53e00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 0x7f6d5419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d53e00640 1 -- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 msgr2=0x7f6d5419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d53e00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 0x7f6d5419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d53e00640 1 -- 172.21.15.175:0/1022024749 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5419f1e0 con 0x7f6d54074310 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d59200640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 0x7f6d5419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d4b400640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 0x7f6d5419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d53e00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d54074310 0x7f6d5419a230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d44005940 tx=0x7f6d440040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.099+0000 7f6d51600640 1 -- 172.21.15.175:0/1022024749 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4403f030 con 0x7f6d54074310 2024-07-29T00:46:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.100+0000 7f6d51600640 1 -- 172.21.15.175:0/1022024749 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d440044c0 con 0x7f6d54074310 2024-07-29T00:46:30.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.100+0000 7f6d51600640 1 -- 172.21.15.175:0/1022024749 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44035070 con 0x7f6d54074310 2024-07-29T00:46:30.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.100+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5419f470 con 0x7f6d54074310 2024-07-29T00:46:30.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.100+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5419f900 con 0x7f6d54074310 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.103+0000 7f6d5dca0740 1 Processor -- start 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.103+0000 7f6d51600640 1 -- 172.21.15.175:0/1022024749 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6d44042030 con 0x7f6d54074310 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.103+0000 7f6d5dca0740 1 -- start start 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e80e0 0x563c7e5e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c7e54cab0 con 0x563c7e5f1910 2024-07-29T00:46:30.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563c7e4e4a80 con 0x563c7e5e6b70 2024-07-29T00:46:30.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563c7e548a70 con 0x563c7e5e80e0 2024-07-29T00:46:30.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d59200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e80e0 0x563c7e5e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d4b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56142/0 (socket says 172.21.15.175:56142) 2024-07-29T00:46:30.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.104+0000 7f6d53e00640 1 -- 172.21.15.175:0/327227909 learned_addr learned my addr 172.21.15.175:0/327227909 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:30.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d4b400640 1 -- 172.21.15.175:0/327227909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e80e0 msgr2=0x563c7e5e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d4b400640 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e80e0 0x563c7e5e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d4b400640 1 -- 172.21.15.175:0/327227909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 msgr2=0x563c7e5e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d4b400640 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d4b400640 1 -- 172.21.15.175:0/327227909 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7e2be3c0 con 0x563c7e5f1910 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d53e00640 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d4b400640 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f3d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009fa0 tx=0x7f6d3c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a309c25d20c372 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d51600640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d380a2cf0 0x7f6d380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d49600640 1 -- 172.21.15.175:0/327227909 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004020 con 0x563c7e5f1910 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d49600640 1 -- 172.21.15.175:0/327227909 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3c0041c0 con 0x563c7e5f1910 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d49600640 1 -- 172.21.15.175:0/327227909 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00f960 con 0x563c7e5f1910 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.105+0000 7f6d51600640 1 -- 172.21.15.175:0/1022024749 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f6d440ef790 con 0x7f6d54074310 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d59200640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d380a2cf0 0x7f6d380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 -- 172.21.15.175:0/327227909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 msgr2=0x563c7e5f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f3d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009fa0 tx=0x7f6d3c0118d0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 -- 172.21.15.175:0/327227909 shutdown_connections 2024-07-29T00:46:30.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f3d00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/327227909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e80e0 0x563c7e5e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 -- 172.21.15.175:0/327227909 >> 172.21.15.175:0/327227909 conn(0x563c7e5af950 msgr2=0x563c7e5e3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:30.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d59200640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d380a2cf0 0x7f6d380a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0049d0 tx=0x7f6d4c009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:30.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.106+0000 7f6d5dca0740 1 -- 172.21.15.175:0/327227909 shutdown_connections 2024-07-29T00:46:30.110 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.107+0000 7f6d5dca0740 1 -- 172.21.15.175:0/327227909 wait complete. 2024-07-29T00:46:30.113 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.107+0000 7f6d5dca0740 1 -- start start 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.107+0000 7f6d5dca0740 1 Processor -- start 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 0x563c7e5ee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e5eed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c7e5f46b0 con 0x563c7e5e6b70 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563c7e507c10 con 0x563c7e5e80e0 2024-07-29T00:46:30.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563c7e5f4830 con 0x563c7e5f1910 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d5dca0740 10 client.0 _ll_register_callbacks cb 0x563c7e4d0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 0x563c7e5ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 0x563c7e5ee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50968/0 (socket says 172.21.15.175:50968) 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d53e00640 1 -- 172.21.15.175:0/3888457484 learned_addr learned my addr 172.21.15.175:0/3888457484 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d59200640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.108+0000 7f6d4b400640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e5eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d53e00640 1 -- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 msgr2=0x563c7e5ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d53e00640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.115 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d53e00640 1 -- 172.21.15.175:0/3888457484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 msgr2=0x563c7e5eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d53e00640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e5eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d53e00640 1 -- 172.21.15.175:0/3888457484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7e50b1e0 con 0x563c7e5e6b70 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d53e00640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 0x563c7e5ee850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b8680 tx=0x7f6d4402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4758426b498143b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d42000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d42000640 1 -- 172.21.15.175:0/3888457484 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44049070 con 0x563c7e5e6b70 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d42000640 1 -- 172.21.15.175:0/3888457484 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f6d440ed0b0 con 0x563c7e5e6b70 2024-07-29T00:46:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.109+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3c0041c0 con 0x563c7e5e6b70 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d42000640 1 -- 172.21.15.175:0/3888457484 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6d440d2050 con 0x563c7e5e6b70 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 10 client.17187 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563c7e600450 con 0x563c7e5e6b70 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d42000640 1 -- 172.21.15.175:0/3888457484 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3c0041c0 con 0x563c7e5e6b70 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d42000640 1 -- 172.21.15.175:0/3888457484 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6d440c43e0 con 0x563c7e5e6b70 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 10 client.17187 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 10 client.17187 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:46:30.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 20 client.17187 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e6017f0 0x563c7e603c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 4 client.17187 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.110+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c7e55a020 con 0x563c7e6017f0 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.111+0000 7f6d4b400640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e6017f0 0x563c7e603c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.111+0000 7f6d4b400640 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e6017f0 0x563c7e603c60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a374cd7304e7f119 server_cookie=de979d846d71ca9d in_seq=0 out_seq=0 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.111+0000 7f6d42000640 10 client.17187 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.112+0000 7f6d42000640 1 -- 172.21.15.175:0/3888457484 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c7e55a020 con 0x563c7e6017f0 2024-07-29T00:46:30.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.112+0000 7f6d42000640 10 client.17187 handle_command_reply: tid=0 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.112+0000 7f6d5dca0740 1 client.17187 shutdown 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.112+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 msgr2=0x563c7e5ee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.112+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 0x563c7e5ee850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b8680 tx=0x7f6d4402a480 comp rx=0 tx=0).stop 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 shutdown_connections 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e6017f0 0x563c7e603c60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e5eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/3888457484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e6b70 0x563c7e5ee850 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d42000640 0 client.17187 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:46:30.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 >> 172.21.15.175:0/3888457484 conn(0x563c7e5af950 msgr2=0x563c7e5e8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:30.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 shutdown_connections 2024-07-29T00:46:30.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 1 -- 172.21.15.175:0/3888457484 wait complete. 2024-07-29T00:46:30.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.113+0000 7f6d5dca0740 20 client.17187 trim_cache size 0 max 16384 2024-07-29T00:46:30.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 Processor -- start 2024-07-29T00:46:30.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 -- start start 2024-07-29T00:46:30.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 0x563c7e609510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e609a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541af740 con 0x563c7e5f1910 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541b1280 con 0x563c7e5e80e0 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541afe10 con 0x563c7e5e6b70 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 0x563c7e609510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 0x563c7e609510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33708/0 (socket says 172.21.15.175:33708) 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d53e00640 1 -- 172.21.15.175:0/1563045364 learned_addr learned my addr 172.21.15.175:0/1563045364 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.128+0000 7f6d59200640 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d4b400640 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e609a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d53e00640 1 -- 172.21.15.175:0/1563045364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 msgr2=0x563c7e609a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d53e00640 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e609a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d53e00640 1 -- 172.21.15.175:0/1563045364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 msgr2=0x563c7e5f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d53e00640 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d53e00640 1 -- 172.21.15.175:0/1563045364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7e5e3f00 con 0x563c7e5e6b70 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d59200640 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:46:30.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d53e00640 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 0x563c7e609510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6d440bb9b0 tx=0x7f6d440ba1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb1d7bf4ad21d44 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:30.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d48c00640 1 -- 172.21.15.175:0/1563045364 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440d9070 con 0x563c7e5e6b70 2024-07-29T00:46:30.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d48c00640 1 -- 172.21.15.175:0/1563045364 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d440f5220 con 0x563c7e5e6b70 2024-07-29T00:46:30.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.129+0000 7f6d48c00640 1 -- 172.21.15.175:0/1563045364 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44049070 con 0x563c7e5e6b70 2024-07-29T00:46:30.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 -- 172.21.15.175:0/1563045364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 msgr2=0x563c7e609510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 0x563c7e609510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6d440bb9b0 tx=0x7f6d440ba1c0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 -- 172.21.15.175:0/1563045364 shutdown_connections 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5f1910 0x563c7e5f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e80e0 0x563c7e609a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/1563045364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5e6b70 0x563c7e609510 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 -- 172.21.15.175:0/1563045364 >> 172.21.15.175:0/1563045364 conn(0x563c7e5af950 msgr2=0x563c7e5e9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 -- 172.21.15.175:0/1563045364 shutdown_connections 2024-07-29T00:46:30.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.130+0000 7f6d5dca0740 1 -- 172.21.15.175:0/1563045364 wait complete. 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.131+0000 7f6d5dca0740 1 -- start start 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.131+0000 7f6d5dca0740 1 Processor -- start 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 0x563c7e5f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541b0bb0 con 0x563c7e5e80e0 2024-07-29T00:46:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541b04e0 con 0x563c7e5e6b70 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541ad740 con 0x563c7e5f1910 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d5dca0740 10 client.0 _ll_register_callbacks cb 0x7f6d44009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d4b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 0x563c7e5f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d59200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d4b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 0x563c7e5f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50990/0 (socket says 172.21.15.175:50990) 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d4b400640 1 -- 172.21.15.175:0/771939521 learned_addr learned my addr 172.21.15.175:0/771939521 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d53e00640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d59200640 1 -- 172.21.15.175:0/771939521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 msgr2=0x563c7e5f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d59200640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d59200640 1 -- 172.21.15.175:0/771939521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 msgr2=0x563c7e5f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d59200640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 0x563c7e5f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d59200640 1 -- 172.21.15.175:0/771939521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7e66b4e0 con 0x563c7e5f1910 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.132+0000 7f6d4b400640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 0x563c7e5f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.133+0000 7f6d59200640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5f62a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x563c7e5ed030 tx=0x7f6d4c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185d89fe68a779b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:30.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.133+0000 7f6d42000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.133+0000 7f6d42000640 1 -- 172.21.15.175:0/771939521 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0112a0 con 0x563c7e5f1910 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.133+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d440f5220 con 0x563c7e5f1910 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.133+0000 7f6d42000640 1 -- 172.21.15.175:0/771939521 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f6d4c004b20 con 0x563c7e5f1910 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d42000640 1 -- 172.21.15.175:0/771939521 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6d4c004d20 con 0x563c7e5f1910 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 10 client.27292 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563c7e66c810 con 0x563c7e5f1910 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d42000640 1 -- 172.21.15.175:0/771939521 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d440f5220 con 0x563c7e5f1910 2024-07-29T00:46:30.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d42000640 1 -- 172.21.15.175:0/771939521 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f6d4c005140 con 0x563c7e5f1910 2024-07-29T00:46:30.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 10 client.27292 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:30.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 10 client.27292 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:46:30.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 20 client.27292 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:30.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e66cf90 0x563c7e66f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:30.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 4 client.27292 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:46:30.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.134+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c7e554530 con 0x563c7e66cf90 2024-07-29T00:46:30.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d53e00640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e66cf90 0x563c7e66f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:30.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d53e00640 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e66cf90 0x563c7e66f3b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbba5b8c40ba7bee server_cookie=21d88b0d17c07ec6 in_seq=0 out_seq=0 2024-07-29T00:46:30.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d42000640 10 client.27292 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:46:30.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d42000640 1 -- 172.21.15.175:0/771939521 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c7e554530 con 0x563c7e66cf90 2024-07-29T00:46:30.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d42000640 10 client.27292 handle_command_reply: tid=0 2024-07-29T00:46:30.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 client.27292 shutdown 2024-07-29T00:46:30.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 msgr2=0x563c7e5f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5f62a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x563c7e5ed030 tx=0x7f6d4c004a40 comp rx=0 tx=0).stop 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 shutdown_connections 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c7e5f1910 0x563c7e5f62a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c7e5e80e0 0x563c7e5f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x563c7e66cf90 0x563c7e66f3b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 --2- 172.21.15.175:0/771939521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c7e5e6b70 0x563c7e5f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.135+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 >> 172.21.15.175:0/771939521 conn(0x563c7e5af950 msgr2=0x563c7e5f00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d42000640 0 client.27292 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:46:30.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 shutdown_connections 2024-07-29T00:46:30.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d5dca0740 1 -- 172.21.15.175:0/771939521 wait complete. 2024-07-29T00:46:30.144 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d5dca0740 20 client.27292 trim_cache size 0 max 16384 2024-07-29T00:46:30.144 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d380a2cf0 msgr2=0x7f6d380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d380a2cf0 0x7f6d380a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0049d0 tx=0x7f6d4c009500 comp rx=0 tx=0).stop 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d54074310 msgr2=0x7f6d5419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.136+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d54074310 0x7f6d5419a230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d44005940 tx=0x7f6d440040c0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 shutdown_connections 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d380a2cf0 0x7f6d380a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.145 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d5410df00 0x7f6d5419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d54074c30 0x7f6d5419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 --2- 172.21.15.175:0/1022024749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d54074310 0x7f6d5419a230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:30.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 >> 172.21.15.175:0/1022024749 conn(0x7f6d540fd400 msgr2=0x7f6d540fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:30.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 shutdown_connections 2024-07-29T00:46:30.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:30.137+0000 7f6d59c00640 1 -- 172.21.15.175:0/1022024749 wait complete. 2024-07-29T00:46:30.150 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:29 smithi181 ceph-mon[36706]: pgmap v310: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:30.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:29 smithi204 ceph-mon[37368]: pgmap v310: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:31.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:46:31.662 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:32.065 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:31 smithi175 ceph-mon[32138]: pgmap v311: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:32.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:31 smithi181 ceph-mon[36706]: pgmap v311: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:32.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:31 smithi204 ceph-mon[37368]: pgmap v311: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:32.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.349+0000 7fe963e00640 1 -- 172.21.15.175:0/1524404831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964107d10 msgr2=0x7fe964108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.349+0000 7fe963e00640 1 --2- 172.21.15.175:0/1524404831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964107d10 0x7fe964108130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe950009930 tx=0x7fe95002a420 comp rx=0 tx=0).stop 2024-07-29T00:46:32.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 -- 172.21.15.175:0/1524404831 shutdown_connections 2024-07-29T00:46:32.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 --2- 172.21.15.175:0/1524404831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964108670 0x7fe964114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 --2- 172.21.15.175:0/1524404831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964107d10 0x7fe964108130 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 --2- 172.21.15.175:0/1524404831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe96410dd50 0x7fe96410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 -- 172.21.15.175:0/1524404831 >> 172.21.15.175:0/1524404831 conn(0x7fe964077670 msgr2=0x7fe964079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:32.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 -- 172.21.15.175:0/1524404831 shutdown_connections 2024-07-29T00:46:32.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.350+0000 7fe963e00640 1 -- 172.21.15.175:0/1524404831 wait complete. 2024-07-29T00:46:32.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 Processor -- start 2024-07-29T00:46:32.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 -- start start 2024-07-29T00:46:32.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 0x7fe96419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964108670 0x7fe96419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe96410dd50 0x7fe9641a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe964118050 con 0x7fe964108670 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe964117ed0 con 0x7fe964107d10 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.351+0000 7fe963e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641181d0 con 0x7fe96410dd50 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 0x7fe96419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964108670 0x7fe96419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 0x7fe96419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56190/0 (socket says 172.21.15.175:56190) 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 -- 172.21.15.175:0/4187138381 learned_addr learned my addr 172.21.15.175:0/4187138381 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe963400640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe96410dd50 0x7fe9641a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 -- 172.21.15.175:0/4187138381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe96410dd50 msgr2=0x7fe9641a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe96410dd50 0x7fe9641a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 -- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964108670 msgr2=0x7fe96419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964108670 0x7fe96419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.352+0000 7fe962a00640 1 -- 172.21.15.175:0/4187138381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9641a3560 con 0x7fe964107d10 2024-07-29T00:46:32.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.353+0000 7fe962a00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 0x7fe96419e5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe95400e9f0 tx=0x7fe95400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:32.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.353+0000 7fe95b400640 1 -- 172.21.15.175:0/4187138381 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95400cd80 con 0x7fe964107d10 2024-07-29T00:46:32.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.353+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9641a3850 con 0x7fe964107d10 2024-07-29T00:46:32.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.353+0000 7fe95b400640 1 -- 172.21.15.175:0/4187138381 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe954010910 con 0x7fe964107d10 2024-07-29T00:46:32.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.353+0000 7fe95b400640 1 -- 172.21.15.175:0/4187138381 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954018990 con 0x7fe964107d10 2024-07-29T00:46:32.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.353+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641ab130 con 0x7fe964107d10 2024-07-29T00:46:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.356+0000 7fe96c809740 1 Processor -- start 2024-07-29T00:46:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.356+0000 7fe95b400640 1 -- 172.21.15.175:0/4187138381 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe954010ac0 con 0x7fe964107d10 2024-07-29T00:46:32.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.356+0000 7fe96c809740 1 -- start start 2024-07-29T00:46:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe96c809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530af9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe96c809740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe96c809740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530b04910 0x55a530b06d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe96c809740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a530a5fab0 con 0x55a530b04910 2024-07-29T00:46:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe96c809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a5309f7a80 con 0x55a530afb0e0 2024-07-29T00:46:32.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe96c809740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a530a5ba70 con 0x55a530af9b70 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe963400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530af9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe963400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530af9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56200/0 (socket says 172.21.15.175:56200) 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe963400640 1 -- 172.21.15.175:0/942185878 learned_addr learned my addr 172.21.15.175:0/942185878 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe962a00640 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.357+0000 7fe962000640 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530b04910 0x55a530b06d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe963400640 1 -- 172.21.15.175:0/942185878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 msgr2=0x55a530af9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.360 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe963400640 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe963400640 1 -- 172.21.15.175:0/942185878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530b04910 msgr2=0x55a530b06d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe963400640 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530b04910 0x55a530b06d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe963400640 1 -- 172.21.15.175:0/942185878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5307d13c0 con 0x55a530afb0e0 2024-07-29T00:46:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe962a00640 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe963400640 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530af9630 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009fc0 tx=0x7fe95c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b617cf96d47283 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:32.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.358+0000 7fe95b400640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe9480a2cf0 0x7fe9480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe94fe00640 1 -- 172.21.15.175:0/942185878 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c004d20 con 0x55a530afb0e0 2024-07-29T00:46:32.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe94fe00640 1 -- 172.21.15.175:0/942185878 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c004050 con 0x55a530afb0e0 2024-07-29T00:46:32.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe94fe00640 1 -- 172.21.15.175:0/942185878 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c005260 con 0x55a530afb0e0 2024-07-29T00:46:32.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe95b400640 1 -- 172.21.15.175:0/4187138381 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fe9540cf790 con 0x7fe964107d10 2024-07-29T00:46:32.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe962a00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe9480a2cf0 0x7fe9480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe96c809740 1 -- 172.21.15.175:0/942185878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 msgr2=0x55a530af9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe96c809740 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530af9630 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009fc0 tx=0x7fe95c0118d0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe962a00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe9480a2cf0 0x7fe9480a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55a530af7060 tx=0x7fe9540d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe96c809740 1 -- 172.21.15.175:0/942185878 shutdown_connections 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe96c809740 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530b04910 0x55a530b06d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.359+0000 7fe96c809740 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.360+0000 7fe96c809740 1 --2- 172.21.15.175:0/942185878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530af9630 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.360+0000 7fe96c809740 1 -- 172.21.15.175:0/942185878 >> 172.21.15.175:0/942185878 conn(0x55a530ac2950 msgr2=0x55a530af6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.360+0000 7fe96c809740 1 -- 172.21.15.175:0/942185878 shutdown_connections 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.360+0000 7fe96c809740 1 -- 172.21.15.175:0/942185878 wait complete. 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.360+0000 7fe96c809740 1 -- start start 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.360+0000 7fe96c809740 1 Processor -- start 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530af9b70 0x55a530b018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530b01e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530b04910 0x55a530b02350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a530b076b0 con 0x55a530af9b70 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a530a1ac10 con 0x55a530afb0e0 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a530b07830 con 0x55a530b04910 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe96c809740 10 client.0 _ll_register_callbacks cb 0x55a5309e3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe963400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530af9b70 0x55a530b018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe962000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530b01e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe962000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530b01e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56202/0 (socket says 172.21.15.175:56202) 2024-07-29T00:46:32.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe962000640 1 -- 172.21.15.175:0/2487204717 learned_addr learned my addr 172.21.15.175:0/2487204717 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.361+0000 7fe963400640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530b04910 0x55a530b02350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe962000640 1 -- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530b04910 msgr2=0x55a530b02350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe962000640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530b04910 0x55a530b02350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe962000640 1 -- 172.21.15.175:0/2487204717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530af9b70 msgr2=0x55a530b018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe962000640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530af9b70 0x55a530b018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe962000640 1 -- 172.21.15.175:0/2487204717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a530a1e1e0 con 0x55a530afb0e0 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe963400640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530b04910 0x55a530b02350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:46:32.369 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe962000640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530b01e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe9500317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3bae5c877cce35 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe94cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe94cc00640 1 -- 172.21.15.175:0/2487204717 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035040 con 0x55a530afb0e0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95c004050 con 0x55a530afb0e0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.362+0000 7fe94cc00640 1 -- 172.21.15.175:0/2487204717 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fe950004240 con 0x55a530afb0e0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe94cc00640 1 -- 172.21.15.175:0/2487204717 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe950004440 con 0x55a530afb0e0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 10 client.27356 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a530b134b0 con 0x55a530afb0e0 2024-07-29T00:46:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe94cc00640 1 -- 172.21.15.175:0/2487204717 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c004050 con 0x55a530afb0e0 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe94cc00640 1 -- 172.21.15.175:0/2487204717 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fe950004c50 con 0x55a530afb0e0 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 10 client.27356 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 10 client.27356 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 20 client.27356 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b148c0 0x55a530b16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 4 client.27356 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:32.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.363+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a530a6d020 con 0x55a530b148c0 2024-07-29T00:46:32.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.364+0000 7fe963400640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b148c0 0x55a530b16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.364+0000 7fe963400640 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b148c0 0x55a530b16cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0c0e2cdcbd6cfe server_cookie=ab0a6ea247a39d76 in_seq=0 out_seq=0 2024-07-29T00:46:32.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.364+0000 7fe94cc00640 10 client.27356 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:46:32.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe94cc00640 1 -- 172.21.15.175:0/2487204717 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a530a6d020 con 0x55a530b148c0 2024-07-29T00:46:32.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe94cc00640 10 client.27356 handle_command_reply: tid=0 2024-07-29T00:46:32.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 client.27356 shutdown 2024-07-29T00:46:32.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 msgr2=0x55a530b01e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530b01e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe9500317e0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 shutdown_connections 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b148c0 0x55a530b16cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530b04910 0x55a530b02350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530afb0e0 0x55a530b01e10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe94cc00640 0 client.27356 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 --2- 172.21.15.175:0/2487204717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530af9b70 0x55a530b018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.365+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 >> 172.21.15.175:0/2487204717 conn(0x55a530ac2950 msgr2=0x55a530afbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:32.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.366+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 shutdown_connections 2024-07-29T00:46:32.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.366+0000 7fe96c809740 1 -- 172.21.15.175:0/2487204717 wait complete. 2024-07-29T00:46:32.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.366+0000 7fe96c809740 20 client.27356 trim_cache size 0 max 16384 2024-07-29T00:46:32.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.380+0000 7fe96c809740 1 Processor -- start 2024-07-29T00:46:32.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.380+0000 7fe96c809740 1 -- start start 2024-07-29T00:46:32.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.380+0000 7fe96c809740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe96c809740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe96c809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b1cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe96c809740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b3c10 con 0x55a530afb0e0 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe96c809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b5750 con 0x55a530b04910 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe96c809740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b42e0 con 0x55a530af9b70 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe963400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b1c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51012/0 (socket says 172.21.15.175:51012) 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 -- 172.21.15.175:0/4290110154 learned_addr learned my addr 172.21.15.175:0/4290110154 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:32.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe963400640 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b1cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 -- 172.21.15.175:0/4290110154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 msgr2=0x55a530af63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 -- 172.21.15.175:0/4290110154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 msgr2=0x55a530b1cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b1cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe962000640 1 -- 172.21.15.175:0/4290110154 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a530b17330 con 0x55a530afb0e0 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.381+0000 7fe963400640 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.382+0000 7fe962000640 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b1c580 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe950009850 tx=0x7fe95002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0c13ddf3a9adcd server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.382+0000 7fe94f400640 1 -- 172.21.15.175:0/4290110154 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035070 con 0x55a530afb0e0 2024-07-29T00:46:32.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.382+0000 7fe94f400640 1 -- 172.21.15.175:0/4290110154 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe950005030 con 0x55a530afb0e0 2024-07-29T00:46:32.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.382+0000 7fe94f400640 1 -- 172.21.15.175:0/4290110154 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500049b0 con 0x55a530afb0e0 2024-07-29T00:46:32.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 -- 172.21.15.175:0/4290110154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 msgr2=0x55a530b1c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b1c580 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe950009850 tx=0x7fe95002a420 comp rx=0 tx=0).stop 2024-07-29T00:46:32.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 -- 172.21.15.175:0/4290110154 shutdown_connections 2024-07-29T00:46:32.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b1c580 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 --2- 172.21.15.175:0/4290110154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530af63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 -- 172.21.15.175:0/4290110154 >> 172.21.15.175:0/4290110154 conn(0x55a530ac2950 msgr2=0x55a530af5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:32.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 -- 172.21.15.175:0/4290110154 shutdown_connections 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 -- 172.21.15.175:0/4290110154 wait complete. 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.383+0000 7fe96c809740 1 -- start start 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 Processor -- start 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530b776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b77c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b0c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b5080 con 0x55a530afb0e0 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b49b0 con 0x55a530b04910 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b1c10 con 0x55a530af9b70 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe96c809740 10 client.0 _ll_register_callbacks cb 0x7fe950009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.384+0000 7fe963400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530b776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b77c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b77c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51020/0 (socket says 172.21.15.175:51020) 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 -- 172.21.15.175:0/1881445569 learned_addr learned my addr 172.21.15.175:0/1881445569 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:32.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe963400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530b776f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33772/0 (socket says 172.21.15.175:33772) 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe963400640 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b0c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 -- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 msgr2=0x55a530b776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530b776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 -- 172.21.15.175:0/1881445569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 msgr2=0x55a530b0c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b0c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 -- 172.21.15.175:0/1881445569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a530b0cdc0 con 0x55a530afb0e0 2024-07-29T00:46:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe962000640 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b77c30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe950005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5c93357e1957d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.385+0000 7fe94cc00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe94cc00640 1 -- 172.21.15.175:0/1881445569 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035040 con 0x55a530afb0e0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe94cc00640 1 -- 172.21.15.175:0/1881445569 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fe950003eb0 con 0x55a530afb0e0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe950005030 con 0x55a530afb0e0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe94cc00640 1 -- 172.21.15.175:0/1881445569 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe950064050 con 0x55a530afb0e0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe96c809740 10 client.17211 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a530b7ef20 con 0x55a530afb0e0 2024-07-29T00:46:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe94cc00640 1 -- 172.21.15.175:0/1881445569 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe950005030 con 0x55a530afb0e0 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.386+0000 7fe94cc00640 1 -- 172.21.15.175:0/1881445569 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fe950004c50 con 0x55a530afb0e0 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe96c809740 10 client.17211 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe96c809740 10 client.17211 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe96c809740 20 client.17211 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe96c809740 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b7f5e0 0x55a530b81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe96c809740 4 client.17211 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:46:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a530a67530 con 0x55a530b7f5e0 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.387+0000 7fe963400640 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b7f5e0 0x55a530b81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe963400640 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b7f5e0 0x55a530b81a00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd1a4f46e02fb2cf server_cookie=581695930512d672 in_seq=0 out_seq=0 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe94cc00640 10 client.17211 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe94cc00640 1 -- 172.21.15.175:0/1881445569 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a530a67530 con 0x55a530b7f5e0 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe94cc00640 10 client.17211 handle_command_reply: tid=0 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 client.17211 shutdown 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 msgr2=0x55a530b77c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b77c30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe950005c90 comp rx=0 tx=0).stop 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 shutdown_connections 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x55a530b7f5e0 0x55a530b81a00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a530b04910 0x55a530b0c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a530afb0e0 0x55a530b77c30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 --2- 172.21.15.175:0/1881445569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a530af9b70 0x55a530b776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 >> 172.21.15.175:0/1881445569 conn(0x55a530ac2950 msgr2=0x55a530b03130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe94cc00640 0 client.17211 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:46:32.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 shutdown_connections 2024-07-29T00:46:32.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 1 -- 172.21.15.175:0/1881445569 wait complete. 2024-07-29T00:46:32.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.388+0000 7fe96c809740 20 client.17211 trim_cache size 0 max 16384 2024-07-29T00:46:32.397 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:46:32.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe9480a2cf0 msgr2=0x7fe9480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe9480a2cf0 0x7fe9480a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55a530af7060 tx=0x7fe9540d5530 comp rx=0 tx=0).stop 2024-07-29T00:46:32.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 msgr2=0x7fe96419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:32.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 0x7fe96419e5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe95400e9f0 tx=0x7fe95400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.398 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 shutdown_connections 2024-07-29T00:46:32.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe96410dd50 0x7fe9641a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fe9480a2cf0 0x7fe9480a51b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe964108670 0x7fe96419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 --2- 172.21.15.175:0/4187138381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe964107d10 0x7fe96419e5b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:32.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 >> 172.21.15.175:0/4187138381 conn(0x7fe964077670 msgr2=0x7fe964078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:32.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.389+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 shutdown_connections 2024-07-29T00:46:32.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:32.390+0000 7fe963e00640 1 -- 172.21.15.175:0/4187138381 wait complete. 2024-07-29T00:46:33.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:46:33.925 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:34.172 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:33 smithi175 ceph-mon[32138]: pgmap v312: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:34.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:33 smithi181 ceph-mon[36706]: pgmap v312: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:34.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:33 smithi204 ceph-mon[37368]: pgmap v312: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:34.804 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 -- 172.21.15.175:0/1808750490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c102a30 msgr2=0x7f491c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 --2- 172.21.15.175:0/1808750490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c102a30 0x7f491c102e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f490c009900 tx=0x7f490c02a490 comp rx=0 tx=0).stop 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 -- 172.21.15.175:0/1808750490 shutdown_connections 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 --2- 172.21.15.175:0/1808750490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f491c10be60 0x7f491c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 --2- 172.21.15.175:0/1808750490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c103350 0x7f491c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 --2- 172.21.15.175:0/1808750490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c102a30 0x7f491c102e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 -- 172.21.15.175:0/1808750490 >> 172.21.15.175:0/1808750490 conn(0x7f491c0fc490 msgr2=0x7f491c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 -- 172.21.15.175:0/1808750490 shutdown_connections 2024-07-29T00:46:34.805 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.727+0000 7f4922c00640 1 -- 172.21.15.175:0/1808750490 wait complete. 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.728+0000 7f4922c00640 1 Processor -- start 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.728+0000 7f4922c00640 1 -- start start 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.728+0000 7f4922c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 0x7f491c100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4922c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f491c103350 0x7f491c100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4922c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c10be60 0x7f491c101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4922c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f491c111cb0 con 0x7f491c103350 2024-07-29T00:46:34.806 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4922c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f491c111b30 con 0x7f491c102a30 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4922c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f491c111e30 con 0x7f491c10be60 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 0x7f491c100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 0x7f491c100690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56234/0 (socket says 172.21.15.175:56234) 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 -- 172.21.15.175:0/2138543137 learned_addr learned my addr 172.21.15.175:0/2138543137 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4913e00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f491c103350 0x7f491c100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f4920c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c10be60 0x7f491c101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 -- 172.21.15.175:0/2138543137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c10be60 msgr2=0x7f491c101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c10be60 0x7f491c101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 -- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f491c103350 msgr2=0x7f491c100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f491c103350 0x7f491c100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.729+0000 7f491be00640 1 -- 172.21.15.175:0/2138543137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c1a65f0 con 0x7f491c102a30 2024-07-29T00:46:34.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.730+0000 7f491be00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 0x7f491c100690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f490c004770 tx=0x7f490c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:34.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.730+0000 7f4919600640 1 -- 172.21.15.175:0/2138543137 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c03f070 con 0x7f491c102a30 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.730+0000 7f4919600640 1 -- 172.21.15.175:0/2138543137 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c031ca0 con 0x7f491c102a30 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.730+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c1a6880 con 0x7f491c102a30 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.730+0000 7f4919600640 1 -- 172.21.15.175:0/2138543137 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c035030 con 0x7f491c102a30 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.730+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c1a6d60 con 0x7f491c102a30 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f4919600640 1 -- 172.21.15.175:0/2138543137 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f490c042070 con 0x7f491c102a30 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 Processor -- start 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 -- start start 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5cfef0 0x556fec5ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5d9720 0x556fec5dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556fec534ab0 con 0x556fec5d9720 2024-07-29T00:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556fec4cca80 con 0x556fec5ce980 2024-07-29T00:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.734+0000 7f492568c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556fec530a70 con 0x556fec5cfef0 2024-07-29T00:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56240/0 (socket says 172.21.15.175:56240) 2024-07-29T00:46:34.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 -- 172.21.15.175:0/3723635838 learned_addr learned my addr 172.21.15.175:0/3723635838 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:34.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f4913e00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5d9720 0x556fec5dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f4920c00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5cfef0 0x556fec5ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 -- 172.21.15.175:0/3723635838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5cfef0 msgr2=0x556fec5ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5cfef0 0x556fec5ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 -- 172.21.15.175:0/3723635838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5d9720 msgr2=0x556fec5dbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5d9720 0x556fec5dbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f491be00640 1 -- 172.21.15.175:0/3723635838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fec2a63c0 con 0x556fec5ce980 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f4920c00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5cfef0 0x556fec5ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.735+0000 7f4913e00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5d9720 0x556fec5dbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f4919600640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f49000a2cf0 0x7f49000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f491be00640 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5cedc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f490c0ecd50 tx=0x7f490c0f5cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d943deb888a1aa0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f4919600640 1 -- 172.21.15.175:0/2138543137 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f490c0ef770 con 0x7f491c102a30 2024-07-29T00:46:34.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f4920c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f49000a2cf0 0x7f49000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f4911200640 1 -- 172.21.15.175:0/3723635838 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b7ff0 con 0x556fec5ce980 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f4911200640 1 -- 172.21.15.175:0/3723635838 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c0bbd50 con 0x556fec5ce980 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.736+0000 7f4911200640 1 -- 172.21.15.175:0/3723635838 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c005d50 con 0x556fec5ce980 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 -- 172.21.15.175:0/3723635838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 msgr2=0x556fec5cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5cedc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f490c0ecd50 tx=0x7f490c0f5cb0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f4920c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f49000a2cf0 0x7f49000a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4914005fd0 tx=0x7f4914009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 -- 172.21.15.175:0/3723635838 shutdown_connections 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5d9720 0x556fec5dbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5cedc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 --2- 172.21.15.175:0/3723635838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5cfef0 0x556fec5ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 -- 172.21.15.175:0/3723635838 >> 172.21.15.175:0/3723635838 conn(0x556fec597950 msgr2=0x556fec5cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:34.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 -- 172.21.15.175:0/3723635838 shutdown_connections 2024-07-29T00:46:34.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.737+0000 7f492568c740 1 -- 172.21.15.175:0/3723635838 wait complete. 2024-07-29T00:46:34.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.738+0000 7f492568c740 1 -- start start 2024-07-29T00:46:34.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.738+0000 7f492568c740 1 Processor -- start 2024-07-29T00:46:34.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.738+0000 7f492568c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5ce980 0x556fec5d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.738+0000 7f492568c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 0x556fec5d6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f492568c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f492568c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556fec5dc1e0 con 0x556fec5ce980 2024-07-29T00:46:34.815 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f492568c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556fec4efc10 con 0x556fec5cfef0 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f492568c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556fec5dc360 con 0x556fec5d9720 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f492568c740 10 client.0 _ll_register_callbacks cb 0x556fec4b8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f491be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5ce980 0x556fec5d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 0x556fec5d6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 0x556fec5d6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56248/0 (socket says 172.21.15.175:56248) 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f491be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5ce980 0x556fec5d66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51060/0 (socket says 172.21.15.175:51060) 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 -- 172.21.15.175:0/3811531621 learned_addr learned my addr 172.21.15.175:0/3811531621 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4920c00640 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 -- 172.21.15.175:0/3811531621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 msgr2=0x556fec5d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 -- 172.21.15.175:0/3811531621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5ce980 msgr2=0x556fec5d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5ce980 0x556fec5d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.739+0000 7f4913e00640 1 -- 172.21.15.175:0/3811531621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fec4f31e0 con 0x556fec5cfef0 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f4913e00640 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 0x556fec5d6be0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4904012770 tx=0x7f4904012c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129dfb69b9523f64 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f4909600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f4909600640 1 -- 172.21.15.175:0/3811531621 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490400f8c0 con 0x556fec5cfef0 2024-07-29T00:46:34.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f4909600640 1 -- 172.21.15.175:0/3811531621 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f490400fa60 con 0x556fec5cfef0 2024-07-29T00:46:34.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c0bbd50 con 0x556fec5cfef0 2024-07-29T00:46:34.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f4909600640 1 -- 172.21.15.175:0/3811531621 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f490400fc60 con 0x556fec5cfef0 2024-07-29T00:46:34.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f492568c740 10 client.27374 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:34.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.740+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556fec5e8000 con 0x556fec5cfef0 2024-07-29T00:46:34.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f4909600640 1 -- 172.21.15.175:0/3811531621 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c0bbd50 con 0x556fec5cfef0 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f4909600640 1 -- 172.21.15.175:0/3811531621 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f490401c420 con 0x556fec5cfef0 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f492568c740 10 client.27374 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f492568c740 10 client.27374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f492568c740 20 client.27374 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f492568c740 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec5e94c0 0x556fec5eb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f492568c740 4 client.27374 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fec542020 con 0x556fec5e94c0 2024-07-29T00:46:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.741+0000 7f491be00640 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec5e94c0 0x556fec5eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.742+0000 7f491be00640 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec5e94c0 0x556fec5eb940 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b411a425141b2797 server_cookie=bf4210eb17c437f3 in_seq=0 out_seq=0 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.742+0000 7f4909600640 10 client.27374 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f4909600640 1 -- 172.21.15.175:0/3811531621 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556fec542020 con 0x556fec5e94c0 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f4909600640 10 client.27374 handle_command_reply: tid=0 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 client.27374 shutdown 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 msgr2=0x556fec5d6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 0x556fec5d6be0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4904012770 tx=0x7f4904012c30 comp rx=0 tx=0).stop 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 shutdown_connections 2024-07-29T00:46:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec5e94c0 0x556fec5eb940 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5cfef0 0x556fec5d6be0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f4909600640 0 client.27374 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 --2- 172.21.15.175:0/3811531621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5ce980 0x556fec5d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.743+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 >> 172.21.15.175:0/3811531621 conn(0x556fec597950 msgr2=0x556fec5d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.744+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 shutdown_connections 2024-07-29T00:46:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.744+0000 7f492568c740 1 -- 172.21.15.175:0/3811531621 wait complete. 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.744+0000 7f492568c740 20 client.27374 trim_cache size 0 max 16384 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.759+0000 7f492568c740 1 Processor -- start 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.759+0000 7f492568c740 1 -- start start 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f492568c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f492568c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f492568c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f492568c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1af8a0 con 0x556fec5cfef0 2024-07-29T00:46:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f492568c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1b13e0 con 0x556fec5ce980 2024-07-29T00:46:34.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f492568c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1aff70 con 0x556fec5d9720 2024-07-29T00:46:34.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5f1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56250/0 (socket says 172.21.15.175:56250) 2024-07-29T00:46:34.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 -- 172.21.15.175:0/4265351467 learned_addr learned my addr 172.21.15.175:0/4265351467 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:34.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f4920c00640 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f4913e00640 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 -- 172.21.15.175:0/4265351467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 msgr2=0x556fec5c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 -- 172.21.15.175:0/4265351467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 msgr2=0x556fec5f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.760+0000 7f491be00640 1 -- 172.21.15.175:0/4265351467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fec5d14d0 con 0x556fec5ce980 2024-07-29T00:46:34.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.761+0000 7f491be00640 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5f1210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f490c000c00 tx=0x7f490c02a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1316191dd3761af server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:34.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.761+0000 7f490be00640 1 -- 172.21.15.175:0/4265351467 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0d9070 con 0x556fec5ce980 2024-07-29T00:46:34.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.761+0000 7f490be00640 1 -- 172.21.15.175:0/4265351467 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c0c0050 con 0x556fec5ce980 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.761+0000 7f490be00640 1 -- 172.21.15.175:0/4265351467 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c049040 con 0x556fec5ce980 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 -- 172.21.15.175:0/4265351467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 msgr2=0x556fec5f1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5f1210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f490c000c00 tx=0x7f490c02a490 comp rx=0 tx=0).stop 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 -- 172.21.15.175:0/4265351467 shutdown_connections 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 --2- 172.21.15.175:0/4265351467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5f1210 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 -- 172.21.15.175:0/4265351467 >> 172.21.15.175:0/4265351467 conn(0x556fec597950 msgr2=0x556fec5d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 -- 172.21.15.175:0/4265351467 shutdown_connections 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.762+0000 7f492568c740 1 -- 172.21.15.175:0/4265351467 wait complete. 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 -- start start 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 Processor -- start 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5dd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1b0d10 con 0x556fec5cfef0 2024-07-29T00:46:34.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1b0640 con 0x556fec5ce980 2024-07-29T00:46:34.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.763+0000 7f492568c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1ad8a0 con 0x556fec5d9720 2024-07-29T00:46:34.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f492568c740 10 client.0 _ll_register_callbacks cb 0x7f4904009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:34.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f4913e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5e1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56252/0 (socket says 172.21.15.175:56252) 2024-07-29T00:46:34.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 -- 172.21.15.175:0/2594923430 learned_addr learned my addr 172.21.15.175:0/2594923430 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:34.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f4913e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5dd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51088/0 (socket says 172.21.15.175:51088) 2024-07-29T00:46:34.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f4920c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5dd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 -- 172.21.15.175:0/2594923430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 msgr2=0x556fec5dd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5dd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 -- 172.21.15.175:0/2594923430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 msgr2=0x556fec5dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:34.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 -- 172.21.15.175:0/2594923430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fec5de000 con 0x556fec5ce980 2024-07-29T00:46:34.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f491be00640 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5e1eb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f490c03b4a0 tx=0x7f490c031e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa87b0825a8fa057 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:34.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.764+0000 7f4909600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:34.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f4909600640 1 -- 172.21.15.175:0/2594923430 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c049070 con 0x556fec5ce980 2024-07-29T00:46:34.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f4909600640 1 -- 172.21.15.175:0/2594923430 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f490c0b9df0 con 0x556fec5ce980 2024-07-29T00:46:34.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c0c0050 con 0x556fec5ce980 2024-07-29T00:46:34.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f4909600640 1 -- 172.21.15.175:0/2594923430 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f490c0d9070 con 0x556fec5ce980 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 10 client.27380 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556fec5de000 con 0x556fec5ce980 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f4909600640 1 -- 172.21.15.175:0/2594923430 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c0c0050 con 0x556fec5ce980 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f4909600640 1 -- 172.21.15.175:0/2594923430 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f490c03b4f0 con 0x556fec5ce980 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 10 client.27380 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 10 client.27380 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:46:34.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 20 client.27380 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec6543b0 0x556fec6567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 4 client.27380 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.765+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fec53c530 con 0x556fec6543b0 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.766+0000 7f4913e00640 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec6543b0 0x556fec6567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.766+0000 7f4913e00640 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec6543b0 0x556fec6567a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6217aa6a770d2d80 server_cookie=5b64b099717afd37 in_seq=0 out_seq=0 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.766+0000 7f4909600640 10 client.27380 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:46:34.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.766+0000 7f4909600640 1 -- 172.21.15.175:0/2594923430 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fec53c530 con 0x556fec6543b0 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.766+0000 7f4909600640 10 client.27380 handle_command_reply: tid=0 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.766+0000 7f492568c740 1 client.27380 shutdown 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 msgr2=0x556fec5e1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5e1eb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f490c03b4a0 tx=0x7f490c031e50 comp rx=0 tx=0).stop 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 shutdown_connections 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fec5d9720 0x556fec5dd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fec5cfef0 0x556fec5dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x556fec6543b0 0x556fec6567a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 --2- 172.21.15.175:0/2594923430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fec5ce980 0x556fec5e1eb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 >> 172.21.15.175:0/2594923430 conn(0x556fec597950 msgr2=0x556fec5c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:34.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f4909600640 0 client.27380 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:46:34.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 shutdown_connections 2024-07-29T00:46:34.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 1 -- 172.21.15.175:0/2594923430 wait complete. 2024-07-29T00:46:34.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.767+0000 7f492568c740 20 client.27380 trim_cache size 0 max 16384 2024-07-29T00:46:34.837 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:46:34.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f49000a2cf0 msgr2=0x7f49000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f49000a2cf0 0x7f49000a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4914005fd0 tx=0x7f4914009500 comp rx=0 tx=0).stop 2024-07-29T00:46:34.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 msgr2=0x7f491c100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 0x7f491c100690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f490c004770 tx=0x7f490c003c10 comp rx=0 tx=0).stop 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 shutdown_connections 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491c10be60 0x7f491c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f49000a2cf0 0x7f49000a51b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f491c103350 0x7f491c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 --2- 172.21.15.175:0/2138543137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f491c102a30 0x7f491c100690 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 >> 172.21.15.175:0/2138543137 conn(0x7f491c0fc490 msgr2=0x7f491c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:34.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 shutdown_connections 2024-07-29T00:46:34.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:34.768+0000 7f4922c00640 1 -- 172.21.15.175:0/2138543137 wait complete. 2024-07-29T00:46:35.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:46:36.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:35 smithi175 ceph-mon[32138]: pgmap v313: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:36.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:35 smithi181 ceph-mon[36706]: pgmap v313: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:36.250 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:36.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:35 smithi204 ceph-mon[37368]: pgmap v313: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:36.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 -- 172.21.15.175:0/1079534250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd538108670 msgr2=0x7fd538114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 --2- 172.21.15.175:0/1079534250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd538108670 0x7fd538114b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd530009cd0 tx=0x7fd53002a8f0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 -- 172.21.15.175:0/1079534250 shutdown_connections 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 --2- 172.21.15.175:0/1079534250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd538108670 0x7fd538114b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 --2- 172.21.15.175:0/1079534250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538107d10 0x7fd538108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 --2- 172.21.15.175:0/1079534250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd53810dd50 0x7fd53810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.922+0000 7fd53ea00640 1 -- 172.21.15.175:0/1079534250 >> 172.21.15.175:0/1079534250 conn(0x7fd538077670 msgr2=0x7fd538079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.923+0000 7fd53ea00640 1 -- 172.21.15.175:0/1079534250 shutdown_connections 2024-07-29T00:46:36.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.923+0000 7fd53ea00640 1 -- 172.21.15.175:0/1079534250 wait complete. 2024-07-29T00:46:36.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 Processor -- start 2024-07-29T00:46:36.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 -- start start 2024-07-29T00:46:36.926 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 0x7fd53819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 0x7fd53819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd53810dd50 0x7fd5381a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381180b0 con 0x7fd53810dd50 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd538117f30 con 0x7fd538108670 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.924+0000 7fd53ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd538118230 con 0x7fd538107d10 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 0x7fd53819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 0x7fd53819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 0x7fd53819e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33848/0 (socket says 172.21.15.175:33848) 2024-07-29T00:46:36.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd53810dd50 0x7fd5381a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53d600640 1 -- 172.21.15.175:0/554462277 learned_addr learned my addr 172.21.15.175:0/554462277 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 0x7fd53819ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56268/0 (socket says 172.21.15.175:56268) 2024-07-29T00:46:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 -- 172.21.15.175:0/554462277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 msgr2=0x7fd53819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 0x7fd53819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 -- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd53810dd50 msgr2=0x7fd5381a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd53810dd50 0x7fd5381a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53cc00640 1 -- 172.21.15.175:0/554462277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5381a34a0 con 0x7fd538108670 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.925+0000 7fd53d600640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 0x7fd53819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.926+0000 7fd53cc00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 0x7fd53819ea30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd52400d6d0 tx=0x7fd52400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.926+0000 7fd536000640 1 -- 172.21.15.175:0/554462277 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524017070 con 0x7fd538108670 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.926+0000 7fd536000640 1 -- 172.21.15.175:0/554462277 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd524009820 con 0x7fd538108670 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.926+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5381a3790 con 0x7fd538108670 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.926+0000 7fd536000640 1 -- 172.21.15.175:0/554462277 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524004d00 con 0x7fd538108670 2024-07-29T00:46:36.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.926+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381ab070 con 0x7fd538108670 2024-07-29T00:46:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.929+0000 7fd536000640 1 -- 172.21.15.175:0/554462277 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd5240099d0 con 0x7fd538108670 2024-07-29T00:46:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.929+0000 7fd542bfc740 1 Processor -- start 2024-07-29T00:46:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 -- start start 2024-07-29T00:46:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 0x55c9526c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c952626ab0 con 0x55c9526cb910 2024-07-29T00:46:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c9525bea80 con 0x55c9526c20e0 2024-07-29T00:46:36.931 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd542bfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c952622a70 con 0x55c9526c0b70 2024-07-29T00:46:36.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd53d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 0x55c9526c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd53e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.930+0000 7fd53d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 0x55c9526c0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33850/0 (socket says 172.21.15.175:33850) 2024-07-29T00:46:36.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 -- 172.21.15.175:0/213532088 learned_addr learned my addr 172.21.15.175:0/213532088 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:36.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53cc00640 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 -- 172.21.15.175:0/213532088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 msgr2=0x55c9526c0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526c0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 -- 172.21.15.175:0/213532088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 msgr2=0x55c9526cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 -- 172.21.15.175:0/213532088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9523983c0 con 0x55c9526c0b70 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd536000640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5200a2cf0 0x7fd5200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53d600640 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 0x55c9526c0fb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd52c010580 tx=0x7fd52c010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b6e20a5e8ac683 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd536000640 1 -- 172.21.15.175:0/554462277 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fd5240cefd0 con 0x7fd538108670 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.931+0000 7fd53e000640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5200a2cf0 0x7fd5200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.932+0000 7fd52a600640 1 -- 172.21.15.175:0/213532088 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c011820 con 0x55c9526c0b70 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.932+0000 7fd52a600640 1 -- 172.21.15.175:0/213532088 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c01c480 con 0x55c9526c0b70 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.932+0000 7fd52a600640 1 -- 172.21.15.175:0/213532088 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00e670 con 0x55c9526c0b70 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.932+0000 7fd542bfc740 1 -- 172.21.15.175:0/213532088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 msgr2=0x55c9526c0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.932+0000 7fd53e000640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5200a2cf0 0x7fd5200a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd530004af0 tx=0x7fd530005d80 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.932+0000 7fd542bfc740 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 0x55c9526c0fb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd52c010580 tx=0x7fd52c010a40 comp rx=0 tx=0).stop 2024-07-29T00:46:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 -- 172.21.15.175:0/213532088 shutdown_connections 2024-07-29T00:46:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c0b70 0x55c9526c0fb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 --2- 172.21.15.175:0/213532088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526c0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 -- 172.21.15.175:0/213532088 >> 172.21.15.175:0/213532088 conn(0x55c952689950 msgr2=0x55c9526bd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 -- 172.21.15.175:0/213532088 shutdown_connections 2024-07-29T00:46:36.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 -- 172.21.15.175:0/213532088 wait complete. 2024-07-29T00:46:36.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.933+0000 7fd542bfc740 1 -- start start 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 Processor -- start 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526c8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 0x55c9526c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c9526ce6b0 con 0x55c9526c20e0 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c9525e1c10 con 0x55c9526c0b70 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c9526ce830 con 0x55c9526cb910 2024-07-29T00:46:36.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd542bfc740 10 client.0 _ll_register_callbacks cb 0x55c9525aaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.934+0000 7fd53cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 0x55c9526c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 0x55c9526c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51134/0 (socket says 172.21.15.175:51134) 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53cc00640 1 -- 172.21.15.175:0/3475887529 learned_addr learned my addr 172.21.15.175:0/3475887529 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53e000640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 -- 172.21.15.175:0/3475887529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 msgr2=0x55c9526c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 -- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 msgr2=0x55c9526c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 0x55c9526c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 -- 172.21.15.175:0/3475887529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9525e51e0 con 0x55c9526c0b70 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53cc00640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 0x55c9526c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd53d600640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526c8890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd52c010550 tx=0x7fd52c01c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c12363cc704f9391 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.935+0000 7fd51ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd51ea00640 1 -- 172.21.15.175:0/3475887529 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c02a060 con 0x55c9526c0b70 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd51ea00640 1 -- 172.21.15.175:0/3475887529 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fd52c005250 con 0x55c9526c0b70 2024-07-29T00:46:36.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c01c480 con 0x55c9526c0b70 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd51ea00640 1 -- 172.21.15.175:0/3475887529 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd52c00bd20 con 0x55c9526c0b70 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd542bfc740 10 client.27392 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c9526da420 con 0x55c9526c0b70 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd51ea00640 1 -- 172.21.15.175:0/3475887529 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c01c480 con 0x55c9526c0b70 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd51ea00640 1 -- 172.21.15.175:0/3475887529 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fd52c011620 con 0x55c9526c0b70 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd542bfc740 10 client.27392 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.936+0000 7fd542bfc740 10 client.27392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:46:36.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd542bfc740 20 client.27392 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd542bfc740 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9526db7d0 0x55c9526ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd542bfc740 4 client.27392 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c952634020 con 0x55c9526db7d0 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd53cc00640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9526db7d0 0x55c9526ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd53cc00640 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9526db7d0 0x55c9526ddbc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a66c45a5830c70c server_cookie=7e39076cab4ab63a in_seq=0 out_seq=0 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.937+0000 7fd51ea00640 10 client.27392 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd51ea00640 1 -- 172.21.15.175:0/3475887529 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c952634020 con 0x55c9526db7d0 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd51ea00640 10 client.27392 handle_command_reply: tid=0 2024-07-29T00:46:36.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 client.27392 shutdown 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 msgr2=0x55c9526c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526c8890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd52c010550 tx=0x7fd52c01c860 comp rx=0 tx=0).stop 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 shutdown_connections 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9526db7d0 0x55c9526ddbc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.939+0000 7fd51ea00640 0 client.27392 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.938+0000 7fd542bfc740 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c20e0 0x55c9526c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.939+0000 7fd542bfc740 1 --2- 172.21.15.175:0/3475887529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526c8890 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.939+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 >> 172.21.15.175:0/3475887529 conn(0x55c952689950 msgr2=0x55c9526c2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:36.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.939+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 shutdown_connections 2024-07-29T00:46:36.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.939+0000 7fd542bfc740 1 -- 172.21.15.175:0/3475887529 wait complete. 2024-07-29T00:46:36.946 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.939+0000 7fd542bfc740 20 client.27392 trim_cache size 0 max 16384 2024-07-29T00:46:36.953 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.953+0000 7fd542bfc740 1 Processor -- start 2024-07-29T00:46:36.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.953+0000 7fd542bfc740 1 -- start start 2024-07-29T00:46:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 0x55c9526c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526bc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd542bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381b3b80 con 0x55c9526c0b70 2024-07-29T00:46:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd542bfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381b56c0 con 0x55c9526c20e0 2024-07-29T00:46:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd542bfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381b4250 con 0x55c9526cb910 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 0x55c9526c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 0x55c9526c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51142/0 (socket says 172.21.15.175:51142) 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 -- 172.21.15.175:0/2983354438 learned_addr learned my addr 172.21.15.175:0/2983354438 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53cc00640 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526bc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53e000640 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 -- 172.21.15.175:0/2983354438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 msgr2=0x55c9526ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 -- 172.21.15.175:0/2983354438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 msgr2=0x55c9526bc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.954+0000 7fd53d600640 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526bc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.955+0000 7fd53d600640 1 -- 172.21.15.175:0/2983354438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9526bd430 con 0x55c9526c0b70 2024-07-29T00:46:36.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.955+0000 7fd53cc00640 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526bc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:36.958 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.955+0000 7fd53d600640 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 0x55c9526c8040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd52c010580 tx=0x7fd52c010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d60adfe3723859 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:36.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.955+0000 7fd529c00640 1 -- 172.21.15.175:0/2983354438 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c011d90 con 0x55c9526c0b70 2024-07-29T00:46:36.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.955+0000 7fd529c00640 1 -- 172.21.15.175:0/2983354438 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c00bc70 con 0x55c9526c0b70 2024-07-29T00:46:36.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.955+0000 7fd529c00640 1 -- 172.21.15.175:0/2983354438 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0035f0 con 0x55c9526c0b70 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 -- 172.21.15.175:0/2983354438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 msgr2=0x55c9526c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 0x55c9526c8040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd52c010580 tx=0x7fd52c010fc0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 -- 172.21.15.175:0/2983354438 shutdown_connections 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526cb910 0x55c9526ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c20e0 0x55c9526bc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 --2- 172.21.15.175:0/2983354438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526c0b70 0x55c9526c8040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 -- 172.21.15.175:0/2983354438 >> 172.21.15.175:0/2983354438 conn(0x55c952689950 msgr2=0x55c9526bbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 -- 172.21.15.175:0/2983354438 shutdown_connections 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 -- 172.21.15.175:0/2983354438 wait complete. 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.956+0000 7fd542bfc740 1 -- start start 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 Processor -- start 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 0x55c9526c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381b4ff0 con 0x55c9526cb910 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381b4920 con 0x55c9526c0b70 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381b1b80 con 0x55c9526c20e0 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd542bfc740 10 client.0 _ll_register_callbacks cb 0x7fd530009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.957+0000 7fd53cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 0x55c9526c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526d3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:56296/0 (socket says 172.21.15.175:56296) 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 0x55c9526c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:33886/0 (socket says 172.21.15.175:33886) 2024-07-29T00:46:36.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53d600640 1 -- 172.21.15.175:0/123985030 learned_addr learned my addr 172.21.15.175:0/123985030 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 -- 172.21.15.175:0/123985030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 msgr2=0x55c9526d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526d3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 -- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 msgr2=0x55c9526c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 -- 172.21.15.175:0/123985030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9526c36c0 con 0x55c9526c20e0 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53d600640 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd53cc00640 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 0x55c9526c82f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd524009610 tx=0x7fd524000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767eee5d34ac816 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:36.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd51ea00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.958+0000 7fd51ea00640 1 -- 172.21.15.175:0/123985030 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524020040 con 0x55c9526c20e0 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c00bc70 con 0x55c9526c20e0 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd51ea00640 1 -- 172.21.15.175:0/123985030 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fd5240b4310 con 0x55c9526c20e0 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd51ea00640 1 -- 172.21.15.175:0/123985030 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd52409b4f0 con 0x55c9526c20e0 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 10 client.27343 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c9526d25e0 con 0x55c9526c20e0 2024-07-29T00:46:36.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd51ea00640 1 -- 172.21.15.175:0/123985030 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c00bc70 con 0x55c9526c20e0 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd51ea00640 1 -- 172.21.15.175:0/123985030 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fd5240cc8f0 con 0x55c9526c20e0 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 10 client.27343 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 10 client.27343 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 20 client.27343 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9527463c0 0x55c9527487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 4 client.27343 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.959+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c95262e530 con 0x55c9527463c0 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.960+0000 7fd53d600640 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9527463c0 0x55c9527487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.960+0000 7fd53d600640 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9527463c0 0x55c9527487e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bb2d52c64c8b89c server_cookie=aadd4dd822d0c4e4 in_seq=0 out_seq=0 2024-07-29T00:46:36.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.960+0000 7fd51ea00640 10 client.27343 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd51ea00640 1 -- 172.21.15.175:0/123985030 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c95262e530 con 0x55c9527463c0 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd51ea00640 10 client.27343 handle_command_reply: tid=0 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 client.27343 shutdown 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 msgr2=0x55c9526c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 0x55c9526c82f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd524009610 tx=0x7fd524000bc0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 shutdown_connections 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x55c9527463c0 0x55c9527487e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c9526cb910 0x55c9526c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9526c20e0 0x55c9526c82f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 --2- 172.21.15.175:0/123985030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9526c0b70 0x55c9526d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd51ea00640 0 client.27343 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 >> 172.21.15.175:0/123985030 conn(0x55c952689950 msgr2=0x55c9526c2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 shutdown_connections 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 1 -- 172.21.15.175:0/123985030 wait complete. 2024-07-29T00:46:36.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.961+0000 7fd542bfc740 20 client.27343 trim_cache size 0 max 16384 2024-07-29T00:46:36.970 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5200a2cf0 msgr2=0x7fd5200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5200a2cf0 0x7fd5200a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd530004af0 tx=0x7fd530005d80 comp rx=0 tx=0).stop 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 msgr2=0x7fd53819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 0x7fd53819ea30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd52400d6d0 tx=0x7fd52400db90 comp rx=0 tx=0).stop 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 shutdown_connections 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd5200a2cf0 0x7fd5200a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd53810dd50 0x7fd5381a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd538108670 0x7fd53819ea30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 --2- 172.21.15.175:0/554462277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538107d10 0x7fd53819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:36.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 >> 172.21.15.175:0/554462277 conn(0x7fd538077670 msgr2=0x7fd538078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:36.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 shutdown_connections 2024-07-29T00:46:36.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:36.962+0000 7fd53ea00640 1 -- 172.21.15.175:0/554462277 wait complete. 2024-07-29T00:46:37.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:46:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:37 smithi181 ceph-mon[36706]: pgmap v314: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:38.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:37 smithi175 ceph-mon[32138]: pgmap v314: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:38.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:37 smithi204 ceph-mon[37368]: pgmap v314: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:38.367 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:39.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 -- 172.21.15.175:0/3041530331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 msgr2=0x7f15700746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 --2- 172.21.15.175:0/3041530331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f15700746f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1560009900 tx=0x7f156002a480 comp rx=0 tx=0).stop 2024-07-29T00:46:39.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 -- 172.21.15.175:0/3041530331 shutdown_connections 2024-07-29T00:46:39.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 --2- 172.21.15.175:0/3041530331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157010df00 0x7f1570110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 --2- 172.21.15.175:0/3041530331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 --2- 172.21.15.175:0/3041530331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f15700746f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 -- 172.21.15.175:0/3041530331 >> 172.21.15.175:0/3041530331 conn(0x7f15700fd400 msgr2=0x7f15700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:39.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 -- 172.21.15.175:0/3041530331 shutdown_connections 2024-07-29T00:46:39.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.001+0000 7f156fe00640 1 -- 172.21.15.175:0/3041530331 wait complete. 2024-07-29T00:46:39.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.002+0000 7f156fe00640 1 Processor -- start 2024-07-29T00:46:39.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.002+0000 7f156fe00640 1 -- start start 2024-07-29T00:46:39.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f157019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157010df00 0x7f157019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1570113cc0 con 0x7f1570074310 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1570113b40 con 0x7f157010df00 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1570113e40 con 0x7f1570074c30 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f157019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45134/0 (socket says 172.21.15.175:45134) 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156e000640 1 -- 172.21.15.175:0/3123311152 learned_addr learned my addr 172.21.15.175:0/3123311152 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:39.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.003+0000 7f156f400640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157010df00 0x7f157019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156e000640 1 -- 172.21.15.175:0/3123311152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157010df00 msgr2=0x7f157019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156e000640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157010df00 0x7f157019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156e000640 1 -- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 msgr2=0x7f157019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156e000640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f157019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156e000640 1 -- 172.21.15.175:0/3123311152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157019f1b0 con 0x7f1570074c30 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156ea00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f157019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156e000640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157019a740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f155c00ea60 tx=0x7f155c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f1567400640 1 -- 172.21.15.175:0/3123311152 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c00cde0 con 0x7f1570074c30 2024-07-29T00:46:39.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f1567400640 1 -- 172.21.15.175:0/3123311152 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f155c0035f0 con 0x7f1570074c30 2024-07-29T00:46:39.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157019f4a0 con 0x7f1570074c30 2024-07-29T00:46:39.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f1567400640 1 -- 172.21.15.175:0/3123311152 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c010670 con 0x7f1570074c30 2024-07-29T00:46:39.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.004+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15701a6d80 con 0x7f1570074c30 2024-07-29T00:46:39.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.007+0000 7f1567400640 1 -- 172.21.15.175:0/3123311152 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f155c0037a0 con 0x7f1570074c30 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 Processor -- start 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 -- start start 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d720630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d720fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d72dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f3d686ab0 con 0x558f3d720b70 2024-07-29T00:46:39.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558f3d61ea80 con 0x558f3d72b910 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.008+0000 7f157843a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3d682a70 con 0x558f3d7220e0 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d720fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d720630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d72dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d720630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45140/0 (socket says 172.21.15.175:45140) 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d720fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55180/0 (socket says 172.21.15.175:55180) 2024-07-29T00:46:39.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156f400640 1 -- 172.21.15.175:0/1493598377 learned_addr learned my addr 172.21.15.175:0/1493598377 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 -- 172.21.15.175:0/1493598377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 msgr2=0x558f3d720630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d720630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 -- 172.21.15.175:0/1493598377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 msgr2=0x558f3d72dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d72dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156ea00640 1 -- 172.21.15.175:0/1493598377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3d3f83c0 con 0x558f3d720b70 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f156f400640 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d720630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:46:39.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.009+0000 7f1567400640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f15540a2cf0 0x7f15540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.010+0000 7f1567400640 1 -- 172.21.15.175:0/3123311152 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f155c0cf770 con 0x7f1570074c30 2024-07-29T00:46:39.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.010+0000 7f156ea00640 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d720fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f156002aa70 tx=0x7f1560005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d67c51199eb4c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:39.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.010+0000 7f156f400640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f15540a2cf0 0x7f15540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.010+0000 7f155be00640 1 -- 172.21.15.175:0/1493598377 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560035040 con 0x558f3d720b70 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.010+0000 7f155be00640 1 -- 172.21.15.175:0/1493598377 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560004760 con 0x558f3d720b70 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.010+0000 7f155be00640 1 -- 172.21.15.175:0/1493598377 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15600038e0 con 0x558f3d720b70 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 -- 172.21.15.175:0/1493598377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 msgr2=0x558f3d720fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d720fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f156002aa70 tx=0x7f1560005de0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f156f400640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f15540a2cf0 0x7f15540a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1568012640 tx=0x7f1568009500 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 -- 172.21.15.175:0/1493598377 shutdown_connections 2024-07-29T00:46:39.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d72dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d720fb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 --2- 172.21.15.175:0/1493598377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d720630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 -- 172.21.15.175:0/1493598377 >> 172.21.15.175:0/1493598377 conn(0x558f3d6e9950 msgr2=0x558f3d71d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 -- 172.21.15.175:0/1493598377 shutdown_connections 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.011+0000 7f157843a740 1 -- 172.21.15.175:0/1493598377 wait complete. 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 -- start start 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 Processor -- start 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d720b70 0x558f3d728910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d728e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d729390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f3d72e650 con 0x558f3d7220e0 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558f3d641c10 con 0x558f3d72b910 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.012+0000 7f157843a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3d72e7d0 con 0x558f3d720b70 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f157843a740 10 client.0 _ll_register_callbacks cb 0x558f3d60aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d729390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d728e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.018 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d729390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:32960/0 (socket says 172.21.15.175:32960) 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 -- 172.21.15.175:0/2324899745 learned_addr learned my addr 172.21.15.175:0/2324899745 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156ea00640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d720b70 0x558f3d728910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 -- 172.21.15.175:0/2324899745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d720b70 msgr2=0x558f3d728910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d720b70 0x558f3d728910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 -- 172.21.15.175:0/2324899745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 msgr2=0x558f3d728e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d728e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156f400640 1 -- 172.21.15.175:0/2324899745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3d6451e0 con 0x558f3d72b910 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.013+0000 7f156e000640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d728e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:39.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f156f400640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d729390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1568003f50 tx=0x7f1568004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eff8882b8b9e6e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f1558c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f1558c00640 1 -- 172.21.15.175:0/2324899745 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568004b20 con 0x558f3d72b910 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1560004760 con 0x558f3d72b910 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f1558c00640 1 -- 172.21.15.175:0/2324899745 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f1568004cc0 con 0x558f3d72b910 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f1558c00640 1 -- 172.21.15.175:0/2324899745 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f156800f830 con 0x558f3d72b910 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f157843a740 10 client.27410 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.014+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558f3d73a510 con 0x558f3d72b910 2024-07-29T00:46:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f1558c00640 1 -- 172.21.15.175:0/2324899745 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560004760 con 0x558f3d72b910 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f1558c00640 1 -- 172.21.15.175:0/2324899745 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1568027030 con 0x558f3d72b910 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f157843a740 10 client.27410 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f157843a740 10 client.27410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f157843a740 20 client.27410 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f157843a740 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d73b920 0x558f3d73dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f157843a740 4 client.27410 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.015+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3d694020 con 0x558f3d73b920 2024-07-29T00:46:39.022 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.016+0000 7f156ea00640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d73b920 0x558f3d73dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.016+0000 7f156ea00640 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d73b920 0x558f3d73dd30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638cae49c3dbac1a server_cookie=7e3815bf2f591192 in_seq=0 out_seq=0 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.016+0000 7f1558c00640 10 client.27410 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f1558c00640 1 -- 172.21.15.175:0/2324899745 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f3d694020 con 0x558f3d73b920 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f1558c00640 10 client.27410 handle_command_reply: tid=0 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 client.27410 shutdown 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 msgr2=0x558f3d729390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d729390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1568003f50 tx=0x7f1568004ab0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 shutdown_connections 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d73b920 0x558f3d73dd30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d729390 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d728e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 --2- 172.21.15.175:0/2324899745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d720b70 0x558f3d728910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f1558c00640 0 client.27410 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.017+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 >> 172.21.15.175:0/2324899745 conn(0x558f3d6e9950 msgr2=0x558f3d722f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.018+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 shutdown_connections 2024-07-29T00:46:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.018+0000 7f157843a740 1 -- 172.21.15.175:0/2324899745 wait complete. 2024-07-29T00:46:39.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.018+0000 7f157843a740 20 client.27410 trim_cache size 0 max 16384 2024-07-29T00:46:39.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 Processor -- start 2024-07-29T00:46:39.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 -- start start 2024-07-29T00:46:39.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 0x558f3d71d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d72cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 0x558f3d72d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15701af7f0 con 0x558f3d7220e0 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f15701b1330 con 0x558f3d720b70 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.033+0000 7f157843a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15701afec0 con 0x558f3d72b910 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 0x558f3d71d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 0x558f3d72d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 0x558f3d71d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:32966/0 (socket says 172.21.15.175:32966) 2024-07-29T00:46:39.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 -- 172.21.15.175:0/769322400 learned_addr learned my addr 172.21.15.175:0/769322400 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156e000640 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d72cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 0x558f3d72d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45168/0 (socket says 172.21.15.175:45168) 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 -- 172.21.15.175:0/769322400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 msgr2=0x558f3d72d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 0x558f3d72d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 -- 172.21.15.175:0/769322400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 msgr2=0x558f3d72cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d72cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 -- 172.21.15.175:0/769322400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3d726ea0 con 0x558f3d720b70 2024-07-29T00:46:39.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156f400640 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 0x558f3d72d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:46:39.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.034+0000 7f156ea00640 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 0x558f3d71d3b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x558f3d71e060 tx=0x7f1560031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de8fe3bbf744cce server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:39.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.035+0000 7f155b400640 1 -- 172.21.15.175:0/769322400 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560035050 con 0x558f3d720b70 2024-07-29T00:46:39.039 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.035+0000 7f155b400640 1 -- 172.21.15.175:0/769322400 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560003de0 con 0x558f3d720b70 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.035+0000 7f155b400640 1 -- 172.21.15.175:0/769322400 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156004f6b0 con 0x558f3d720b70 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 -- 172.21.15.175:0/769322400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 msgr2=0x558f3d71d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 0x558f3d71d3b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x558f3d71e060 tx=0x7f1560031510 comp rx=0 tx=0).stop 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 -- 172.21.15.175:0/769322400 shutdown_connections 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d72b910 0x558f3d72d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d7220e0 0x558f3d72cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 --2- 172.21.15.175:0/769322400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d720b70 0x558f3d71d3b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 -- 172.21.15.175:0/769322400 >> 172.21.15.175:0/769322400 conn(0x558f3d6e9950 msgr2=0x558f3d71cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 -- 172.21.15.175:0/769322400 shutdown_connections 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 -- 172.21.15.175:0/769322400 wait complete. 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.036+0000 7f157843a740 1 -- start start 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 Processor -- start 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d733d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d7342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d725cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15701b0c60 con 0x558f3d720b70 2024-07-29T00:46:39.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f15701b0590 con 0x558f3d72b910 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15701ad7f0 con 0x558f3d7220e0 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f157843a740 10 client.0 _ll_register_callbacks cb 0x7f1568009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f156ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d733d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f156ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d733d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55212/0 (socket says 172.21.15.175:55212) 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f156e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d7342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f156ea00640 1 -- 172.21.15.175:0/313166724 learned_addr learned my addr 172.21.15.175:0/313166724 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.037+0000 7f156f400640 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d725cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f156e000640 1 -- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 msgr2=0x558f3d725cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f156e000640 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d725cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f156e000640 1 -- 172.21.15.175:0/313166724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 msgr2=0x558f3d733d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f156e000640 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d733d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f156e000640 1 -- 172.21.15.175:0/313166724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3d7264f0 con 0x558f3d7220e0 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f156e000640 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d7342c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f155c0cce10 tx=0x7f155c09b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b7cdef6986837e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f1558c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f1558c00640 1 -- 172.21.15.175:0/313166724 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c0cd290 con 0x558f3d7220e0 2024-07-29T00:46:39.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f1558c00640 1 -- 172.21.15.175:0/313166724 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f155c0cfb60 con 0x558f3d7220e0 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.038+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1560003de0 con 0x558f3d7220e0 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f1558c00640 1 -- 172.21.15.175:0/313166724 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f155c09c2e0 con 0x558f3d7220e0 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 10 client.27361 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558f3d743800 con 0x558f3d7220e0 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f1558c00640 1 -- 172.21.15.175:0/313166724 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1560003de0 con 0x558f3d7220e0 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f1558c00640 1 -- 172.21.15.175:0/313166724 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f155c09b150 con 0x558f3d7220e0 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 10 client.27361 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:39.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 10 client.27361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:39.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 20 client.27361 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:39.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d7af180 0x558f3d7b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:39.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 4 client.27361 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:46:39.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.039+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f3d68e530 con 0x558f3d7af180 2024-07-29T00:46:39.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.040+0000 7f156ea00640 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d7af180 0x558f3d7b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:39.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.040+0000 7f156ea00640 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d7af180 0x558f3d7b15a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f14d84ce6b7b7db server_cookie=d935f85de7b55477 in_seq=0 out_seq=0 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.040+0000 7f1558c00640 10 client.27361 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.040+0000 7f1558c00640 1 -- 172.21.15.175:0/313166724 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f3d68e530 con 0x558f3d7af180 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.040+0000 7f1558c00640 10 client.27361 handle_command_reply: tid=0 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.040+0000 7f157843a740 1 client.27361 shutdown 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 msgr2=0x558f3d7342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d7342c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f155c0cce10 tx=0x7f155c09b720 comp rx=0 tx=0).stop 2024-07-29T00:46:39.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 shutdown_connections 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x558f3d7af180 0x558f3d7b15a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f3d72b910 0x558f3d725cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3d7220e0 0x558f3d7342c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 --2- 172.21.15.175:0/313166724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f3d720b70 0x558f3d733d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 >> 172.21.15.175:0/313166724 conn(0x558f3d6e9950 msgr2=0x558f3d72a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f1558c00640 0 client.27361 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 shutdown_connections 2024-07-29T00:46:39.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 1 -- 172.21.15.175:0/313166724 wait complete. 2024-07-29T00:46:39.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.041+0000 7f157843a740 20 client.27361 trim_cache size 0 max 16384 2024-07-29T00:46:39.049 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:46:39.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f15540a2cf0 msgr2=0x7f15540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f15540a2cf0 0x7f15540a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1568012640 tx=0x7f1568009500 comp rx=0 tx=0).stop 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 msgr2=0x7f157019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157019a740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f155c00ea60 tx=0x7f155c00ef20 comp rx=0 tx=0).stop 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 shutdown_connections 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157010df00 0x7f157019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1570074c30 0x7f157019a740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f15540a2cf0 0x7f15540a51b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 --2- 172.21.15.175:0/3123311152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570074310 0x7f157019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:39.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 >> 172.21.15.175:0/3123311152 conn(0x7f15700fd400 msgr2=0x7f15700fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:39.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 shutdown_connections 2024-07-29T00:46:39.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:39.042+0000 7f156fe00640 1 -- 172.21.15.175:0/3123311152 wait complete. 2024-07-29T00:46:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:39 smithi181 ceph-mon[36706]: pgmap v315: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:40.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:39 smithi175 ceph-mon[32138]: pgmap v315: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:40.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:39 smithi204 ceph-mon[37368]: pgmap v315: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:41.013 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-29T00:46:41.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-29T00:46:41.478 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:42.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 -- 172.21.15.175:0/72231195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc103a90 msgr2=0x7febfc103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 --2- 172.21.15.175:0/72231195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc103a90 0x7febfc103ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7febe8009890 tx=0x7febe802a360 comp rx=0 tx=0).stop 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 -- 172.21.15.175:0/72231195 shutdown_connections 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 --2- 172.21.15.175:0/72231195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc104430 0x7febfc110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 --2- 172.21.15.175:0/72231195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc103a90 0x7febfc103ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 --2- 172.21.15.175:0/72231195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc109a90 0x7febfc109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 -- 172.21.15.175:0/72231195 >> 172.21.15.175:0/72231195 conn(0x7febfc0ff710 msgr2=0x7febfc101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:42.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 -- 172.21.15.175:0/72231195 shutdown_connections 2024-07-29T00:46:42.148 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.145+0000 7fec03c00640 1 -- 172.21.15.175:0/72231195 wait complete. 2024-07-29T00:46:42.148 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:41 smithi175 ceph-mon[32138]: pgmap v316: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:42.149 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.146+0000 7fec03c00640 1 Processor -- start 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.146+0000 7fec03c00640 1 -- start start 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec03c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc103a90 0x7febfc19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec03c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc104430 0x7febfc19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec03c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc109a90 0x7febfc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec03c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febfc113e90 con 0x7febfc103a90 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec03c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febfc113d10 con 0x7febfc109a90 2024-07-29T00:46:42.150 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec03c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febfc114010 con 0x7febfc104430 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec01200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc103a90 0x7febfc19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec01200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc103a90 0x7febfc19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55232/0 (socket says 172.21.15.175:55232) 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7febfbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc104430 0x7febfc19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec01200640 1 -- 172.21.15.175:0/1272342534 learned_addr learned my addr 172.21.15.175:0/1272342534 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.147+0000 7fec01c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc109a90 0x7febfc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febfbe00640 1 -- 172.21.15.175:0/1272342534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc109a90 msgr2=0x7febfc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febfbe00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc109a90 0x7febfc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febfbe00640 1 -- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc103a90 msgr2=0x7febfc19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.151 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febfbe00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc103a90 0x7febfc19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febfbe00640 1 -- 172.21.15.175:0/1272342534 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc19f300 con 0x7febfc104430 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febfbe00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc104430 0x7febfc19a890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7febe8035040 tx=0x7febe8033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febf9600640 1 -- 172.21.15.175:0/1272342534 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8037040 con 0x7febfc104430 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febf9600640 1 -- 172.21.15.175:0/1272342534 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe8033d90 con 0x7febfc104430 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7febf9600640 1 -- 172.21.15.175:0/1272342534 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8003830 con 0x7febfc104430 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.148+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc19f590 con 0x7febfc104430 2024-07-29T00:46:42.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.149+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc1a6ed0 con 0x7febfc104430 2024-07-29T00:46:42.154 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 Processor -- start 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 -- start start 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7febf9600640 1 -- 172.21.15.175:0/1272342534 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7febe8042030 con 0x7febfc104430 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e92b40 0x555a62e92f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 0x555a62e9d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e9dd20 0x555a62ea0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a62df88a0 con 0x555a62e92b40 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555a62d90710 con 0x555a62e93460 2024-07-29T00:46:42.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.152+0000 7fec0680c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a62df4860 con 0x555a62e9dd20 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 0x555a62e9d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e92b40 0x555a62e92f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 0x555a62e9d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:33016/0 (socket says 172.21.15.175:33016) 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 -- 172.21.15.175:0/3427506587 learned_addr learned my addr 172.21.15.175:0/3427506587 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7febfbe00640 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e9dd20 0x555a62ea0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 -- 172.21.15.175:0/3427506587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e9dd20 msgr2=0x555a62ea0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e9dd20 0x555a62ea0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 -- 172.21.15.175:0/3427506587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e92b40 msgr2=0x555a62e92f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e92b40 0x555a62e92f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7fec01200640 1 -- 172.21.15.175:0/3427506587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a62b6a6a0 con 0x555a62e93460 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.153+0000 7febfbe00640 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e9dd20 0x555a62ea0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.154+0000 7fec01200640 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 0x555a62e9d7e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7febf0011510 tx=0x7febf00119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e70e3e791430089 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.154+0000 7febee000640 1 -- 172.21.15.175:0/3427506587 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf001e070 con 0x555a62e93460 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.154+0000 7febee000640 1 -- 172.21.15.175:0/3427506587 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf000faa0 con 0x555a62e93460 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.154+0000 7febee000640 1 -- 172.21.15.175:0/3427506587 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0019c40 con 0x555a62e93460 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.154+0000 7febf9600640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7febe40a2d70 0x7febe40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.154+0000 7febf9600640 1 -- 172.21.15.175:0/1272342534 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7febe80ee710 con 0x7febfc104430 2024-07-29T00:46:42.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec01c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7febe40a2d70 0x7febe40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 -- 172.21.15.175:0/3427506587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 msgr2=0x555a62e9d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 0x555a62e9d7e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7febf0011510 tx=0x7febf00119d0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 -- 172.21.15.175:0/3427506587 shutdown_connections 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e9dd20 0x555a62ea0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e93460 0x555a62e9d7e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 --2- 172.21.15.175:0/3427506587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e92b40 0x555a62e92f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 -- 172.21.15.175:0/3427506587 >> 172.21.15.175:0/3427506587 conn(0x555a62e8f0d0 msgr2=0x555a62e8f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 -- 172.21.15.175:0/3427506587 shutdown_connections 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec0680c740 1 -- 172.21.15.175:0/3427506587 wait complete. 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.155+0000 7fec01c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7febe40a2d70 0x7febe40a5230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7febf4009530 tx=0x7febf4009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.156+0000 7fec0680c740 1 -- start start 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.156+0000 7fec0680c740 1 Processor -- start 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e92b40 0x555a62e9ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e93460 0x555a62e9b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.162 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 0x555a62e9b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a62ea08f0 con 0x555a62e9dd20 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555a62db38c0 con 0x555a62e92b40 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555a62ea0a70 con 0x555a62e93460 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec0680c740 10 client.0 _ll_register_callbacks cb 0x555a62d9fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 0x555a62e9b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e92b40 0x555a62e9ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 0x555a62e9b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55256/0 (socket says 172.21.15.175:55256) 2024-07-29T00:46:42.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01c00640 1 -- 172.21.15.175:0/2090172457 learned_addr learned my addr 172.21.15.175:0/2090172457 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7febfbe00640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e93460 0x555a62e9b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01200640 1 -- 172.21.15.175:0/2090172457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e93460 msgr2=0x555a62e9b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01200640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e93460 0x555a62e9b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01200640 1 -- 172.21.15.175:0/2090172457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 msgr2=0x555a62e9b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01200640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 0x555a62e9b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01200640 1 -- 172.21.15.175:0/2090172457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a62d96bc0 con 0x555a62e92b40 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.157+0000 7fec01c00640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 0x555a62e9b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.158+0000 7fec01200640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e92b40 0x555a62e9ab70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7febf0019500 tx=0x7febf000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae71135c499aae5d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.158+0000 7febe2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.158+0000 7febe2a00640 1 -- 172.21.15.175:0/2090172457 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf001e040 con 0x555a62e92b40 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.158+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf000faa0 con 0x555a62e92b40 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.158+0000 7febe2a00640 1 -- 172.21.15.175:0/2090172457 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7febf0004a40 con 0x555a62e92b40 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7febe2a00640 1 -- 172.21.15.175:0/2090172457 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7febf0004c40 con 0x555a62e92b40 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 10 client.27428 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555a62eac880 con 0x555a62e92b40 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7febe2a00640 1 -- 172.21.15.175:0/2090172457 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf000faa0 con 0x555a62e92b40 2024-07-29T00:46:42.165 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7febe2a00640 1 -- 172.21.15.175:0/2090172457 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7febf0021070 con 0x555a62e92b40 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 10 client.27428 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 20 client.27428 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62eadc90 0x555a62eb0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 4 client.27428 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.159+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a62db8c90 con 0x555a62eadc90 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.160+0000 7febfbe00640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62eadc90 0x555a62eb0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.160+0000 7febfbe00640 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62eadc90 0x555a62eb0080 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1f8924ce4f6979 server_cookie=b4873aaae21f3ccc in_seq=0 out_seq=0 2024-07-29T00:46:42.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.160+0000 7febe2a00640 10 client.27428 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:42.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.161+0000 7febe2a00640 1 -- 172.21.15.175:0/2090172457 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a62db8c90 con 0x555a62eadc90 2024-07-29T00:46:42.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.161+0000 7febe2a00640 10 client.27428 handle_command_reply: tid=0 2024-07-29T00:46:42.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.161+0000 7fec0680c740 1 client.27428 shutdown 2024-07-29T00:46:42.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e92b40 msgr2=0x555a62e9ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e92b40 0x555a62e9ab70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7febf0019500 tx=0x7febf000f7f0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 shutdown_connections 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62eadc90 0x555a62eb0080 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e9dd20 0x555a62e9b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e93460 0x555a62e9b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 --2- 172.21.15.175:0/2090172457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e92b40 0x555a62e9ab70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7febe2a00640 0 client.27428 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 >> 172.21.15.175:0/2090172457 conn(0x555a62e8f0d0 msgr2=0x555a62e94e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 shutdown_connections 2024-07-29T00:46:42.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 1 -- 172.21.15.175:0/2090172457 wait complete. 2024-07-29T00:46:42.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.162+0000 7fec0680c740 20 client.27428 trim_cache size 0 max 16384 2024-07-29T00:46:42.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 Processor -- start 2024-07-29T00:46:42.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 -- start start 2024-07-29T00:46:42.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62e99df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 0x555a62eb5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62eb5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1af950 con 0x555a62e93460 2024-07-29T00:46:42.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1b1490 con 0x555a62e9dd20 2024-07-29T00:46:42.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.181+0000 7fec0680c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1b0020 con 0x555a62e92b40 2024-07-29T00:46:42.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 0x555a62eb5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 0x555a62eb5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55268/0 (socket says 172.21.15.175:55268) 2024-07-29T00:46:42.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 -- 172.21.15.175:0/1176505942 learned_addr learned my addr 172.21.15.175:0/1176505942 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:42.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7fec01200640 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62e99df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7fec01c00640 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62eb5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 -- 172.21.15.175:0/1176505942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 msgr2=0x555a62e99df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62e99df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 -- 172.21.15.175:0/1176505942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 msgr2=0x555a62eb5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62eb5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7febfbe00640 1 -- 172.21.15.175:0/1176505942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a62eb03f0 con 0x555a62e93460 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.182+0000 7fec01200640 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62e99df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.183+0000 7febfbe00640 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 0x555a62eb5860 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7febe80b7180 tx=0x7febe8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdcce4dd36954b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:42.186 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.183+0000 7febed600640 1 -- 172.21.15.175:0/1176505942 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe80c3900 con 0x555a62e93460 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.183+0000 7febed600640 1 -- 172.21.15.175:0/1176505942 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe80b9020 con 0x555a62e93460 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.183+0000 7febed600640 1 -- 172.21.15.175:0/1176505942 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe80c3470 con 0x555a62e93460 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 -- 172.21.15.175:0/1176505942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 msgr2=0x555a62eb5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 0x555a62eb5860 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7febe80b7180 tx=0x7febe8005f90 comp rx=0 tx=0).stop 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 -- 172.21.15.175:0/1176505942 shutdown_connections 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62eb5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e93460 0x555a62eb5860 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 --2- 172.21.15.175:0/1176505942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62e99df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 -- 172.21.15.175:0/1176505942 >> 172.21.15.175:0/1176505942 conn(0x555a62e8f0d0 msgr2=0x555a62e9cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 -- 172.21.15.175:0/1176505942 shutdown_connections 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 -- 172.21.15.175:0/1176505942 wait complete. 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 -- start start 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.184+0000 7fec0680c740 1 Processor -- start 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62ea60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62ea65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 0x555a62e97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1b0dc0 con 0x555a62e97620 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1b06f0 con 0x555a62e9dd20 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1ad950 con 0x555a62e92b40 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec0680c740 10 client.0 _ll_register_callbacks cb 0x7febf4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec01c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 0x555a62e97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec01c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 0x555a62e97aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55278/0 (socket says 172.21.15.175:55278) 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec01c00640 1 -- 172.21.15.175:0/409459456 learned_addr learned my addr 172.21.15.175:0/409459456 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7febfbe00640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62ea65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.185+0000 7fec01200640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62ea60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01c00640 1 -- 172.21.15.175:0/409459456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 msgr2=0x555a62ea60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01c00640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62ea60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01c00640 1 -- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 msgr2=0x555a62ea65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01c00640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62ea65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01c00640 1 -- 172.21.15.175:0/409459456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a62e982d0 con 0x555a62e97620 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01200640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62ea60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7febfbe00640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62ea65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7fec01c00640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 0x555a62e97aa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7febf4009be0 tx=0x7febf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c737c8f721f9d93c server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7febe2a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.186+0000 7febe2a00640 1 -- 172.21.15.175:0/409459456 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf402f060 con 0x555a62e97620 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe80b9020 con 0x555a62e97620 2024-07-29T00:46:42.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7febe2a00640 1 -- 172.21.15.175:0/409459456 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7febf401fb90 con 0x555a62e97620 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7febe2a00640 1 -- 172.21.15.175:0/409459456 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7febf401fdd0 con 0x555a62e97620 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7fec0680c740 10 client.17283 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555a62f189a0 con 0x555a62e97620 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7febe2a00640 1 -- 172.21.15.175:0/409459456 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe80b9020 con 0x555a62e97620 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7febe2a00640 1 -- 172.21.15.175:0/409459456 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7febf4032400 con 0x555a62e97620 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7fec0680c740 10 client.17283 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7fec0680c740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.187+0000 7fec0680c740 20 client.17283 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.188+0000 7fec0680c740 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62f19150 0x555a62f1b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.188+0000 7fec0680c740 4 client.17283 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.188+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a62e05e10 con 0x555a62f19150 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.188+0000 7fec01200640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62f19150 0x555a62f1b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.189+0000 7fec01200640 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62f19150 0x555a62f1b570 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303fadd9885f4229 server_cookie=fd8fcb7160afb989 in_seq=0 out_seq=0 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.189+0000 7febe2a00640 10 client.17283 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7febe2a00640 1 -- 172.21.15.175:0/409459456 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a62e05e10 con 0x555a62f19150 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7febe2a00640 10 client.17283 handle_command_reply: tid=0 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 client.17283 shutdown 2024-07-29T00:46:42.195 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 msgr2=0x555a62e97aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 0x555a62e97aa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7febf4009be0 tx=0x7febf40046c0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 shutdown_connections 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x555a62f19150 0x555a62f1b570 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a62e97620 0x555a62e97aa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555a62e9dd20 0x555a62ea65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 --2- 172.21.15.175:0/409459456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a62e92b40 0x555a62ea60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 >> 172.21.15.175:0/409459456 conn(0x555a62e8f0d0 msgr2=0x555a62e8da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7febe2a00640 0 client.17283 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:42.196 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 shutdown_connections 2024-07-29T00:46:42.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 1 -- 172.21.15.175:0/409459456 wait complete. 2024-07-29T00:46:42.197 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.190+0000 7fec0680c740 20 client.17283 trim_cache size 0 max 16384 2024-07-29T00:46:42.197 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:46:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "return_code": 0, 2024-07-29T00:46:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_tag": "a2888099-4fbb-4bd1-81c4-6fc3e864d7b5", 2024-07-29T00:46:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": "asynchronous" 2024-07-29T00:46:42.198 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:46:42.198 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7febe40a2d70 msgr2=0x7febe40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7febe40a2d70 0x7febe40a5230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7febf4009530 tx=0x7febf4009260 comp rx=0 tx=0).stop 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc104430 msgr2=0x7febfc19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc104430 0x7febfc19a890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7febe8035040 tx=0x7febe8033ae0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 shutdown_connections 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc109a90 0x7febfc19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febfc104430 0x7febfc19a890 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7febe40a2d70 0x7febe40a5230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 --2- 172.21.15.175:0/1272342534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febfc103a90 0x7febfc19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:42.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 >> 172.21.15.175:0/1272342534 conn(0x7febfc0ff710 msgr2=0x7febfc100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:42.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.191+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 shutdown_connections 2024-07-29T00:46:42.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:42.192+0000 7fec03c00640 1 -- 172.21.15.175:0/1272342534 wait complete. 2024-07-29T00:46:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:41 smithi181 ceph-mon[36706]: pgmap v316: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:41 smithi204 ceph-mon[37368]: pgmap v316: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:43.002 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub queued for path: / 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.003 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:42 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:46:43.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub queued for path: / 2024-07-29T00:46:43.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:42 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub queued for path: / 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:42 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.546 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:46:43.954 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.954 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.954 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.954 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.955 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: scrub summary: active paths [/] 2024-07-29T00:46:43.955 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:43.955 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:43 smithi175 ceph-mon[32138]: pgmap v317: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:44.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 -- 172.21.15.175:0/2500399617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c108670 msgr2=0x7f196c114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 --2- 172.21.15.175:0/2500399617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c108670 0x7f196c114b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1964009cd0 tx=0x7f196402a8f0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 -- 172.21.15.175:0/2500399617 shutdown_connections 2024-07-29T00:46:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 --2- 172.21.15.175:0/2500399617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c108670 0x7f196c114b90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 --2- 172.21.15.175:0/2500399617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c107d10 0x7f196c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 --2- 172.21.15.175:0/2500399617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c10dd50 0x7f196c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 -- 172.21.15.175:0/2500399617 >> 172.21.15.175:0/2500399617 conn(0x7f196c077670 msgr2=0x7f196c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 -- 172.21.15.175:0/2500399617 shutdown_connections 2024-07-29T00:46:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.217+0000 7f1972800640 1 -- 172.21.15.175:0/2500399617 wait complete. 2024-07-29T00:46:44.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.218+0000 7f1972800640 1 Processor -- start 2024-07-29T00:46:44.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 -- start start 2024-07-29T00:46:44.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 0x7f196c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c108670 0x7f196c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c10dd50 0x7f196c1a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1180b0 con 0x7f196c10dd50 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f196c117f30 con 0x7f196c108670 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1972800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f196c118230 con 0x7f196c107d10 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1971400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 0x7f196c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.219+0000 7f1971400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 0x7f196c19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45230/0 (socket says 172.21.15.175:45230) 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 -- 172.21.15.175:0/4097321797 learned_addr learned my addr 172.21.15.175:0/4097321797 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1970a00640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c108670 0x7f196c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971e00640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c10dd50 0x7f196c1a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 -- 172.21.15.175:0/4097321797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c108670 msgr2=0x7f196c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c108670 0x7f196c19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 -- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c10dd50 msgr2=0x7f196c1a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c10dd50 0x7f196c1a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 -- 172.21.15.175:0/4097321797 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c1a34a0 con 0x7f196c107d10 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.220+0000 7f1971400640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 0x7f196c19e4f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9f0 tx=0x7f196000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:44.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.221+0000 7f196a000640 1 -- 172.21.15.175:0/4097321797 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196000cd80 con 0x7f196c107d10 2024-07-29T00:46:44.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.221+0000 7f196a000640 1 -- 172.21.15.175:0/4097321797 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1960010910 con 0x7f196c107d10 2024-07-29T00:46:44.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.221+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c1a3790 con 0x7f196c107d10 2024-07-29T00:46:44.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.221+0000 7f196a000640 1 -- 172.21.15.175:0/4097321797 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960018990 con 0x7f196c107d10 2024-07-29T00:46:44.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.221+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1ab070 con 0x7f196c107d10 2024-07-29T00:46:44.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.224+0000 7f196a000640 1 -- 172.21.15.175:0/4097321797 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f1960010ac0 con 0x7f196c107d10 2024-07-29T00:46:44.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.224+0000 7f19768bb740 1 Processor -- start 2024-07-29T00:46:44.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.224+0000 7f19768bb740 1 -- start start 2024-07-29T00:46:44.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f19768bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42cb10 0x564aea42cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f19768bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea4377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f19768bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea437cf0 0x564aea43a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f19768bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564aea391ad0 con 0x564aea42d430 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f19768bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564aea329af0 con 0x564aea42cb10 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f19768bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564aea38da90 con 0x564aea437cf0 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f1971400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea4377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f1971400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea4377b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55310/0 (socket says 172.21.15.175:55310) 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f1971400640 1 -- 172.21.15.175:0/789548179 learned_addr learned my addr 172.21.15.175:0/789548179 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.225+0000 7f1971e00640 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42cb10 0x564aea42cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1970a00640 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea437cf0 0x564aea43a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1971e00640 1 -- 172.21.15.175:0/789548179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea437cf0 msgr2=0x564aea43a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1971e00640 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea437cf0 0x564aea43a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1971e00640 1 -- 172.21.15.175:0/789548179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 msgr2=0x564aea4377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1971e00640 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea4377b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1971e00640 1 -- 172.21.15.175:0/789548179 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aea1033c0 con 0x564aea42cb10 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1970a00640 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea437cf0 0x564aea43a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f196a000640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19540a2d70 0x7f19540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f1971e00640 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42cb10 0x564aea42cef0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f196402ae10 tx=0x7f1964005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243676b0567b0751 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:44.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.226+0000 7f196a000640 1 -- 172.21.15.175:0/4097321797 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f19600cf6d0 con 0x7f196c107d10 2024-07-29T00:46:44.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f1971400640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19540a2d70 0x7f19540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f195e600640 1 -- 172.21.15.175:0/789548179 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964003970 con 0x564aea42cb10 2024-07-29T00:46:44.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f195e600640 1 -- 172.21.15.175:0/789548179 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1964036030 con 0x564aea42cb10 2024-07-29T00:46:44.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f195e600640 1 -- 172.21.15.175:0/789548179 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964004460 con 0x564aea42cb10 2024-07-29T00:46:44.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f19768bb740 1 -- 172.21.15.175:0/789548179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42cb10 msgr2=0x564aea42cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f1971400640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19540a2d70 0x7f19540a5230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f196009b8f0 tx=0x7f19600d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.227+0000 7f19768bb740 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42cb10 0x564aea42cef0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f196402ae10 tx=0x7f1964005fa0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 -- 172.21.15.175:0/789548179 shutdown_connections 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea437cf0 0x564aea43a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea4377b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 --2- 172.21.15.175:0/789548179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42cb10 0x564aea42cef0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 -- 172.21.15.175:0/789548179 >> 172.21.15.175:0/789548179 conn(0x564aea4290a0 msgr2=0x564aea429480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:44.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 -- 172.21.15.175:0/789548179 shutdown_connections 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 -- 172.21.15.175:0/789548179 wait complete. 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.228+0000 7f19768bb740 1 -- start start 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 Processor -- start 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea434b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42d430 0x564aea4350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea437cf0 0x564aea4355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564aea43a8c0 con 0x564aea437cf0 2024-07-29T00:46:44.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564aea34cbc0 con 0x564aea42d430 2024-07-29T00:46:44.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564aea43aa40 con 0x564aea42cb10 2024-07-29T00:46:44.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f19768bb740 10 client.0 _ll_register_callbacks cb 0x564aea338d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:44.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f1971e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea434b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f1970a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42d430 0x564aea4350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f1971e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea434b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:45250/0 (socket says 172.21.15.175:45250) 2024-07-29T00:46:44.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.229+0000 7f1971e00640 1 -- 172.21.15.175:0/915239173 learned_addr learned my addr 172.21.15.175:0/915239173 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:44.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1971e00640 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea437cf0 0x564aea4355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1970a00640 1 -- 172.21.15.175:0/915239173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 msgr2=0x564aea434b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1970a00640 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea434b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1970a00640 1 -- 172.21.15.175:0/915239173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea437cf0 msgr2=0x564aea4355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1970a00640 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea437cf0 0x564aea4355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1970a00640 1 -- 172.21.15.175:0/915239173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aea443d30 con 0x564aea42d430 2024-07-29T00:46:44.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1970a00640 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42d430 0x564aea4350a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f19580126b0 tx=0x7f1958012ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8380b567600a7734 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:44.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1952a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:46:44.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.230+0000 7f1952a00640 1 -- 172.21.15.175:0/915239173 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958004030 con 0x564aea42d430 2024-07-29T00:46:44.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1964036030 con 0x564aea42d430 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f1952a00640 1 -- 172.21.15.175:0/915239173 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f19580041d0 con 0x564aea42d430 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f1952a00640 1 -- 172.21.15.175:0/915239173 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f19580043d0 con 0x564aea42d430 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f19768bb740 10 client.27446 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564aea446770 con 0x564aea42d430 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f1952a00640 1 -- 172.21.15.175:0/915239173 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1964036030 con 0x564aea42d430 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f1952a00640 1 -- 172.21.15.175:0/915239173 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1958003950 con 0x564aea42d430 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f19768bb740 10 client.27446 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:44.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.231+0000 7f19768bb740 10 client.27446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f19768bb740 20 client.27446 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f19768bb740 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea44a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f19768bb740 4 client.27446 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aea39f040 con 0x564aea447c80 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f1971e00640 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea44a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f1971e00640 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea44a0a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f62ef555344a4a server_cookie=dd401e6ec41f0c86 in_seq=0 out_seq=0 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.232+0000 7f1952a00640 10 client.27446 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:44.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.233+0000 7f1952a00640 1 -- 172.21.15.175:0/915239173 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564aea39f040 con 0x564aea447c80 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.233+0000 7f1952a00640 10 client.27446 handle_command_reply: tid=0 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.233+0000 7f19768bb740 1 client.27446 shutdown 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.233+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42d430 msgr2=0x564aea4350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.233+0000 7f19768bb740 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42d430 0x564aea4350a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f19580126b0 tx=0x7f1958012ab0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 shutdown_connections 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea44a0a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea437cf0 0x564aea4355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea42d430 0x564aea4350a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f1952a00640 0 client.27446 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 --2- 172.21.15.175:0/915239173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea434b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 >> 172.21.15.175:0/915239173 conn(0x564aea4290a0 msgr2=0x564aea42edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 shutdown_connections 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 1 -- 172.21.15.175:0/915239173 wait complete. 2024-07-29T00:46:44.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.234+0000 7f19768bb740 20 client.27446 trim_cache size 0 max 16384 2024-07-29T00:46:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: scrub summary: active paths [/] 2024-07-29T00:46:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:44.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:43 smithi181 ceph-mon[36706]: pgmap v317: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:44.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 Processor -- start 2024-07-29T00:46:44.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 -- start start 2024-07-29T00:46:44.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42cb10 0x564aea435430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42d430 0x564aea4280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea44f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b3a90 con 0x564aea42cb10 2024-07-29T00:46:44.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b55d0 con 0x564aea437cf0 2024-07-29T00:46:44.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f19768bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b4160 con 0x564aea42d430 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.253+0000 7f1971e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42cb10 0x564aea435430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1971e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42cb10 0x564aea435430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55332/0 (socket says 172.21.15.175:55332) 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42d430 0x564aea4280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1971e00640 1 -- 172.21.15.175:0/2817213138 learned_addr learned my addr 172.21.15.175:0/2817213138 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1971e00640 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea44f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 -- 172.21.15.175:0/2817213138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 msgr2=0x564aea44f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea44f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 -- 172.21.15.175:0/2817213138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42cb10 msgr2=0x564aea435430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42cb10 0x564aea435430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 -- 172.21.15.175:0/2817213138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aea428ed0 con 0x564aea42d430 2024-07-29T00:46:44.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f1970a00640 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42d430 0x564aea4280f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1958003d70 tx=0x7f1958009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a549eade9aa4b8d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:44.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f195dc00640 1 -- 172.21.15.175:0/2817213138 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958010430 con 0x564aea42d430 2024-07-29T00:46:44.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f195dc00640 1 -- 172.21.15.175:0/2817213138 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1958003ad0 con 0x564aea42d430 2024-07-29T00:46:44.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.254+0000 7f195dc00640 1 -- 172.21.15.175:0/2817213138 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195800fc10 con 0x564aea42d430 2024-07-29T00:46:44.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 -- 172.21.15.175:0/2817213138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42d430 msgr2=0x564aea4280f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42d430 0x564aea4280f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1958003d70 tx=0x7f1958009f50 comp rx=0 tx=0).stop 2024-07-29T00:46:44.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 -- 172.21.15.175:0/2817213138 shutdown_connections 2024-07-29T00:46:44.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea44f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42d430 0x564aea4280f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 --2- 172.21.15.175:0/2817213138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42cb10 0x564aea435430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 -- 172.21.15.175:0/2817213138 >> 172.21.15.175:0/2817213138 conn(0x564aea4290a0 msgr2=0x564aea4279e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 -- 172.21.15.175:0/2817213138 shutdown_connections 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.255+0000 7f19768bb740 1 -- 172.21.15.175:0/2817213138 wait complete. 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 -- start start 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 Processor -- start 2024-07-29T00:46:44.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea43c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea43c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea4b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b4f00 con 0x564aea42d430 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b4830 con 0x564aea437cf0 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.256+0000 7f19768bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b1a90 con 0x564aea42cb10 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f19768bb740 10 client.0 _ll_register_callbacks cb 0x7f1958009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:46:44.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea43c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea43c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55348/0 (socket says 172.21.15.175:55348) 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 -- 172.21.15.175:0/1162504519 learned_addr learned my addr 172.21.15.175:0/1162504519 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1971e00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea43c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1971e00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea4b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 -- 172.21.15.175:0/1162504519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 msgr2=0x564aea43c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea43c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 -- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 msgr2=0x564aea4b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:46:44.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea4b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.257+0000 7f1970a00640 1 -- 172.21.15.175:0/1162504519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aea4b1e60 con 0x564aea42d430 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f1970a00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea43c630 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1958003ee0 tx=0x7f195800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed25328c15a205db server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f1952a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f1952a00640 1 -- 172.21.15.175:0/1162504519 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580038c0 con 0x564aea42d430 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f1952a00640 1 -- 172.21.15.175:0/1162504519 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f195800f990 con 0x564aea42d430 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1958003ad0 con 0x564aea42d430 2024-07-29T00:46:44.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f1952a00640 1 -- 172.21.15.175:0/1162504519 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f195800fbb0 con 0x564aea42d430 2024-07-29T00:46:44.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f19768bb740 10 client.17301 fetch_fsmap learned FSMap version 43 2024-07-29T00:46:44.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564aea4b30d0 con 0x564aea42d430 2024-07-29T00:46:44.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.258+0000 7f1952a00640 1 -- 172.21.15.175:0/1162504519 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1958003ad0 con 0x564aea42d430 2024-07-29T00:46:44.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f1952a00640 1 -- 172.21.15.175:0/1162504519 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1958035460 con 0x564aea42d430 2024-07-29T00:46:44.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f19768bb740 10 client.17301 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:46:44.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f19768bb740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f19768bb740 20 client.17301 populate_metadata read hostname 'smithi175' 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f19768bb740 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea4b5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f19768bb740 4 client.17301 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564aea399550 con 0x564aea447c80 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.259+0000 7f1971e00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea4b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f1971e00640 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea4b5710 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891c5b99a5145f0e server_cookie=8be95398573b8ea4 in_seq=0 out_seq=0 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f1952a00640 10 client.17301 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f1952a00640 1 -- 172.21.15.175:0/1162504519 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564aea399550 con 0x564aea447c80 2024-07-29T00:46:44.267 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f1952a00640 10 client.17301 handle_command_reply: tid=0 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 client.17301 shutdown 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 msgr2=0x564aea43c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea43c630 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1958003ee0 tx=0x7f195800f870 comp rx=0 tx=0).stop 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 shutdown_connections 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564aea447c80 0x564aea4b5710 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aea437cf0 0x564aea4b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564aea42d430 0x564aea43c630 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 --2- 172.21.15.175:0/1162504519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aea42cb10 0x564aea43c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 >> 172.21.15.175:0/1162504519 conn(0x564aea4290a0 msgr2=0x564aea42ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:44.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f1952a00640 0 client.17301 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:46:44.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 shutdown_connections 2024-07-29T00:46:44.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.260+0000 7f19768bb740 1 -- 172.21.15.175:0/1162504519 wait complete. 2024-07-29T00:46:44.269 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f19768bb740 20 client.17301 trim_cache size 0 max 16384 2024-07-29T00:46:44.269 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": { 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "a2888099-4fbb-4bd1-81c4-6fc3e864d7b5": { 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "path": "/", 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "tag": "a2888099-4fbb-4bd1-81c4-6fc3e864d7b5", 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "options": "recursive,force" 2024-07-29T00:46:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:46:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:46:44.271 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:46:44.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19540a2d70 msgr2=0x7f19540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19540a2d70 0x7f19540a5230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f196009b8f0 tx=0x7f19600d5530 comp rx=0 tx=0).stop 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 msgr2=0x7f196c19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 0x7f196c19e4f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9f0 tx=0x7f196000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 shutdown_connections 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f19540a2d70 0x7f19540a5230 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c10dd50 0x7f196c1a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c108670 0x7f196c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 --2- 172.21.15.175:0/4097321797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f196c107d10 0x7f196c19e4f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:44.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.261+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 >> 172.21.15.175:0/4097321797 conn(0x7f196c077670 msgr2=0x7f196c078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:44.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.262+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 shutdown_connections 2024-07-29T00:46:44.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:44.262+0000 7f1972800640 1 -- 172.21.15.175:0/4097321797 wait complete. 2024-07-29T00:46:44.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:44.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:44.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:44.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:44.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: scrub summary: active paths [/] 2024-07-29T00:46:44.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: scrub summary: idle+waiting paths [/] 2024-07-29T00:46:44.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:43 smithi204 ceph-mon[37368]: pgmap v317: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:46:45.139 INFO:tasks.cephfs.filesystem:scrub status for tag:a2888099-4fbb-4bd1-81c4-6fc3e864d7b5 - {'path': '/', 'tag': 'a2888099-4fbb-4bd1-81c4-6fc3e864d7b5', 'options': 'recursive,force'} 2024-07-29T00:46:45.736 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-29T00:46:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 -- 172.21.15.175:0/1480604758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 msgr2=0x7fd890114a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 --2- 172.21.15.175:0/1480604758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd890114a80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd888009cd0 tx=0x7fd88802a8f0 comp rx=0 tx=0).stop 2024-07-29T00:46:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 -- 172.21.15.175:0/1480604758 shutdown_connections 2024-07-29T00:46:45.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 --2- 172.21.15.175:0/1480604758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd890114a80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 --2- 172.21.15.175:0/1480604758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd890107d10 0x7fd890108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 --2- 172.21.15.175:0/1480604758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89010dc70 0x7fd89010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 -- 172.21.15.175:0/1480604758 >> 172.21.15.175:0/1480604758 conn(0x7fd890077550 msgr2=0x7fd890079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 -- 172.21.15.175:0/1480604758 shutdown_connections 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.934+0000 7fd895800640 1 -- 172.21.15.175:0/1480604758 wait complete. 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 Processor -- start 2024-07-29T00:46:45.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 -- start start 2024-07-29T00:46:45.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd890107d10 0x7fd89019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:45.938 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd89019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89010dc70 0x7fd8901a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd890118020 con 0x7fd890108690 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd890117ea0 con 0x7fd89010dc70 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.935+0000 7fd895800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901181a0 con 0x7fd890107d10 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd89019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd88fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd890107d10 0x7fd89019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:45.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd89019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55372/0 (socket says 172.21.15.175:55372) 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 -- 172.21.15.175:0/1981809467 learned_addr learned my addr 172.21.15.175:0/1981809467 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd894e00640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89010dc70 0x7fd8901a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 -- 172.21.15.175:0/1981809467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd890107d10 msgr2=0x7fd89019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd890107d10 0x7fd89019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 -- 172.21.15.175:0/1981809467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89010dc70 msgr2=0x7fd8901a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89010dc70 0x7fd8901a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:45.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 -- 172.21.15.175:0/1981809467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8901a3420 con 0x7fd890108690 2024-07-29T00:46:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd88fe00640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd890107d10 0x7fd89019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:46:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd894e00640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89010dc70 0x7fd8901a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:46:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.936+0000 7fd887400640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd89019e970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd87800d950 tx=0x7fd87800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.937+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878017070 con 0x7fd890108690 2024-07-29T00:46:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.937+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd878009c30 con 0x7fd890108690 2024-07-29T00:46:45.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.937+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8901a3710 con 0x7fd890108690 2024-07-29T00:46:45.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.937+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87800f6b0 con 0x7fd890108690 2024-07-29T00:46:45.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.937+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8901aafa0 con 0x7fd890108690 2024-07-29T00:46:45.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.938+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd890108150 con 0x7fd890108690 2024-07-29T00:46:45.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.940+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd8780109d0 con 0x7fd890108690 2024-07-29T00:46:45.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.943+0000 7fd88d600640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8740a2cf0 0x7fd8740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:45.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.943+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fd8780cf150 con 0x7fd890108690 2024-07-29T00:46:45.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.943+0000 7fd88fe00640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8740a2cf0 0x7fd8740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:45.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.943+0000 7fd88fe00640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8740a2cf0 0x7fd8740a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd88000bfd0 tx=0x7fd88000b390 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:45.947 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:45.947+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd878009de0 con 0x7fd890108690 2024-07-29T00:46:46.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:45 smithi175 ceph-mon[32138]: pgmap v318: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:46.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.174+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd890062320 con 0x7fd8740a2cf0 2024-07-29T00:46:46.188 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.188+0000 7fd88d600640 1 -- 172.21.15.175:0/1981809467 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fd890062320 con 0x7fd8740a2cf0 2024-07-29T00:46:46.188 INFO:teuthology.orchestra.run.smithi175.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-29T00:00:00", "created": "2024-07-29T00:36:43", "first": "2024-07-29T00:37:00", "last": "2024-07-29T00:46:00", "last_pruned": "2024-07-29T00:46:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-07-29T00:46:46.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8740a2cf0 msgr2=0x7fd8740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8740a2cf0 0x7fd8740a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd88000bfd0 tx=0x7fd88000b390 comp rx=0 tx=0).stop 2024-07-29T00:46:46.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 msgr2=0x7fd89019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.191 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd89019e970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd87800d950 tx=0x7fd87800de10 comp rx=0 tx=0).stop 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 shutdown_connections 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89010dc70 0x7fd8901a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd8740a2cf0 0x7fd8740a51b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd890108690 0x7fd89019e970 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 --2- 172.21.15.175:0/1981809467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd890107d10 0x7fd89019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 >> 172.21.15.175:0/1981809467 conn(0x7fd890077550 msgr2=0x7fd890078be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:46.192 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 shutdown_connections 2024-07-29T00:46:46.193 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.191+0000 7fd895800640 1 -- 172.21.15.175:0/1981809467 wait complete. 2024-07-29T00:46:46.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-29T00:46:46.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:45 smithi181 ceph-mon[36706]: pgmap v318: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:46.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:45 smithi204 ceph-mon[37368]: pgmap v318: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:46:46.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 -- 172.21.15.175:0/1233299640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074c30 msgr2=0x7f3e6010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/1233299640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074c30 0x7f3e6010d8a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009ca0 tx=0x7f3e4c02a950 comp rx=0 tx=0).stop 2024-07-29T00:46:46.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 -- 172.21.15.175:0/1233299640 shutdown_connections 2024-07-29T00:46:46.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/1233299640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e6010dde0 0x7f3e60110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/1233299640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074c30 0x7f3e6010d8a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/1233299640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e60074310 0x7f3e600746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 -- 172.21.15.175:0/1233299640 >> 172.21.15.175:0/1233299640 conn(0x7f3e600fd2c0 msgr2=0x7f3e600ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:46.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 -- 172.21.15.175:0/1233299640 shutdown_connections 2024-07-29T00:46:46.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.410+0000 7f3e5fe00640 1 -- 172.21.15.175:0/1233299640 wait complete. 2024-07-29T00:46:46.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.411+0000 7f3e5fe00640 1 Processor -- start 2024-07-29T00:46:46.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.411+0000 7f3e5fe00640 1 -- start start 2024-07-29T00:46:46.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 0x7f3e6019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:46.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60074c30 0x7f3e6019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e6010dde0 0x7f3e6019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60113be0 con 0x7f3e60074310 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60113a60 con 0x7f3e60074c30 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60113d60 con 0x7f3e6010dde0 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 0x7f3e6019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 0x7f3e6019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55390/0 (socket says 172.21.15.175:55390) 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5ea00640 1 -- 172.21.15.175:0/2816610682 learned_addr learned my addr 172.21.15.175:0/2816610682 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5f400640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e6010dde0 0x7f3e6019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.412+0000 7f3e5e000640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60074c30 0x7f3e6019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:46.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5ea00640 1 -- 172.21.15.175:0/2816610682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e6010dde0 msgr2=0x7f3e6019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5ea00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e6010dde0 0x7f3e6019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5ea00640 1 -- 172.21.15.175:0/2816610682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60074c30 msgr2=0x7f3e6019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5ea00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60074c30 0x7f3e6019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5ea00640 1 -- 172.21.15.175:0/2816610682 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6019f100 con 0x7f3e60074310 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5ea00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 0x7f3e6019a110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000e9f0 tx=0x7f3e5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000cd80 con 0x7f3e60074310 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e50005500 con 0x7f3e60074310 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6019f3f0 con 0x7f3e60074310 2024-07-29T00:46:46.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50010690 con 0x7f3e60074310 2024-07-29T00:46:46.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.413+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e601a6c80 con 0x7f3e60074310 2024-07-29T00:46:46.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.414+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e600746f0 con 0x7f3e60074310 2024-07-29T00:46:46.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.416+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3e500046c0 con 0x7f3e60074310 2024-07-29T00:46:46.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.418+0000 7f3e64a00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3e440a2cf0 0x7f3e440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:46:46.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.418+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f3e500cf920 con 0x7f3e60074310 2024-07-29T00:46:46.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.418+0000 7f3e5e000640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3e440a2cf0 0x7f3e440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:46:46.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.419+0000 7f3e5e000640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3e440a2cf0 0x7f3e440a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c02af40 tx=0x7f3e4c005d80 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:46:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.422+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e5009c490 con 0x7f3e60074310 2024-07-29T00:46:46.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.647+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 --> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3e60062320 con 0x7f3e440a2cf0 2024-07-29T00:46:46.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.662+0000 7f3e64a00640 1 -- 172.21.15.175:0/2816610682 <== mgr.15033 v2:172.21.15.175:6800/1625031332 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3e60062320 con 0x7f3e440a2cf0 2024-07-29T00:46:46.661 INFO:teuthology.orchestra.run.smithi175.stdout:/ 1m 6m3h 2024-07-29T00:46:46.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3e440a2cf0 msgr2=0x7f3e440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3e440a2cf0 0x7f3e440a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c02af40 tx=0x7f3e4c005d80 comp rx=0 tx=0).stop 2024-07-29T00:46:46.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 msgr2=0x7f3e6019a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:46:46.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 0x7f3e6019a110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000e9f0 tx=0x7f3e5000edf0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 shutdown_connections 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e6010dde0 0x7f3e6019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.664+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60074c30 0x7f3e6019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.665+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f3e440a2cf0 0x7f3e440a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.665+0000 7f3e5fe00640 1 --2- 172.21.15.175:0/2816610682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e60074310 0x7f3e6019a110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.665+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 >> 172.21.15.175:0/2816610682 conn(0x7f3e600fd2c0 msgr2=0x7f3e600fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.665+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 shutdown_connections 2024-07-29T00:46:46.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:46:46.665+0000 7f3e5fe00640 1 -- 172.21.15.175:0/2816610682 wait complete. 2024-07-29T00:46:46.684 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-07-29T00:46:46.697 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-07-29T00:46:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: from='client.17307 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:46:48.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: pgmap v319: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-07-29T00:46:48.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: from='client.17313 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:46:48.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:46:48.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:46:48.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:46:48.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:47 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:46:48.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: from='client.17307 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:46:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: pgmap v319: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-07-29T00:46:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: from='client.17313 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:46:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:46:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:46:48.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:46:48.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:47 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: from='client.17307 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: pgmap v319: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: from='client.17313 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:46:48.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:47 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:46:49.236 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:48 smithi204 ceph-mon[37368]: scrub summary: idle 2024-07-29T00:46:49.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:48 smithi181 ceph-mon[36706]: scrub summary: idle 2024-07-29T00:46:49.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:48 smithi175 ceph-mon[32138]: scrub summary: idle 2024-07-29T00:46:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:49 smithi181 ceph-mon[36706]: pgmap v320: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:50.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:49 smithi175 ceph-mon[32138]: pgmap v320: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:50.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:49 smithi204 ceph-mon[37368]: pgmap v320: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:51 smithi181 ceph-mon[36706]: pgmap v321: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:51 smithi175 ceph-mon[32138]: pgmap v321: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:52.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:51 smithi204 ceph-mon[37368]: pgmap v321: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-07-29T00:46:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:53 smithi181 ceph-mon[36706]: pgmap v322: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-29T00:46:54.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:53 smithi175 ceph-mon[32138]: pgmap v322: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-29T00:46:54.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:53 smithi204 ceph-mon[37368]: pgmap v322: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-29T00:46:55.233 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:54 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:46:55.244 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:54 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:46:55.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:54 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:46:56.233 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:55 smithi175 ceph-mon[32138]: pgmap v323: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:46:56.233 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:55 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.233 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:55 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.233 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:55 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:55 smithi181 ceph-mon[36706]: pgmap v323: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:46:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:55 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:55 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:55 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:55 smithi204 ceph-mon[37368]: pgmap v323: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:46:56.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:55 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:55 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:56.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:55 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:56 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.287 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:56 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:57.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:57.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:56 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:58.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: pgmap v324: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-29T00:46:58.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-07-29T00:46:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-07-29T00:46:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-07-29T00:46:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-07-29T00:46:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:58.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:57 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:58.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: pgmap v324: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-29T00:46:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-07-29T00:46:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-07-29T00:46:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-07-29T00:46:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-07-29T00:46:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:58.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:57 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:58.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: pgmap v324: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-29T00:46:58.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-07-29T00:46:58.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-07-29T00:46:58.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-07-29T00:46:58.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-07-29T00:46:58.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:58.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:57 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:59.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:46:59.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:46:59.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:58 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:46:59.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:46:59.286 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:58 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:46:59.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:46:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:46:59.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:58 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:47:00.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:59 smithi181 ceph-mon[36706]: pgmap v325: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:47:00.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:59 smithi181 ceph-mon[36706]: Adjusting osd_memory_target on smithi175 to 176.4M 2024-07-29T00:47:00.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:46:59 smithi181 ceph-mon[36706]: Unable to set osd_memory_target on smithi175 to 185029017: error parsing value: Value '185029017' is below minimum 939524096 2024-07-29T00:47:00.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:59 smithi175 ceph-mon[32138]: pgmap v325: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:47:00.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:59 smithi175 ceph-mon[32138]: Adjusting osd_memory_target on smithi175 to 176.4M 2024-07-29T00:47:00.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:46:59 smithi175 ceph-mon[32138]: Unable to set osd_memory_target on smithi175 to 185029017: error parsing value: Value '185029017' is below minimum 939524096 2024-07-29T00:47:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:59 smithi204 ceph-mon[37368]: pgmap v325: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:47:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:59 smithi204 ceph-mon[37368]: Adjusting osd_memory_target on smithi175 to 176.4M 2024-07-29T00:47:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:46:59 smithi204 ceph-mon[37368]: Unable to set osd_memory_target on smithi175 to 185029017: error parsing value: Value '185029017' is below minimum 939524096 2024-07-29T00:47:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:02 smithi175 ceph-mon[32138]: pgmap v326: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:47:02.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:02 smithi204 ceph-mon[37368]: pgmap v326: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:47:02.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:02 smithi181 ceph-mon[36706]: pgmap v326: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-29T00:47:03.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:03 smithi175 ceph-mon[32138]: pgmap v327: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 32 op/s 2024-07-29T00:47:03.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:03 smithi204 ceph-mon[37368]: pgmap v327: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 32 op/s 2024-07-29T00:47:03.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:03 smithi181 ceph-mon[36706]: pgmap v327: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 32 op/s 2024-07-29T00:47:05.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:05 smithi204 ceph-mon[37368]: pgmap v328: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:47:05.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:05 smithi181 ceph-mon[36706]: pgmap v328: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:47:05.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:05 smithi175 ceph-mon[32138]: pgmap v328: 97 pgs: 97 active+clean; 125 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:47:07.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:07 smithi204 ceph-mon[37368]: pgmap v329: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:47:07.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:07 smithi181 ceph-mon[36706]: pgmap v329: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:47:07.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:07 smithi175 ceph-mon[32138]: pgmap v329: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-07-29T00:47:09.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:09 smithi204 ceph-mon[37368]: pgmap v330: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:47:09.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:09 smithi204 ceph-mon[37368]: osdmap e101: 12 total, 12 up, 12 in 2024-07-29T00:47:09.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:09 smithi181 ceph-mon[36706]: pgmap v330: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:47:09.750 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:09 smithi181 ceph-mon[36706]: osdmap e101: 12 total, 12 up, 12 in 2024-07-29T00:47:09.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:09 smithi175 ceph-mon[32138]: pgmap v330: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-29T00:47:09.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:09 smithi175 ceph-mon[32138]: osdmap e101: 12 total, 12 up, 12 in 2024-07-29T00:47:11.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:11 smithi204 ceph-mon[37368]: pgmap v332: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:47:11.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:11 smithi181 ceph-mon[36706]: pgmap v332: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:47:11.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:11 smithi175 ceph-mon[32138]: pgmap v332: 97 pgs: 97 active+clean; 125 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-29T00:47:13.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:13 smithi204 ceph-mon[37368]: pgmap v333: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:13.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:13 smithi181 ceph-mon[36706]: pgmap v333: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:13.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:13 smithi175 ceph-mon[32138]: pgmap v333: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:15.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-29T00:47:15.534 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:15 smithi175 ceph-mon[32138]: pgmap v334: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:15.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:15 smithi204 ceph-mon[37368]: pgmap v334: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:15.600 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:15.749 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:15 smithi181 ceph-mon[36706]: pgmap v334: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:16.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 -- 172.21.15.175:0/856202051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f086410be60 msgr2=0x7f086410e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 --2- 172.21.15.175:0/856202051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f086410be60 0x7f086410e300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f085c009ca0 tx=0x7f085c02a950 comp rx=0 tx=0).stop 2024-07-29T00:47:16.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 -- 172.21.15.175:0/856202051 shutdown_connections 2024-07-29T00:47:16.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 --2- 172.21.15.175:0/856202051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f086410be60 0x7f086410e300 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 --2- 172.21.15.175:0/856202051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0864103350 0x7f086410b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 --2- 172.21.15.175:0/856202051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864102a30 0x7f0864102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 -- 172.21.15.175:0/856202051 >> 172.21.15.175:0/856202051 conn(0x7f08640fc490 msgr2=0x7f08640fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:16.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 -- 172.21.15.175:0/856202051 shutdown_connections 2024-07-29T00:47:16.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.277+0000 7f086c000640 1 -- 172.21.15.175:0/856202051 wait complete. 2024-07-29T00:47:16.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 Processor -- start 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 -- start start 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0864102a30 0x7f0864198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864103350 0x7f0864198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f086410be60 0x7f086419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111c60 con 0x7f0864102a30 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111ae0 con 0x7f086410be60 2024-07-29T00:47:16.282 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.279+0000 7f086c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111de0 con 0x7f0864103350 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0868c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864103350 0x7f0864198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0868c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864103350 0x7f0864198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49118/0 (socket says 172.21.15.175:49118) 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0868c00640 1 -- 172.21.15.175:0/2214976410 learned_addr learned my addr 172.21.15.175:0/2214976410 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0869600640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0864102a30 0x7f0864198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f086a000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f086410be60 0x7f086419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0869600640 1 -- 172.21.15.175:0/2214976410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864103350 msgr2=0x7f0864198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0869600640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864103350 0x7f0864198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0869600640 1 -- 172.21.15.175:0/2214976410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f086410be60 msgr2=0x7f086419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0869600640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f086410be60 0x7f086419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.280+0000 7f0869600640 1 -- 172.21.15.175:0/2214976410 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086419d0b0 con 0x7f0864102a30 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.281+0000 7f0869600640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0864102a30 0x7f0864198100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f085800e9f0 tx=0x7f085800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.281+0000 7f0862000640 1 -- 172.21.15.175:0/2214976410 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085800ccc0 con 0x7f0864102a30 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.281+0000 7f0862000640 1 -- 172.21.15.175:0/2214976410 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0858005020 con 0x7f0864102a30 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.281+0000 7f0862000640 1 -- 172.21.15.175:0/2214976410 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580049e0 con 0x7f0864102a30 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.281+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086419d3a0 con 0x7f0864102a30 2024-07-29T00:47:16.284 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.281+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08641a4c80 con 0x7f0864102a30 2024-07-29T00:47:16.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.284+0000 7f086ea17740 1 Processor -- start 2024-07-29T00:47:16.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.284+0000 7f0862000640 1 -- 172.21.15.175:0/2214976410 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0858005560 con 0x7f0864102a30 2024-07-29T00:47:16.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.284+0000 7f086ea17740 1 -- start start 2024-07-29T00:47:16.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086ea17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086ea17740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12545150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086ea17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff1255a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086ea17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ff124a9ad0 con 0x55ff125581b0 2024-07-29T00:47:16.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086ea17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff12441af0 con 0x55ff12545eb0 2024-07-29T00:47:16.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086ea17740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ff124a5a90 con 0x55ff12544d10 2024-07-29T00:47:16.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55454/0 (socket says 172.21.15.175:55454) 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.285+0000 7f086a000640 1 -- 172.21.15.175:0/4192984919 learned_addr learned my addr 172.21.15.175:0/4192984919 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f0869600640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12545150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f0868c00640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff1255a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f086a000640 1 -- 172.21.15.175:0/4192984919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 msgr2=0x55ff12545150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f086a000640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12545150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f086a000640 1 -- 172.21.15.175:0/4192984919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 msgr2=0x55ff1255a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f086a000640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff1255a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f086a000640 1 -- 172.21.15.175:0/4192984919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1221b3c0 con 0x55ff12545eb0 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f0869600640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12545150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f0868c00640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff1255a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f0862000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f084c0a2cf0 0x7f084c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.286+0000 7f086a000640 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125447d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f085c030030 tx=0x7f085c0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5a774b2934eaab server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.287+0000 7f0862000640 1 -- 172.21.15.175:0/2214976410 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f08580cf480 con 0x7f0864102a30 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.287+0000 7f0869600640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f084c0a2cf0 0x7f084c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.287+0000 7f0856600640 1 -- 172.21.15.175:0/4192984919 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c034040 con 0x55ff12545eb0 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.287+0000 7f0856600640 1 -- 172.21.15.175:0/4192984919 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085c02f940 con 0x55ff12545eb0 2024-07-29T00:47:16.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.287+0000 7f0856600640 1 -- 172.21.15.175:0/4192984919 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0042a0 con 0x55ff12545eb0 2024-07-29T00:47:16.292 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 -- 172.21.15.175:0/4192984919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 msgr2=0x55ff125447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125447d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f085c030030 tx=0x7f085c0093e0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f0869600640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f084c0a2cf0 0x7f084c0a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ff12541d20 tx=0x7f08580d2f30 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 -- 172.21.15.175:0/4192984919 shutdown_connections 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff1255a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12545150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 --2- 172.21.15.175:0/4192984919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125447d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 -- 172.21.15.175:0/4192984919 >> 172.21.15.175:0/4192984919 conn(0x55ff12540ec0 msgr2=0x55ff125412a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:16.293 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 -- 172.21.15.175:0/4192984919 shutdown_connections 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.288+0000 7f086ea17740 1 -- 172.21.15.175:0/4192984919 wait complete. 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.289+0000 7f086ea17740 1 -- start start 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.289+0000 7f086ea17740 1 Processor -- start 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12548630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff12548b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ff1255ae80 con 0x55ff125581b0 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff12464bc0 con 0x55ff12545eb0 2024-07-29T00:47:16.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ff1255b000 con 0x55ff12544d10 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086ea17740 10 client.0 _ll_register_callbacks cb 0x55ff12450d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12548630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12548630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49132/0 (socket says 172.21.15.175:49132) 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086a000640 1 -- 172.21.15.175:0/1363797551 learned_addr learned my addr 172.21.15.175:0/1363797551 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f086a000640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.290+0000 7f0868c00640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff12548b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086a000640 1 -- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 msgr2=0x55ff12548b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086a000640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff12548b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086a000640 1 -- 172.21.15.175:0/1363797551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 msgr2=0x55ff125490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086a000640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086a000640 1 -- 172.21.15.175:0/1363797551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff12564500 con 0x55ff12544d10 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086a000640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12548630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f085c02ff70 tx=0x7f085c003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5a92958b313e73 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f084aa00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f084aa00640 1 -- 172.21.15.175:0/1363797551 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c049040 con 0x55ff12544d10 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f085c02f940 con 0x55ff12544d10 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f084aa00640 1 -- 172.21.15.175:0/1363797551 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f085c0046d0 con 0x55ff12544d10 2024-07-29T00:47:16.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f084aa00640 1 -- 172.21.15.175:0/1363797551 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f085c02f3a0 con 0x55ff12544d10 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086ea17740 10 client.27418 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.291+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ff1255ae80 con 0x55ff12544d10 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f084aa00640 1 -- 172.21.15.175:0/1363797551 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085c02f940 con 0x55ff12544d10 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f084aa00640 1 -- 172.21.15.175:0/1363797551 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f085c04f040 con 0x55ff12544d10 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f086ea17740 10 client.27418 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f086ea17740 10 client.27418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f086ea17740 20 client.27418 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f086ea17740 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff12567ff0 0x55ff1256a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f086ea17740 4 client.27418 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:47:16.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.292+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff124b7040 con 0x55ff12567ff0 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.293+0000 7f0868c00640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff12567ff0 0x55ff1256a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.293+0000 7f084aa00640 10 client.27418 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.293+0000 7f0868c00640 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff12567ff0 0x55ff1256a3e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349a246a9531bf8a server_cookie=b30727586938398 in_seq=0 out_seq=0 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.294+0000 7f084aa00640 1 -- 172.21.15.175:0/1363797551 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff124b7040 con 0x55ff12567ff0 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.294+0000 7f084aa00640 10 client.27418 handle_command_reply: tid=0 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.294+0000 7f086ea17740 1 client.27418 shutdown 2024-07-29T00:47:16.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 msgr2=0x55ff12548630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12548630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f085c02ff70 tx=0x7f085c003c00 comp rx=0 tx=0).stop 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 shutdown_connections 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff12567ff0 0x55ff1256a3e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff12548b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 --2- 172.21.15.175:0/1363797551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12548630 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 >> 172.21.15.175:0/1363797551 conn(0x55ff12540ec0 msgr2=0x55ff12546c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:16.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f084aa00640 0 client.27418 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:16.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 shutdown_connections 2024-07-29T00:47:16.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 1 -- 172.21.15.175:0/1363797551 wait complete. 2024-07-29T00:47:16.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.295+0000 7f086ea17740 20 client.27418 trim_cache size 0 max 16384 2024-07-29T00:47:16.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 Processor -- start 2024-07-29T00:47:16.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 -- start start 2024-07-29T00:47:16.313 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12544d10 0x55ff12548f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 0x55ff1254a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff125581b0 0x55ff1256fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ab4c0 con 0x55ff12545eb0 2024-07-29T00:47:16.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ad000 con 0x55ff12544d10 2024-07-29T00:47:16.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.313+0000 7f086ea17740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08641abb90 con 0x55ff125581b0 2024-07-29T00:47:16.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 0x55ff1254a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 0x55ff1254a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33038/0 (socket says 172.21.15.175:33038) 2024-07-29T00:47:16.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 -- 172.21.15.175:0/855815894 learned_addr learned my addr 172.21.15.175:0/855815894 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f086a000640 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12544d10 0x55ff12548f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f086a000640 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff125581b0 0x55ff1256fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 -- 172.21.15.175:0/855815894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff125581b0 msgr2=0x55ff1256fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff125581b0 0x55ff1256fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 -- 172.21.15.175:0/855815894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12544d10 msgr2=0x55ff12548f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12544d10 0x55ff12548f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 -- 172.21.15.175:0/855815894 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff12545720 con 0x55ff12545eb0 2024-07-29T00:47:16.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f086a000640 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff125581b0 0x55ff1256fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:47:16.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.314+0000 7f0868c00640 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 0x55ff1254a5a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f086410d180 tx=0x7f0850005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3798815b1c49bee server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:16.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.315+0000 7f0855c00640 1 -- 172.21.15.175:0/855815894 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850003de0 con 0x55ff12545eb0 2024-07-29T00:47:16.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.315+0000 7f0855c00640 1 -- 172.21.15.175:0/855815894 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0850010910 con 0x55ff12545eb0 2024-07-29T00:47:16.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.315+0000 7f0855c00640 1 -- 172.21.15.175:0/855815894 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085001f9d0 con 0x55ff12545eb0 2024-07-29T00:47:16.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 -- 172.21.15.175:0/855815894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 msgr2=0x55ff1254a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 0x55ff1254a5a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f086410d180 tx=0x7f0850005f90 comp rx=0 tx=0).stop 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 -- 172.21.15.175:0/855815894 shutdown_connections 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff125581b0 0x55ff1256fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff12545eb0 0x55ff1254a5a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 --2- 172.21.15.175:0/855815894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12544d10 0x55ff12548f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 -- 172.21.15.175:0/855815894 >> 172.21.15.175:0/855815894 conn(0x55ff12540ec0 msgr2=0x55ff12549e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 -- 172.21.15.175:0/855815894 shutdown_connections 2024-07-29T00:47:16.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.316+0000 7f086ea17740 1 -- 172.21.15.175:0/855815894 wait complete. 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.317+0000 7f086ea17740 1 -- start start 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.317+0000 7f086ea17740 1 Processor -- start 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.317+0000 7f086ea17740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12556e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.317+0000 7f086ea17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.317+0000 7f086ea17740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086ea17740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ac930 con 0x55ff125581b0 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086ea17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ac260 con 0x55ff12545eb0 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086ea17740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08641a94c0 con 0x55ff12544d10 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086ea17740 10 client.0 _ll_register_callbacks cb 0x7f085c003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:16.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f0868c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f0868c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:55480/0 (socket says 172.21.15.175:55480) 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f0868c00640 1 -- 172.21.15.175:0/2688936161 learned_addr learned my addr 172.21.15.175:0/2688936161 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12556e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 -- 172.21.15.175:0/2688936161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 msgr2=0x55ff12556e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12556e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 -- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 msgr2=0x55ff125602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.318+0000 7f086a000640 1 -- 172.21.15.175:0/2688936161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff12554eb0 con 0x55ff125581b0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f086a000640 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125608d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f085c002230 tx=0x7f085c003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842a0223b377034 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f084aa00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f084aa00640 1 -- 172.21.15.175:0/2688936161 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c034020 con 0x55ff125581b0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0850010910 con 0x55ff125581b0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f084aa00640 1 -- 172.21.15.175:0/2688936161 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f085c002030 con 0x55ff125581b0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f084aa00640 1 -- 172.21.15.175:0/2688936161 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f085c05e040 con 0x55ff125581b0 2024-07-29T00:47:16.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f086ea17740 10 client.17331 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.319+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ff125d2a30 con 0x55ff125581b0 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f084aa00640 1 -- 172.21.15.175:0/2688936161 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0850010910 con 0x55ff125581b0 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f084aa00640 1 -- 172.21.15.175:0/2688936161 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f085c04f040 con 0x55ff125581b0 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f086ea17740 10 client.17331 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f086ea17740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f086ea17740 20 client.17331 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f086ea17740 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff125d30d0 0x55ff125d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:16.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f086ea17740 4 client.17331 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff124b1550 con 0x55ff125d30d0 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.320+0000 7f0868c00640 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff125d30d0 0x55ff125d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f0868c00640 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff125d30d0 0x55ff125d54f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1bd296d0ce74c6 server_cookie=63c6b23b04a564a5 in_seq=0 out_seq=0 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f084aa00640 10 client.17331 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f084aa00640 1 -- 172.21.15.175:0/2688936161 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff124b1550 con 0x55ff125d30d0 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f084aa00640 10 client.17331 handle_command_reply: tid=0 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f086ea17740 1 client.17331 shutdown 2024-07-29T00:47:16.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 msgr2=0x55ff125608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.321+0000 7f086ea17740 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125608d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f085c002230 tx=0x7f085c003b60 comp rx=0 tx=0).stop 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 shutdown_connections 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x55ff125d30d0 0x55ff125d54f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ff125581b0 0x55ff125608d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff12545eb0 0x55ff125602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 --2- 172.21.15.175:0/2688936161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff12544d10 0x55ff12556e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 >> 172.21.15.175:0/2688936161 conn(0x55ff12540ec0 msgr2=0x55ff12546c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f084aa00640 0 client.17331 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:16.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 shutdown_connections 2024-07-29T00:47:16.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 1 -- 172.21.15.175:0/2688936161 wait complete. 2024-07-29T00:47:16.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.322+0000 7f086ea17740 20 client.17331 trim_cache size 0 max 16384 2024-07-29T00:47:16.328 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:47:16.328 INFO:teuthology.orchestra.run.smithi175.stdout: "status": "no active scrubs running", 2024-07-29T00:47:16.328 INFO:teuthology.orchestra.run.smithi175.stdout: "scrubs": {} 2024-07-29T00:47:16.329 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:47:16.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f084c0a2cf0 msgr2=0x7f084c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f084c0a2cf0 0x7f084c0a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ff12541d20 tx=0x7f08580d2f30 comp rx=0 tx=0).stop 2024-07-29T00:47:16.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0864102a30 msgr2=0x7f0864198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0864102a30 0x7f0864198100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f085800e9f0 tx=0x7f085800eeb0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 shutdown_connections 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f086410be60 0x7f086419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0864103350 0x7f0864198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f084c0a2cf0 0x7f084c0a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 --2- 172.21.15.175:0/2214976410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0864102a30 0x7f0864198100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 >> 172.21.15.175:0/2214976410 conn(0x7f08640fc490 msgr2=0x7f08640fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:16.330 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 shutdown_connections 2024-07-29T00:47:16.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:16.323+0000 7f086c000640 1 -- 172.21.15.175:0/2214976410 wait complete. 2024-07-29T00:47:17.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:16 smithi175 ceph-mon[32138]: osdmap e102: 12 total, 12 up, 12 in 2024-07-29T00:47:17.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:16 smithi204 ceph-mon[37368]: osdmap e102: 12 total, 12 up, 12 in 2024-07-29T00:47:17.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:16 smithi181 ceph-mon[36706]: osdmap e102: 12 total, 12 up, 12 in 2024-07-29T00:47:17.485 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-29T00:47:17.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:47:17.946 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:18.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:17 smithi175 ceph-mon[32138]: pgmap v336: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-07-29T00:47:18.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:17 smithi204 ceph-mon[37368]: pgmap v336: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-07-29T00:47:18.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:17 smithi181 ceph-mon[36706]: pgmap v336: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-07-29T00:47:18.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 -- 172.21.15.175:0/1826577941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec10d220 msgr2=0x7fd7ec114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:18.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 --2- 172.21.15.175:0/1826577941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec10d220 0x7fd7ec114da0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009ca0 tx=0x7fd7e402a8c0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 -- 172.21.15.175:0/1826577941 shutdown_connections 2024-07-29T00:47:18.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 --2- 172.21.15.175:0/1826577941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec10d220 0x7fd7ec114da0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 --2- 172.21.15.175:0/1826577941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7ec108af0 0x7fd7ec10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 --2- 172.21.15.175:0/1826577941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ec108140 0x7fd7ec108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 -- 172.21.15.175:0/1826577941 >> 172.21.15.175:0/1826577941 conn(0x7fd7ec077890 msgr2=0x7fd7ec079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:18.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 -- 172.21.15.175:0/1826577941 shutdown_connections 2024-07-29T00:47:18.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.669+0000 7fd7f2000640 1 -- 172.21.15.175:0/1826577941 wait complete. 2024-07-29T00:47:18.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.670+0000 7fd7f2000640 1 Processor -- start 2024-07-29T00:47:18.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.670+0000 7fd7f2000640 1 -- start start 2024-07-29T00:47:18.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.670+0000 7fd7f2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 0x7fd7ec19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:18.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.670+0000 7fd7f2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ec108af0 0x7fd7ec19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.670+0000 7fd7f2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7ec10d220 0x7fd7ec1a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7f2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec1182a0 con 0x7fd7ec108140 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7f2000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec118120 con 0x7fd7ec108af0 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7f2000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec118420 con 0x7fd7ec10d220 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 0x7fd7ec19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 0x7fd7ec19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33074/0 (socket says 172.21.15.175:33074) 2024-07-29T00:47:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 -- 172.21.15.175:0/2865619656 learned_addr learned my addr 172.21.15.175:0/2865619656 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eaa00640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ec108af0 0x7fd7ec19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7ebe00640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7ec10d220 0x7fd7ec1a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 -- 172.21.15.175:0/2865619656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7ec10d220 msgr2=0x7fd7ec1a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7ec10d220 0x7fd7ec1a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 -- 172.21.15.175:0/2865619656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ec108af0 msgr2=0x7fd7ec19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ec108af0 0x7fd7ec19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.671+0000 7fd7eb400640 1 -- 172.21.15.175:0/2865619656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec1a3690 con 0x7fd7ec108140 2024-07-29T00:47:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.672+0000 7fd7eb400640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 0x7fd7ec19e6e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00d950 tx=0x7fd7dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.672+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc017030 con 0x7fd7ec108140 2024-07-29T00:47:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.672+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7dc004020 con 0x7fd7ec108140 2024-07-29T00:47:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.672+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0047a0 con 0x7fd7ec108140 2024-07-29T00:47:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.672+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ec1a3980 con 0x7fd7ec108140 2024-07-29T00:47:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.672+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ec1ab260 con 0x7fd7ec108140 2024-07-29T00:47:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.673+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec108520 con 0x7fd7ec108140 2024-07-29T00:47:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.675+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd7dc0041d0 con 0x7fd7ec108140 2024-07-29T00:47:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.677+0000 7fd7e3e00640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd7d00a2cf0 0x7fd7d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.677+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fd7dc0cee40 con 0x7fd7ec108140 2024-07-29T00:47:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.677+0000 7fd7eaa00640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd7d00a2cf0 0x7fd7d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.678+0000 7fd7eaa00640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd7d00a2cf0 0x7fd7d00a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec107c40 tx=0x7fd7d8005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.681+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd7dc09b9b0 con 0x7fd7ec108140 2024-07-29T00:47:18.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.910+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7ec19fa40 con 0x7fd7ec108140 2024-07-29T00:47:18.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.912+0000 7fd7e3e00640 1 -- 172.21.15.175:0/2865619656 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11994 (secure 0 0 0) 0x7fd7dc095e20 con 0x7fd7ec108140 2024-07-29T00:47:18.914 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:47:18.914 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":43,"btime":"2024-07-29T00:43:57:876956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:57.442709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:47:18.914 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 43 2024-07-29T00:47:18.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd7d00a2cf0 msgr2=0x7fd7d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:18.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd7d00a2cf0 0x7fd7d00a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec107c40 tx=0x7fd7d8005f90 comp rx=0 tx=0).stop 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 msgr2=0x7fd7ec19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 0x7fd7ec19e6e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00d950 tx=0x7fd7dc00de10 comp rx=0 tx=0).stop 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 shutdown_connections 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7ec10d220 0x7fd7ec1a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ec108af0 0x7fd7ec19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd7d00a2cf0 0x7fd7d00a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 --2- 172.21.15.175:0/2865619656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7ec108140 0x7fd7ec19e6e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:18.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.915+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 >> 172.21.15.175:0/2865619656 conn(0x7fd7ec077890 msgr2=0x7fd7ec079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:18.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.916+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 shutdown_connections 2024-07-29T00:47:18.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:18.916+0000 7fd7f2000640 1 -- 172.21.15.175:0/2865619656 wait complete. 2024-07-29T00:47:19.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-29T00:47:20.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:19 smithi204 ceph-mon[37368]: pgmap v337: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-07-29T00:47:20.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:19 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2865619656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:47:20.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:19 smithi181 ceph-mon[36706]: pgmap v337: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-07-29T00:47:20.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:19 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2865619656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:47:20.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:19 smithi175 ceph-mon[32138]: pgmap v337: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-07-29T00:47:20.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:19 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2865619656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:47:20.362 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:21.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.047+0000 7fa931800640 1 -- 172.21.15.175:0/2037568093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 msgr2=0x7fa92c10b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.047+0000 7fa931800640 1 --2- 172.21.15.175:0/2037568093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c10b690 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa918009900 tx=0x7fa91802a480 comp rx=0 tx=0).stop 2024-07-29T00:47:21.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.047+0000 7fa931800640 1 -- 172.21.15.175:0/2037568093 shutdown_connections 2024-07-29T00:47:21.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.048+0000 7fa931800640 1 --2- 172.21.15.175:0/2037568093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c10bcb0 0x7fa92c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.048+0000 7fa931800640 1 --2- 172.21.15.175:0/2037568093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c10b690 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.048+0000 7fa931800640 1 --2- 172.21.15.175:0/2037568093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c102880 0x7fa92c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.048+0000 7fa931800640 1 -- 172.21.15.175:0/2037568093 >> 172.21.15.175:0/2037568093 conn(0x7fa92c0fc320 msgr2=0x7fa92c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:21.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.048+0000 7fa931800640 1 -- 172.21.15.175:0/2037568093 shutdown_connections 2024-07-29T00:47:21.050 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.048+0000 7fa931800640 1 -- 172.21.15.175:0/2037568093 wait complete. 2024-07-29T00:47:21.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 Processor -- start 2024-07-29T00:47:21.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 -- start start 2024-07-29T00:47:21.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c102880 0x7fa92c19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c10bcb0 0x7fa92c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c111b00 con 0x7fa92c1031a0 2024-07-29T00:47:21.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c111980 con 0x7fa92c102880 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.049+0000 7fa931800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c111c80 con 0x7fa92c10bcb0 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58636/0 (socket says 172.21.15.175:58636) 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c102880 0x7fa92c19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 -- 172.21.15.175:0/1920653079 learned_addr learned my addr 172.21.15.175:0/1920653079 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa930e00640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c10bcb0 0x7fa92c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 -- 172.21.15.175:0/1920653079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c10bcb0 msgr2=0x7fa92c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c10bcb0 0x7fa92c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 -- 172.21.15.175:0/1920653079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c102880 msgr2=0x7fa92c19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c102880 0x7fa92c19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.050+0000 7fa92b400640 1 -- 172.21.15.175:0/1920653079 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c19f140 con 0x7fa92c1031a0 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.051+0000 7fa92b400640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c19a6d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa918004770 tx=0x7fa918003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.051+0000 7fa928c00640 1 -- 172.21.15.175:0/1920653079 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91803f070 con 0x7fa92c1031a0 2024-07-29T00:47:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.051+0000 7fa928c00640 1 -- 172.21.15.175:0/1920653079 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa918031c90 con 0x7fa92c1031a0 2024-07-29T00:47:21.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.051+0000 7fa928c00640 1 -- 172.21.15.175:0/1920653079 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918035030 con 0x7fa92c1031a0 2024-07-29T00:47:21.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.051+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c19f3d0 con 0x7fa92c1031a0 2024-07-29T00:47:21.055 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.051+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c19f860 con 0x7fa92c1031a0 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.054+0000 7fa9357fc740 1 Processor -- start 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.054+0000 7fa928c00640 1 -- 172.21.15.175:0/1920653079 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa918042070 con 0x7fa92c1031a0 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.054+0000 7fa9357fc740 1 -- start start 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d6fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d6fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d714510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa9357fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c82d664ab0 con 0x55c82d7000e0 2024-07-29T00:47:21.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa9357fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c82d5fca80 con 0x55c82d6feb70 2024-07-29T00:47:21.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa9357fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c82d660a70 con 0x55c82d712120 2024-07-29T00:47:21.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa930e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d6fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa930e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d6fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58642/0 (socket says 172.21.15.175:58642) 2024-07-29T00:47:21.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa930e00640 1 -- 172.21.15.175:0/3446935193 learned_addr learned my addr 172.21.15.175:0/3446935193 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa92b400640 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d714510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.055+0000 7fa92be00640 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d6fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa930e00640 1 -- 172.21.15.175:0/3446935193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 msgr2=0x55c82d714510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa930e00640 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d714510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa930e00640 1 -- 172.21.15.175:0/3446935193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 msgr2=0x55c82d6fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa930e00640 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d6fefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa930e00640 1 -- 172.21.15.175:0/3446935193 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82d3d63c0 con 0x55c82d7000e0 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa928c00640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa9100a2d70 0x7fa9100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa930e00640 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d6fe630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa924009f50 tx=0x7fa9240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5e049fa38f0d4e server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:21.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.056+0000 7fa928c00640 1 -- 172.21.15.175:0/1920653079 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fa9180ef9c0 con 0x7fa92c1031a0 2024-07-29T00:47:21.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa921600640 1 -- 172.21.15.175:0/3446935193 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924004d20 con 0x55c82d7000e0 2024-07-29T00:47:21.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa92be00640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa9100a2d70 0x7fa9100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa921600640 1 -- 172.21.15.175:0/3446935193 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924004050 con 0x55c82d7000e0 2024-07-29T00:47:21.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa921600640 1 -- 172.21.15.175:0/3446935193 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924005280 con 0x55c82d7000e0 2024-07-29T00:47:21.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa9357fc740 1 -- 172.21.15.175:0/3446935193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 msgr2=0x55c82d6fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa92be00640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa9100a2d70 0x7fa9100a5230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55c82d6fc060 tx=0x7fa91c005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:21.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.057+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d6fe630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa924009f50 tx=0x7fa9240118d0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 -- 172.21.15.175:0/3446935193 shutdown_connections 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d714510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d6fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3446935193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d6fe630 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 -- 172.21.15.175:0/3446935193 >> 172.21.15.175:0/3446935193 conn(0x55c82d6c7950 msgr2=0x55c82d6fb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 -- 172.21.15.175:0/3446935193 shutdown_connections 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 -- 172.21.15.175:0/3446935193 wait complete. 2024-07-29T00:47:21.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.058+0000 7fa9357fc740 1 -- start start 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 Processor -- start 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 0x55c82d70f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d70f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d712120 0x55c82d70fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c82d714ec0 con 0x55c82d7000e0 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c82d61fc10 con 0x55c82d712120 2024-07-29T00:47:21.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c82d715040 con 0x55c82d6feb70 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa9357fc740 10 client.0 _ll_register_callbacks cb 0x55c82d5e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa92be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 0x55c82d70f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa92be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 0x55c82d70f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49088/0 (socket says 172.21.15.175:49088) 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.059+0000 7fa92be00640 1 -- 172.21.15.175:0/3823606449 learned_addr learned my addr 172.21.15.175:0/3823606449 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92b400640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d70f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa930e00640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d712120 0x55c82d70fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.067 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92be00640 1 -- 172.21.15.175:0/3823606449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d712120 msgr2=0x55c82d70fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92be00640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d712120 0x55c82d70fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92be00640 1 -- 172.21.15.175:0/3823606449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 msgr2=0x55c82d70f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92be00640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d70f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92be00640 1 -- 172.21.15.175:0/3823606449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82d6231e0 con 0x55c82d6feb70 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92b400640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d70f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa92be00640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 0x55c82d70f060 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009730 tx=0x7fa91c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62f09061b3a6bfb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa916000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.060+0000 7fa916000640 1 -- 172.21.15.175:0/3823606449 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c01f640 con 0x55c82d6feb70 2024-07-29T00:47:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa924004050 con 0x55c82d6feb70 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa916000640 1 -- 172.21.15.175:0/3823606449 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fa91c01f7e0 con 0x55c82d6feb70 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa916000640 1 -- 172.21.15.175:0/3823606449 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa91c01f9e0 con 0x55c82d6feb70 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 10 client.27439 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c82d720cc0 con 0x55c82d6feb70 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa916000640 1 -- 172.21.15.175:0/3823606449 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924004050 con 0x55c82d6feb70 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa916000640 1 -- 172.21.15.175:0/3823606449 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fa91c027070 con 0x55c82d6feb70 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 10 client.27439 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:21.069 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 10 client.27439 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 20 client.27439 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d722120 0x55c82d724590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 4 client.27439 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.061+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82d672020 con 0x55c82d722120 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.062+0000 7fa92b400640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d722120 0x55c82d724590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.062+0000 7fa92b400640 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d722120 0x55c82d724590 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa95dd047f6decf8 server_cookie=6e90407046adebad in_seq=0 out_seq=0 2024-07-29T00:47:21.070 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.062+0000 7fa916000640 10 client.27439 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa916000640 1 -- 172.21.15.175:0/3823606449 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c82d672020 con 0x55c82d722120 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa916000640 10 client.27439 handle_command_reply: tid=0 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 client.27439 shutdown 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 msgr2=0x55c82d70f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 0x55c82d70f060 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009730 tx=0x7fa91c004730 comp rx=0 tx=0).stop 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 shutdown_connections 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d722120 0x55c82d724590 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.071 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d712120 0x55c82d70fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d70f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 --2- 172.21.15.175:0/3823606449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d6feb70 0x55c82d70f060 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa916000640 0 client.27439 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.063+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 >> 172.21.15.175:0/3823606449 conn(0x55c82d6c7950 msgr2=0x55c82d700ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.064+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 shutdown_connections 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.064+0000 7fa9357fc740 1 -- 172.21.15.175:0/3823606449 wait complete. 2024-07-29T00:47:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.064+0000 7fa9357fc740 20 client.27439 trim_cache size 0 max 16384 2024-07-29T00:47:21.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.078+0000 7fa9357fc740 1 Processor -- start 2024-07-29T00:47:21.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.078+0000 7fa9357fc740 1 -- start start 2024-07-29T00:47:21.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d729e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 0x55c82d72a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d712120 0x55c82d6f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa9357fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1ab3b0 con 0x55c82d712120 2024-07-29T00:47:21.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa9357fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1acef0 con 0x55c82d6feb70 2024-07-29T00:47:21.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa9357fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1aba80 con 0x55c82d7000e0 2024-07-29T00:47:21.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa930e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d712120 0x55c82d6f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 0x55c82d72a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 0x55c82d72a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49090/0 (socket says 172.21.15.175:49090) 2024-07-29T00:47:21.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 -- 172.21.15.175:0/1667280585 learned_addr learned my addr 172.21.15.175:0/1667280585 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92be00640 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d729e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 -- 172.21.15.175:0/1667280585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 msgr2=0x55c82d729e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d729e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 -- 172.21.15.175:0/1667280585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d712120 msgr2=0x55c82d6f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d712120 0x55c82d6f9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.079+0000 7fa92b400640 1 -- 172.21.15.175:0/1667280585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82d6fc1a0 con 0x55c82d7000e0 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.080+0000 7fa92b400640 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 0x55c82d72a3a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9180bbf30 tx=0x7fa9180f54d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4dad68f2310ef38 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:21.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.080+0000 7fa920c00640 1 -- 172.21.15.175:0/1667280585 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918049050 con 0x55c82d7000e0 2024-07-29T00:47:21.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.080+0000 7fa920c00640 1 -- 172.21.15.175:0/1667280585 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9180f5030 con 0x55c82d7000e0 2024-07-29T00:47:21.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.080+0000 7fa920c00640 1 -- 172.21.15.175:0/1667280585 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180c4b60 con 0x55c82d7000e0 2024-07-29T00:47:21.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 -- 172.21.15.175:0/1667280585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 msgr2=0x55c82d72a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 0x55c82d72a3a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9180bbf30 tx=0x7fa9180f54d0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 -- 172.21.15.175:0/1667280585 shutdown_connections 2024-07-29T00:47:21.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d712120 0x55c82d6f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d7000e0 0x55c82d72a3a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 --2- 172.21.15.175:0/1667280585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d729e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 -- 172.21.15.175:0/1667280585 >> 172.21.15.175:0/1667280585 conn(0x55c82d6c7950 msgr2=0x55c82d6f6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:21.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 -- 172.21.15.175:0/1667280585 shutdown_connections 2024-07-29T00:47:21.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.081+0000 7fa9357fc740 1 -- 172.21.15.175:0/1667280585 wait complete. 2024-07-29T00:47:21.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 -- start start 2024-07-29T00:47:21.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 Processor -- start 2024-07-29T00:47:21.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d71ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d715ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d7164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1ac820 con 0x55c82d7000e0 2024-07-29T00:47:21.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1ac150 con 0x55c82d6feb70 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1a93f0 con 0x55c82d712120 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.082+0000 7fa9357fc740 10 client.0 _ll_register_callbacks cb 0x7fa91c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d71ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d715ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d71ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58086/0 (socket says 172.21.15.175:58086) 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa930e00640 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d7164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.092 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 -- 172.21.15.175:0/610677028 learned_addr learned my addr 172.21.15.175:0/610677028 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 -- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 msgr2=0x55c82d7164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d7164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 -- 172.21.15.175:0/610677028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 msgr2=0x55c82d715ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d715ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 -- 172.21.15.175:0/610677028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82d716cd0 con 0x55c82d6feb70 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa92be00640 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d71ab50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa91c029e50 tx=0x7fa91c029670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03fe6db259d8e48 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.083+0000 7fa916000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:47:21.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa916000640 1 -- 172.21.15.175:0/610677028 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c003e50 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9180f5030 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa916000640 1 -- 172.21.15.175:0/610677028 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fa91c029860 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa916000640 1 -- 172.21.15.175:0/610677028 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa91c029a60 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa9357fc740 10 client.27506 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c82d78cf30 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.084+0000 7fa916000640 1 -- 172.21.15.175:0/610677028 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9180f5030 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa916000640 1 -- 172.21.15.175:0/610677028 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fa91c038030 con 0x55c82d6feb70 2024-07-29T00:47:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa9357fc740 10 client.27506 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:21.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa9357fc740 10 client.27506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-29T00:47:21.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa9357fc740 20 client.27506 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:21.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa9357fc740 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d78d680 0x55c82d78faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:21.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa9357fc740 4 client.27506 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:47:21.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 --> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c82d66c530 con 0x55c82d78d680 2024-07-29T00:47:21.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.085+0000 7fa92b400640 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d78d680 0x55c82d78faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:21.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa92b400640 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d78d680 0x55c82d78faa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d49395c6f87571e server_cookie=a953bd564c1a77bc in_seq=0 out_seq=0 2024-07-29T00:47:21.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa916000640 10 client.27506 ms_handle_connect on v2:172.21.15.204:6836/2487746199 2024-07-29T00:47:21.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa916000640 1 -- 172.21.15.175:0/610677028 <== mds.4 v2:172.21.15.204:6836/2487746199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c82d66c530 con 0x55c82d78d680 2024-07-29T00:47:21.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa916000640 10 client.27506 handle_command_reply: tid=0 2024-07-29T00:47:21.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 client.27506 shutdown 2024-07-29T00:47:21.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 msgr2=0x55c82d71ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d71ab50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa91c029e50 tx=0x7fa91c029670 comp rx=0 tx=0).stop 2024-07-29T00:47:21.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 shutdown_connections 2024-07-29T00:47:21.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c82d712120 0x55c82d7164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82d7000e0 0x55c82d715ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.204:6836/2487746199,v1:172.21.15.204:6837/2487746199] conn(0x55c82d78d680 0x55c82d78faa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 --2- 172.21.15.175:0/610677028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c82d6feb70 0x55c82d71ab50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa916000640 0 client.27506 ms_handle_reset on v2:172.21.15.204:6836/2487746199 2024-07-29T00:47:21.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 >> 172.21.15.175:0/610677028 conn(0x55c82d6c7950 msgr2=0x55c82d6f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:21.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.086+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 shutdown_connections 2024-07-29T00:47:21.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.087+0000 7fa9357fc740 1 -- 172.21.15.175:0/610677028 wait complete. 2024-07-29T00:47:21.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.087+0000 7fa9357fc740 20 client.27506 trim_cache size 0 max 16384 2024-07-29T00:47:21.100 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:47:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.087+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa9100a2d70 msgr2=0x7fa9100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.087+0000 7fa931800640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa9100a2d70 0x7fa9100a5230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55c82d6fc060 tx=0x7fa91c005f90 comp rx=0 tx=0).stop 2024-07-29T00:47:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.087+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 msgr2=0x7fa92c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.087+0000 7fa931800640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c19a6d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa918004770 tx=0x7fa918003b60 comp rx=0 tx=0).stop 2024-07-29T00:47:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 shutdown_connections 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c10bcb0 0x7fa92c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa9100a2d70 0x7fa9100a5230 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa92c1031a0 0x7fa92c19a6d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 --2- 172.21.15.175:0/1920653079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c102880 0x7fa92c19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 >> 172.21.15.175:0/1920653079 conn(0x7fa92c0fc320 msgr2=0x7fa92c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 shutdown_connections 2024-07-29T00:47:21.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:21.088+0000 7fa931800640 1 -- 172.21.15.175:0/1920653079 wait complete. 2024-07-29T00:47:22.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:21 smithi204 ceph-mon[37368]: pgmap v338: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:22.108 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:21 smithi175 ceph-mon[32138]: pgmap v338: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:22.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-29T00:47:22.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:21 smithi181 ceph-mon[36706]: pgmap v338: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-29T00:47:22.565 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:23.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 -- 172.21.15.175:0/756397386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102040 msgr2=0x7f455410e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 --2- 172.21.15.175:0/756397386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102040 0x7f455410e570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f454c009c10 tx=0x7f454c02a6e0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 -- 172.21.15.175:0/756397386 shutdown_connections 2024-07-29T00:47:23.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 --2- 172.21.15.175:0/756397386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102040 0x7f455410e570 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 --2- 172.21.15.175:0/756397386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45541016a0 0x7f4554101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 --2- 172.21.15.175:0/756397386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45541076a0 0x7f4554107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 -- 172.21.15.175:0/756397386 >> 172.21.15.175:0/756397386 conn(0x7f45540fd3a0 msgr2=0x7f45540ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:23.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 -- 172.21.15.175:0/756397386 shutdown_connections 2024-07-29T00:47:23.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.255+0000 7f4553200640 1 -- 172.21.15.175:0/756397386 wait complete. 2024-07-29T00:47:23.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 Processor -- start 2024-07-29T00:47:23.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 -- start start 2024-07-29T00:47:23.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 0x7f455419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4554102040 0x7f455419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45541076a0 0x7f455419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111aa0 con 0x7f4554102040 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111920 con 0x7f45541076a0 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.256+0000 7f4553200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111c20 con 0x7f45541016a0 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 0x7f455419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4554102040 0x7f455419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 0x7f455419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49102/0 (socket says 172.21.15.175:49102) 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 -- 172.21.15.175:0/2099243453 learned_addr learned my addr 172.21.15.175:0/2099243453 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4552800640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45541076a0 0x7f455419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 -- 172.21.15.175:0/2099243453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45541076a0 msgr2=0x7f455419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45541076a0 0x7f455419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 -- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4554102040 msgr2=0x7f455419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4554102040 0x7f455419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551e00640 1 -- 172.21.15.175:0/2099243453 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419f100 con 0x7f45541016a0 2024-07-29T00:47:23.261 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.257+0000 7f4551400640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4554102040 0x7f455419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:23.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.258+0000 7f4551e00640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 0x7f455419a150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f454400e9c0 tx=0x7f454400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:23.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.258+0000 7f454aa00640 1 -- 172.21.15.175:0/2099243453 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454400ccc0 con 0x7f45541016a0 2024-07-29T00:47:23.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.258+0000 7f454aa00640 1 -- 172.21.15.175:0/2099243453 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544005500 con 0x7f45541016a0 2024-07-29T00:47:23.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.258+0000 7f454aa00640 1 -- 172.21.15.175:0/2099243453 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544010430 con 0x7f45541016a0 2024-07-29T00:47:23.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.258+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419f3f0 con 0x7f45541016a0 2024-07-29T00:47:23.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.258+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a6cd0 con 0x7f45541016a0 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.262+0000 7f455ba69740 1 Processor -- start 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.262+0000 7f455ba69740 1 -- start start 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f455ba69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e66f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f455ba69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e66ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f455ba69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 0x56071e67cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f455ba69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56071e5d5ab0 con 0x56071e66fb70 2024-07-29T00:47:23.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f455ba69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56071e56da80 con 0x56071e6710e0 2024-07-29T00:47:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f455ba69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56071e5d1a70 con 0x56071e67a910 2024-07-29T00:47:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f454aa00640 1 -- 172.21.15.175:0/2099243453 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f45440046c0 con 0x7f45541016a0 2024-07-29T00:47:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4552800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e66f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4551400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 0x56071e67cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4552800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e66f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58114/0 (socket says 172.21.15.175:58114) 2024-07-29T00:47:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4551400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 0x56071e67cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49112/0 (socket says 172.21.15.175:49112) 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4552800640 1 -- 172.21.15.175:0/1612912482 learned_addr learned my addr 172.21.15.175:0/1612912482 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4551e00640 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e66ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.263+0000 7f4551400640 1 -- 172.21.15.175:0/1612912482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 msgr2=0x56071e66f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f4551400640 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e66f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f4551400640 1 -- 172.21.15.175:0/1612912482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 msgr2=0x56071e66ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f4551400640 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e66ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f4551400640 1 -- 172.21.15.175:0/1612912482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071e3473c0 con 0x56071e67a910 2024-07-29T00:47:23.266 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f4551e00640 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e66ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-29T00:47:23.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f4551400640 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 0x56071e67cd00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4540009fa0 tx=0x7f45400118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c5728b8cb1c8e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:23.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f453f400640 1 -- 172.21.15.175:0/1612912482 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540004030 con 0x56071e67a910 2024-07-29T00:47:23.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f453f400640 1 -- 172.21.15.175:0/1612912482 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4540005500 con 0x56071e67a910 2024-07-29T00:47:23.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.264+0000 7f453f400640 1 -- 172.21.15.175:0/1612912482 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540004d50 con 0x56071e67a910 2024-07-29T00:47:23.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 -- 172.21.15.175:0/1612912482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 msgr2=0x56071e67cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 0x56071e67cd00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4540009fa0 tx=0x7f45400118d0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f454aa00640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45380a2d70 0x7f45380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 -- 172.21.15.175:0/1612912482 shutdown_connections 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e67a910 0x56071e67cd00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e66ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f454aa00640 1 -- 172.21.15.175:0/2099243453 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f45440cf660 con 0x7f45541016a0 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 --2- 172.21.15.175:0/1612912482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e66f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 -- 172.21.15.175:0/1612912482 >> 172.21.15.175:0/1612912482 conn(0x56071e638950 msgr2=0x56071e66c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f4552800640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45380a2d70 0x7f45380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.265+0000 7f455ba69740 1 -- 172.21.15.175:0/1612912482 shutdown_connections 2024-07-29T00:47:23.272 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.266+0000 7f455ba69740 1 -- 172.21.15.175:0/1612912482 wait complete. 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.266+0000 7f4552800640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45380a2d70 0x7f45380a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f454c0099a0 tx=0x7f454c005fd0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.266+0000 7f455ba69740 1 -- start start 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.266+0000 7f455ba69740 1 Processor -- start 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e677860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e677da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e67a910 0x56071e6782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.273 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56071e67d6b0 con 0x56071e66fb70 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56071e590c10 con 0x56071e67a910 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56071e67d830 con 0x56071e6710e0 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f455ba69740 10 client.0 _ll_register_callbacks cb 0x56071e559a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f4551e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e677860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f4551e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e677860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58706/0 (socket says 172.21.15.175:58706) 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f4551e00640 1 -- 172.21.15.175:0/2936006933 learned_addr learned my addr 172.21.15.175:0/2936006933 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f4551400640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e677da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.274 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.267+0000 7f4552800640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e67a910 0x56071e6782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551400640 1 -- 172.21.15.175:0/2936006933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e67a910 msgr2=0x56071e6782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551400640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e67a910 0x56071e6782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551400640 1 -- 172.21.15.175:0/2936006933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 msgr2=0x56071e677860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551400640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e677860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551400640 1 -- 172.21.15.175:0/2936006933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071e5941e0 con 0x56071e6710e0 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4552800640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e67a910 0x56071e6782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:23.275 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551e00640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e677860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4551400640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e677da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4540004600 tx=0x7f454000fad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd85b43fb3e6b0b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4533e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4533e00640 1 -- 172.21.15.175:0/2936006933 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540027070 con 0x56071e6710e0 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4540005500 con 0x56071e6710e0 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.268+0000 7f4533e00640 1 -- 172.21.15.175:0/2936006933 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f4540004a50 con 0x56071e6710e0 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f4533e00640 1 -- 172.21.15.175:0/2936006933 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4540032050 con 0x56071e6710e0 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f455ba69740 10 client.27457 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:23.276 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56071e689370 con 0x56071e6710e0 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f4533e00640 1 -- 172.21.15.175:0/2936006933 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4540005500 con 0x56071e6710e0 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f4533e00640 1 -- 172.21.15.175:0/2936006933 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f4540022460 con 0x56071e6710e0 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f455ba69740 10 client.27457 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f455ba69740 10 client.27457 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f455ba69740 20 client.27457 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.269+0000 7f455ba69740 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e68cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.270+0000 7f455ba69740 4 client.27457 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:47:23.277 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.270+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56071e5e3020 con 0x56071e68a780 2024-07-29T00:47:23.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.270+0000 7f4551e00640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e68cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.270+0000 7f4551e00640 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e68cb70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61f1e47717d69c1c server_cookie=e7b73eb3c011ba34 in_seq=0 out_seq=0 2024-07-29T00:47:23.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.270+0000 7f4533e00640 10 client.27457 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:47:23.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.271+0000 7f4533e00640 1 -- 172.21.15.175:0/2936006933 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56071e5e3020 con 0x56071e68a780 2024-07-29T00:47:23.279 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.271+0000 7f4533e00640 10 client.27457 handle_command_reply: tid=0 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.271+0000 7f455ba69740 1 client.27457 shutdown 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 msgr2=0x56071e677da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e677da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4540004600 tx=0x7f454000fad0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 shutdown_connections 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e68cb70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e67a910 0x56071e6782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f4533e00640 0 client.27457 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:47:23.280 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e677da0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 --2- 172.21.15.175:0/2936006933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e66fb70 0x56071e677860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 >> 172.21.15.175:0/2936006933 conn(0x56071e638950 msgr2=0x56071e671f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:23.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 shutdown_connections 2024-07-29T00:47:23.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 1 -- 172.21.15.175:0/2936006933 wait complete. 2024-07-29T00:47:23.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.272+0000 7f455ba69740 20 client.27457 trim_cache size 0 max 16384 2024-07-29T00:47:23.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.286+0000 7f455ba69740 1 Processor -- start 2024-07-29T00:47:23.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.286+0000 7f455ba69740 1 -- start start 2024-07-29T00:47:23.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f455ba69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e66fb70 0x56071e672bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f455ba69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e692260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f455ba69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e6927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f455ba69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45541af7d0 con 0x56071e67a910 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f455ba69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b1310 con 0x56071e6710e0 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f455ba69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45541afea0 con 0x56071e66fb70 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f4552800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e6927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f4551e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e66fb70 0x56071e672bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f4551e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e66fb70 0x56071e672bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49128/0 (socket says 172.21.15.175:49128) 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f4551e00640 1 -- 172.21.15.175:0/2190998353 learned_addr learned my addr 172.21.15.175:0/2190998353 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:23.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.287+0000 7f4551400640 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e692260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4552800640 1 -- 172.21.15.175:0/2190998353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e66fb70 msgr2=0x56071e672bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4552800640 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e66fb70 0x56071e672bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4552800640 1 -- 172.21.15.175:0/2190998353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 msgr2=0x56071e692260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4552800640 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e692260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4552800640 1 -- 172.21.15.175:0/2190998353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071e6702f0 con 0x56071e67a910 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4551400640 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e692260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:23.290 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f4552800640 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e6927a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f454c009b30 tx=0x7f454c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b05c0a82140fc1d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:23.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f453ea00640 1 -- 172.21.15.175:0/2190998353 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c036040 con 0x56071e67a910 2024-07-29T00:47:23.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f453ea00640 1 -- 172.21.15.175:0/2190998353 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c031b20 con 0x56071e67a910 2024-07-29T00:47:23.294 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.288+0000 7f453ea00640 1 -- 172.21.15.175:0/2190998353 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c03ebe0 con 0x56071e67a910 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 -- 172.21.15.175:0/2190998353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 msgr2=0x56071e6927a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e6927a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f454c009b30 tx=0x7f454c040000 comp rx=0 tx=0).stop 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 -- 172.21.15.175:0/2190998353 shutdown_connections 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e6927a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e6710e0 0x56071e692260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 --2- 172.21.15.175:0/2190998353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e66fb70 0x56071e672bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 -- 172.21.15.175:0/2190998353 >> 172.21.15.175:0/2190998353 conn(0x56071e638950 msgr2=0x56071e676950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:23.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 -- 172.21.15.175:0/2190998353 shutdown_connections 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.289+0000 7f455ba69740 1 -- 172.21.15.175:0/2190998353 wait complete. 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 -- start start 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 Processor -- start 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e66fb70 0x56071e683120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e674450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e674990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b0c40 con 0x56071e67a910 2024-07-29T00:47:23.296 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b0570 con 0x56071e66fb70 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ad7d0 con 0x56071e6710e0 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f455ba69740 10 client.0 _ll_register_callbacks cb 0x7f4544005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f4551400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e674450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.290+0000 7f4552800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e674990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e674450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49144/0 (socket says 172.21.15.175:49144) 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 -- 172.21.15.175:0/1962199217 learned_addr learned my addr 172.21.15.175:0/1962199217 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:23.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551e00640 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e66fb70 0x56071e683120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 -- 172.21.15.175:0/1962199217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e66fb70 msgr2=0x56071e683120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e66fb70 0x56071e683120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 -- 172.21.15.175:0/1962199217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 msgr2=0x56071e674990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e674990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 -- 172.21.15.175:0/1962199217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071e675030 con 0x56071e6710e0 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4551400640 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e674450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45400229d0 tx=0x7f45400059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946ef3ffaef33ab server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4533e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:47:23.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4533e00640 1 -- 172.21.15.175:0/1962199217 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540027050 con 0x56071e6710e0 2024-07-29T00:47:23.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c031b20 con 0x56071e6710e0 2024-07-29T00:47:23.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.291+0000 7f4533e00640 1 -- 172.21.15.175:0/1962199217 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f4540004630 con 0x56071e6710e0 2024-07-29T00:47:23.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f4533e00640 1 -- 172.21.15.175:0/1962199217 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f454001d040 con 0x56071e6710e0 2024-07-29T00:47:23.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 10 client.27463 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:23.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56071e6f50c0 con 0x56071e6710e0 2024-07-29T00:47:23.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f4533e00640 1 -- 172.21.15.175:0/1962199217 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c031b20 con 0x56071e6710e0 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f4533e00640 1 -- 172.21.15.175:0/1962199217 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f454003ea40 con 0x56071e6710e0 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 10 client.27463 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 10 client.27463 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 20 client.27463 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e6f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 4 client.27463 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 --> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56071e5dd530 con 0x56071e68a780 2024-07-29T00:47:23.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.292+0000 7f4551e00640 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e6f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f4551e00640 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e6f7750 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be1e352ae9f32fa2 server_cookie=2dcd66be58f3e0d9 in_seq=0 out_seq=0 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f4533e00640 10 client.27463 ms_handle_connect on v2:172.21.15.204:6838/2455267021 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f4533e00640 1 -- 172.21.15.175:0/1962199217 <== mds.3 v2:172.21.15.204:6838/2455267021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56071e5dd530 con 0x56071e68a780 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f4533e00640 10 client.27463 handle_command_reply: tid=0 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 client.27463 shutdown 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 msgr2=0x56071e674450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e674450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45400229d0 tx=0x7f45400059b0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 shutdown_connections 2024-07-29T00:47:23.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:6838/2455267021,v1:172.21.15.204:6839/2455267021] conn(0x56071e68a780 0x56071e6f7750 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56071e67a910 0x56071e674990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56071e6710e0 0x56071e674450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 --2- 172.21.15.175:0/1962199217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56071e66fb70 0x56071e683120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f4533e00640 0 client.27463 ms_handle_reset on v2:172.21.15.204:6838/2455267021 2024-07-29T00:47:23.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.293+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 >> 172.21.15.175:0/1962199217 conn(0x56071e638950 msgr2=0x56071e6716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:23.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 shutdown_connections 2024-07-29T00:47:23.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f455ba69740 1 -- 172.21.15.175:0/1962199217 wait complete. 2024-07-29T00:47:23.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f455ba69740 20 client.27463 trim_cache size 0 max 16384 2024-07-29T00:47:23.303 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:47:23.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45380a2d70 msgr2=0x7f45380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f4553200640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45380a2d70 0x7f45380a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f454c0099a0 tx=0x7f454c005fd0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 msgr2=0x7f455419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:23.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.294+0000 7f4553200640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 0x7f455419a150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f454400e9c0 tx=0x7f454400ee80 comp rx=0 tx=0).stop 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 shutdown_connections 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45541076a0 0x7f455419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f45380a2d70 0x7f45380a5230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4554102040 0x7f455419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 --2- 172.21.15.175:0/2099243453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45541016a0 0x7f455419a150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 >> 172.21.15.175:0/2099243453 conn(0x7f45540fd3a0 msgr2=0x7f455410ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 shutdown_connections 2024-07-29T00:47:23.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:23.295+0000 7f4553200640 1 -- 172.21.15.175:0/2099243453 wait complete. 2024-07-29T00:47:24.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:23 smithi175 ceph-mon[32138]: pgmap v339: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:47:24.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:23 smithi204 ceph-mon[37368]: pgmap v339: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:47:24.228 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-29T00:47:24.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:23 smithi181 ceph-mon[36706]: pgmap v339: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:47:24.687 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:25.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 -- 172.21.15.175:0/1433276694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 msgr2=0x7ff09410caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 --2- 172.21.15.175:0/1433276694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 0x7ff09410caf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002a3f0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 -- 172.21.15.175:0/1433276694 shutdown_connections 2024-07-29T00:47:25.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 --2- 172.21.15.175:0/1433276694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09410d220 0x7ff094114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 --2- 172.21.15.175:0/1433276694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 0x7ff09410caf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.370 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 --2- 172.21.15.175:0/1433276694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff094108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 -- 172.21.15.175:0/1433276694 >> 172.21.15.175:0/1433276694 conn(0x7ff094077890 msgr2=0x7ff094079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:25.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 -- 172.21.15.175:0/1433276694 shutdown_connections 2024-07-29T00:47:25.371 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.369+0000 7ff09c200640 1 -- 172.21.15.175:0/1433276694 wait complete. 2024-07-29T00:47:25.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 Processor -- start 2024-07-29T00:47:25.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 -- start start 2024-07-29T00:47:25.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff09419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 0x7ff09419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09410d220 0x7ff0941a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff094118270 con 0x7ff094108af0 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941180f0 con 0x7ff09410d220 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.371+0000 7ff09c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941183f0 con 0x7ff094108140 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff09419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff09419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49152/0 (socket says 172.21.15.175:49152) 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 -- 172.21.15.175:0/1927766193 learned_addr learned my addr 172.21.15.175:0/1927766193 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff098e00640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 0x7ff09419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff09a200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09410d220 0x7ff0941a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 -- 172.21.15.175:0/1927766193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09410d220 msgr2=0x7ff0941a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09410d220 0x7ff0941a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 -- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 msgr2=0x7ff09419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 0x7ff09419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 -- 172.21.15.175:0/1927766193 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0941a36a0 con 0x7ff094108140 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.372+0000 7ff099800640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff09419e6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff08800d950 tx=0x7ff08800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.373+0000 7ff092000640 1 -- 172.21.15.175:0/1927766193 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088017030 con 0x7ff094108140 2024-07-29T00:47:25.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.373+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0941a3990 con 0x7ff094108140 2024-07-29T00:47:25.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.373+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941ab270 con 0x7ff094108140 2024-07-29T00:47:25.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.373+0000 7ff092000640 1 -- 172.21.15.175:0/1927766193 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff088004020 con 0x7ff094108140 2024-07-29T00:47:25.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.373+0000 7ff092000640 1 -- 172.21.15.175:0/1927766193 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880047a0 con 0x7ff094108140 2024-07-29T00:47:25.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.376+0000 7ff09ebcb740 1 Processor -- start 2024-07-29T00:47:25.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.376+0000 7ff092000640 1 -- 172.21.15.175:0/1927766193 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff0880041d0 con 0x7ff094108140 2024-07-29T00:47:25.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 -- start start 2024-07-29T00:47:25.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x55954686c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x55954686cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x559546879b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595467d2ab0 con 0x55954686c980 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55954676aa80 con 0x55954686def0 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595467cea70 con 0x559546877720 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff099800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x55954686cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff09a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x55954686c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff099800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x55954686cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58740/0 (socket says 172.21.15.175:58740) 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff099800640 1 -- 172.21.15.175:0/1739830470 learned_addr learned my addr 172.21.15.175:0/1739830470 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.377+0000 7ff098e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x559546879b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff09a200640 1 -- 172.21.15.175:0/1739830470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 msgr2=0x559546879b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff09a200640 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x559546879b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff09a200640 1 -- 172.21.15.175:0/1739830470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 msgr2=0x55954686cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff09a200640 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x55954686cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff09a200640 1 -- 172.21.15.175:0/1739830470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595465443c0 con 0x55954686def0 2024-07-29T00:47:25.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff09a200640 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x55954686c440 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009ef0 tx=0x7ff08c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ddad5b2dcd171d server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:25.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff092000640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff07c0a2d70 0x7ff07c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.378+0000 7ff086a00640 1 -- 172.21.15.175:0/1739830470 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004d20 con 0x55954686def0 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff092000640 1 -- 172.21.15.175:0/1927766193 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff0880ced50 con 0x7ff094108140 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff099800640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff07c0a2d70 0x7ff07c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff086a00640 1 -- 172.21.15.175:0/1739830470 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c004050 con 0x55954686def0 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff086a00640 1 -- 172.21.15.175:0/1739830470 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c005260 con 0x55954686def0 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff09ebcb740 1 -- 172.21.15.175:0/1739830470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 msgr2=0x55954686c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff099800640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff07c0a2d70 0x7ff07c0a5230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff088009610 tx=0x7ff0880d2800 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.379+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x55954686c440 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009ef0 tx=0x7ff08c011810 comp rx=0 tx=0).stop 2024-07-29T00:47:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 -- 172.21.15.175:0/1739830470 shutdown_connections 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x559546879b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x55954686cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/1739830470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x55954686c440 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 -- 172.21.15.175:0/1739830470 >> 172.21.15.175:0/1739830470 conn(0x559546835950 msgr2=0x559546869010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 -- 172.21.15.175:0/1739830470 shutdown_connections 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 -- 172.21.15.175:0/1739830470 wait complete. 2024-07-29T00:47:25.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.380+0000 7ff09ebcb740 1 -- start start 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 Processor -- start 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x559546874640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546874b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x5595468750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55954687a4c0 con 0x55954686c980 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55954678dc10 con 0x55954686def0 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55954687a640 con 0x559546877720 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09ebcb740 10 client.0 _ll_register_callbacks cb 0x559546756a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:25.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff09a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x559546874640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.381+0000 7ff098e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546874b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546874b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:58164/0 (socket says 172.21.15.175:58164) 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 -- 172.21.15.175:0/725582007 learned_addr learned my addr 172.21.15.175:0/725582007 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff09a200640 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x5595468750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 -- 172.21.15.175:0/725582007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 msgr2=0x5595468750c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x5595468750c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 -- 172.21.15.175:0/725582007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 msgr2=0x559546874640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x559546874640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 -- 172.21.15.175:0/725582007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595467911e0 con 0x55954686def0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff098e00640 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546874b80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff080033000 tx=0x7ff080004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9a816ffee91dc3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.382+0000 7ff07b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff07b400640 1 -- 172.21.15.175:0/725582007 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080035040 con 0x55954686def0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff07b400640 1 -- 172.21.15.175:0/725582007 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff080005450 con 0x55954686def0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c004050 con 0x55954686def0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff07b400640 1 -- 172.21.15.175:0/725582007 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff080005650 con 0x55954686def0 2024-07-29T00:47:25.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff09ebcb740 10 client.27536 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559546886270 con 0x55954686def0 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff07b400640 1 -- 172.21.15.175:0/725582007 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c004050 con 0x55954686def0 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.383+0000 7ff07b400640 1 -- 172.21.15.175:0/725582007 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff080031aa0 con 0x55954686def0 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09ebcb740 10 client.27536 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09ebcb740 10 client.27536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09ebcb740 20 client.27536 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:25.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559546887730 0x559546889b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09ebcb740 4 client.27536 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:47:25.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595467e0020 con 0x559546887730 2024-07-29T00:47:25.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.384+0000 7ff09a200640 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559546887730 0x559546889b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.385+0000 7ff09a200640 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559546887730 0x559546889b90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2103405e35131a3 server_cookie=934174638f7cf1da in_seq=0 out_seq=0 2024-07-29T00:47:25.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.385+0000 7ff07b400640 10 client.27536 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:47:25.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.385+0000 7ff07b400640 1 -- 172.21.15.175:0/725582007 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595467e0020 con 0x559546887730 2024-07-29T00:47:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.385+0000 7ff07b400640 10 client.27536 handle_command_reply: tid=0 2024-07-29T00:47:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.385+0000 7ff09ebcb740 1 client.27536 shutdown 2024-07-29T00:47:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 msgr2=0x559546874b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546874b80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff080033000 tx=0x7ff080004420 comp rx=0 tx=0).stop 2024-07-29T00:47:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 shutdown_connections 2024-07-29T00:47:25.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559546877720 0x5595468750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x559546887730 0x559546889b90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546874b80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/725582007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686c980 0x559546874640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff07b400640 0 client.27536 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 >> 172.21.15.175:0/725582007 conn(0x559546835950 msgr2=0x55954686ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 shutdown_connections 2024-07-29T00:47:25.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 1 -- 172.21.15.175:0/725582007 wait complete. 2024-07-29T00:47:25.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.386+0000 7ff09ebcb740 20 client.27536 trim_cache size 0 max 16384 2024-07-29T00:47:25.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 Processor -- start 2024-07-29T00:47:25.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 -- start start 2024-07-29T00:47:25.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x5595468765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686def0 0x55954688f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559546877720 0x55954688f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400e8e0 con 0x55954686def0 2024-07-29T00:47:25.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff074010420 con 0x559546877720 2024-07-29T00:47:25.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.400+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400efb0 con 0x55954686c980 2024-07-29T00:47:25.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x5595468765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x5595468765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49176/0 (socket says 172.21.15.175:49176) 2024-07-29T00:47:25.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 -- 172.21.15.175:0/3483434590 learned_addr learned my addr 172.21.15.175:0/3483434590 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:25.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559546877720 0x55954688f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff098e00640 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686def0 0x55954688f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 -- 172.21.15.175:0/3483434590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559546877720 msgr2=0x55954688f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559546877720 0x55954688f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 -- 172.21.15.175:0/3483434590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686def0 msgr2=0x55954688f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686def0 0x55954688f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff09a200640 1 -- 172.21.15.175:0/3483434590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954686cf90 con 0x55954686c980 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.401+0000 7ff098e00640 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686def0 0x55954688f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.402+0000 7ff09a200640 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x5595468765b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009ec0 tx=0x7ff08c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e5898481b8a85d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:25.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.402+0000 7ff086000640 1 -- 172.21.15.175:0/3483434590 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02c070 con 0x55954686c980 2024-07-29T00:47:25.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.402+0000 7ff086000640 1 -- 172.21.15.175:0/3483434590 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c004d90 con 0x55954686c980 2024-07-29T00:47:25.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.402+0000 7ff086000640 1 -- 172.21.15.175:0/3483434590 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c005280 con 0x55954686c980 2024-07-29T00:47:25.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3483434590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 msgr2=0x5595468765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x5595468765b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009ec0 tx=0x7ff08c0096c0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3483434590 shutdown_connections 2024-07-29T00:47:25.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559546877720 0x55954688f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954686def0 0x55954688f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3483434590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x5595468765b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3483434590 >> 172.21.15.175:0/3483434590 conn(0x559546835950 msgr2=0x559546875ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3483434590 shutdown_connections 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3483434590 wait complete. 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 -- start start 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.403+0000 7ff09ebcb740 1 Processor -- start 2024-07-29T00:47:25.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x55954687fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546871180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559546877720 0x5595468716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400fd50 con 0x559546877720 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400f680 con 0x55954686def0 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400c8e0 con 0x55954686c980 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09ebcb740 10 client.0 _ll_register_callbacks cb 0x7ff080009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:25.410 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559546877720 0x5595468716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559546877720 0x5595468716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58768/0 (socket says 172.21.15.175:58768) 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09a200640 1 -- 172.21.15.175:0/3602255425 learned_addr learned my addr 172.21.15.175:0/3602255425 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff098e00640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546871180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.404+0000 7ff09a200640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x55954687fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff098e00640 1 -- 172.21.15.175:0/3602255425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 msgr2=0x55954687fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff098e00640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x55954687fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff098e00640 1 -- 172.21.15.175:0/3602255425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559546877720 msgr2=0x5595468716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff098e00640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559546877720 0x5595468716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.411 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff098e00640 1 -- 172.21.15.175:0/3602255425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559546871d30 con 0x55954686def0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff098e00640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546871180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff08002a9e0 tx=0x7ff080003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b5945f276c4703 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff07b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff07b400640 1 -- 172.21.15.175:0/3602255425 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080035070 con 0x55954686def0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff07b400640 1 -- 172.21.15.175:0/3602255425 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff0800317b0 con 0x55954686def0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.405+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c004d90 con 0x55954686def0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff07b400640 1 -- 172.21.15.175:0/3602255425 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff0800319d0 con 0x55954686def0 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 10 client.27542 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:25.412 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5595468f2320 con 0x55954686def0 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff07b400640 1 -- 172.21.15.175:0/3602255425 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c004d90 con 0x55954686def0 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff07b400640 1 -- 172.21.15.175:0/3602255425 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff080004ba0 con 0x55954686def0 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 10 client.27542 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 10 client.27542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 20 client.27542 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5595468f2aa0 0x5595468f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 4 client.27542 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:47:25.413 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.406+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 --> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595467da530 con 0x5595468f2aa0 2024-07-29T00:47:25.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.407+0000 7ff09a200640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5595468f2aa0 0x5595468f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:25.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.407+0000 7ff09a200640 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5595468f2aa0 0x5595468f4ec0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c1c3a3b11f5a630 server_cookie=1196450a7ac4a83a in_seq=0 out_seq=0 2024-07-29T00:47:25.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.407+0000 7ff07b400640 10 client.27542 ms_handle_connect on v2:172.21.15.181:6836/753661139 2024-07-29T00:47:25.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.407+0000 7ff07b400640 1 -- 172.21.15.175:0/3602255425 <== mds.2 v2:172.21.15.181:6836/753661139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595467da530 con 0x5595468f2aa0 2024-07-29T00:47:25.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.407+0000 7ff07b400640 10 client.27542 handle_command_reply: tid=0 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.407+0000 7ff09ebcb740 1 client.27542 shutdown 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 msgr2=0x559546871180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546871180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff08002a9e0 tx=0x7ff080003c90 comp rx=0 tx=0).stop 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 shutdown_connections 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559546877720 0x5595468716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:6836/753661139,v1:172.21.15.181:6837/753661139] conn(0x5595468f2aa0 0x5595468f4ec0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954686def0 0x559546871180 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 --2- 172.21.15.175:0/3602255425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55954686c980 0x55954687fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff07b400640 0 client.27542 ms_handle_reset on v2:172.21.15.181:6836/753661139 2024-07-29T00:47:25.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 >> 172.21.15.175:0/3602255425 conn(0x559546835950 msgr2=0x55954686ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:25.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 shutdown_connections 2024-07-29T00:47:25.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 1 -- 172.21.15.175:0/3602255425 wait complete. 2024-07-29T00:47:25.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.408+0000 7ff09ebcb740 20 client.27542 trim_cache size 0 max 16384 2024-07-29T00:47:25.416 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:47:25.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff07c0a2d70 msgr2=0x7ff07c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff07c0a2d70 0x7ff07c0a5230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff088009610 tx=0x7ff0880d2800 comp rx=0 tx=0).stop 2024-07-29T00:47:25.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 msgr2=0x7ff09419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:25.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff09419e6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff08800d950 tx=0x7ff08800de10 comp rx=0 tx=0).stop 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 shutdown_connections 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09410d220 0x7ff0941a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff07c0a2d70 0x7ff07c0a5230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094108af0 0x7ff09419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 --2- 172.21.15.175:0/1927766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff094108140 0x7ff09419e6f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 >> 172.21.15.175:0/1927766193 conn(0x7ff094077890 msgr2=0x7ff094078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:25.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 shutdown_connections 2024-07-29T00:47:25.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:25.409+0000 7ff09c200640 1 -- 172.21.15.175:0/1927766193 wait complete. 2024-07-29T00:47:26.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:25 smithi204 ceph-mon[37368]: pgmap v340: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:47:26.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:25 smithi181 ceph-mon[36706]: pgmap v340: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:47:26.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:25 smithi175 ceph-mon[32138]: pgmap v340: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-29T00:47:26.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-29T00:47:26.866 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:27.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 -- 172.21.15.175:0/2315297571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 msgr2=0x7feec0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 --2- 172.21.15.175:0/2315297571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec0108130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feeac009930 tx=0x7feeac02a420 comp rx=0 tx=0).stop 2024-07-29T00:47:27.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 -- 172.21.15.175:0/2315297571 shutdown_connections 2024-07-29T00:47:27.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 --2- 172.21.15.175:0/2315297571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0108670 0x7feec0114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 --2- 172.21.15.175:0/2315297571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec0108130 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.517 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 --2- 172.21.15.175:0/2315297571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feec010dd50 0x7feec010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 -- 172.21.15.175:0/2315297571 >> 172.21.15.175:0/2315297571 conn(0x7feec0077670 msgr2=0x7feec0079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 -- 172.21.15.175:0/2315297571 shutdown_connections 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.514+0000 7feec6c00640 1 -- 172.21.15.175:0/2315297571 wait complete. 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.515+0000 7feec6c00640 1 Processor -- start 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.515+0000 7feec6c00640 1 -- start start 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0108670 0x7feec019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.518 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feec010dd50 0x7feec01a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feec01180a0 con 0x7feec0108670 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feec0117f20 con 0x7feec010dd50 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feec0118220 con 0x7feec0107d10 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0108670 0x7feec019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49206/0 (socket says 172.21.15.175:49206) 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec5800640 1 -- 172.21.15.175:0/1720508660 learned_addr learned my addr 172.21.15.175:0/1720508660 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec6200640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feec010dd50 0x7feec01a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec5800640 1 -- 172.21.15.175:0/1720508660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feec010dd50 msgr2=0x7feec01a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec5800640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feec010dd50 0x7feec01a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.516+0000 7feec5800640 1 -- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0108670 msgr2=0x7feec019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feec5800640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0108670 0x7feec019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feec5800640 1 -- 172.21.15.175:0/1720508660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec01a3510 con 0x7feec0107d10 2024-07-29T00:47:27.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feec5800640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec019e560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feeb400e9f0 tx=0x7feeb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:27.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feebe000640 1 -- 172.21.15.175:0/1720508660 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb400cd80 con 0x7feec0107d10 2024-07-29T00:47:27.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feebe000640 1 -- 172.21.15.175:0/1720508660 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb4010910 con 0x7feec0107d10 2024-07-29T00:47:27.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec01a3800 con 0x7feec0107d10 2024-07-29T00:47:27.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feebe000640 1 -- 172.21.15.175:0/1720508660 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4018990 con 0x7feec0107d10 2024-07-29T00:47:27.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.517+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01ab0e0 con 0x7feec0107d10 2024-07-29T00:47:27.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.520+0000 7feebe000640 1 -- 172.21.15.175:0/1720508660 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7feeb4010ac0 con 0x7feec0107d10 2024-07-29T00:47:27.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.520+0000 7feecabcc740 1 Processor -- start 2024-07-29T00:47:27.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 -- start start 2024-07-29T00:47:27.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fc2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fc2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559858f28ab0 con 0x559858fc2b70 2024-07-29T00:47:27.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559858ec0a80 con 0x559858fcd910 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feecabcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559858f24a70 con 0x559858fc40e0 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feec6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fc2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feec5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fc2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feec6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fc2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49210/0 (socket says 172.21.15.175:49210) 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.521+0000 7feec6200640 1 -- 172.21.15.175:0/3253507785 learned_addr learned my addr 172.21.15.175:0/3253507785 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec4e00640 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec6200640 1 -- 172.21.15.175:0/3253507785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 msgr2=0x559858fcfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.525 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec6200640 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec6200640 1 -- 172.21.15.175:0/3253507785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 msgr2=0x559858fc2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec6200640 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fc2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec6200640 1 -- 172.21.15.175:0/3253507785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559858c9a3c0 con 0x559858fc40e0 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec4e00640 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feec6200640 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fc2630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009fc0 tx=0x7feeb80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1acafe1b9eb6253b server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.522+0000 7feebe000640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feea80a2d70 0x7feea80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feeb2a00640 1 -- 172.21.15.175:0/3253507785 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004d20 con 0x559858fc40e0 2024-07-29T00:47:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feeb2a00640 1 -- 172.21.15.175:0/3253507785 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb8004050 con 0x559858fc40e0 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feebe000640 1 -- 172.21.15.175:0/1720508660 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7feeb40cf6d0 con 0x7feec0107d10 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feeb2a00640 1 -- 172.21.15.175:0/3253507785 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8005260 con 0x559858fc40e0 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feec5800640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feea80a2d70 0x7feea80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feecabcc740 1 -- 172.21.15.175:0/3253507785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 msgr2=0x559858fc2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.523+0000 7feecabcc740 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fc2630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009fc0 tx=0x7feeb80118d0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feec5800640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feea80a2d70 0x7feea80a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559858fc0060 tx=0x7feeb40d5530 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 -- 172.21.15.175:0/3253507785 shutdown_connections 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fc2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 --2- 172.21.15.175:0/3253507785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fc2630 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 -- 172.21.15.175:0/3253507785 >> 172.21.15.175:0/3253507785 conn(0x559858f8b950 msgr2=0x559858fbf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 -- 172.21.15.175:0/3253507785 shutdown_connections 2024-07-29T00:47:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 -- 172.21.15.175:0/3253507785 wait complete. 2024-07-29T00:47:27.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.524+0000 7feecabcc740 1 -- start start 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 Processor -- start 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559858fd06b0 con 0x559858fc2b70 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559858ee3c10 con 0x559858fcd910 2024-07-29T00:47:27.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559858fd0830 con 0x559858fc40e0 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feecabcc740 10 client.0 _ll_register_callbacks cb 0x559858eaca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feec6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feec4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feec6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58806/0 (socket says 172.21.15.175:58806) 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.525+0000 7feec6200640 1 -- 172.21.15.175:0/1846270381 learned_addr learned my addr 172.21.15.175:0/1846270381 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec6200640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec4e00640 1 -- 172.21.15.175:0/1846270381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 msgr2=0x559858fcb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec4e00640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec4e00640 1 -- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 msgr2=0x559858fca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec4e00640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec4e00640 1 -- 172.21.15.175:0/1846270381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559858ee71e0 con 0x559858fc40e0 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec6200640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feec4e00640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fcae20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feeac02a940 tx=0x7feeac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5ee4faf89c05e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:27.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.526+0000 7feea7400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feea7400640 1 -- 172.21.15.175:0/1846270381 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac035040 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb8004050 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feea7400640 1 -- 172.21.15.175:0/1846270381 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7feeac005230 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feea7400640 1 -- 172.21.15.175:0/1846270381 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feeac005430 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feecabcc740 10 client.27493 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559858fdc4d0 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feea7400640 1 -- 172.21.15.175:0/1846270381 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb8004050 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.527+0000 7feea7400640 1 -- 172.21.15.175:0/1846270381 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7feeac0038c0 con 0x559858fc40e0 2024-07-29T00:47:27.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feecabcc740 10 client.27493 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feecabcc740 10 client.27493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feecabcc740 20 client.27493 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feecabcc740 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fdd990 0x559858fdfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feecabcc740 4 client.27493 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559858f36020 con 0x559858fdd990 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.528+0000 7feec6200640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fdd990 0x559858fdfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.529+0000 7feec6200640 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fdd990 0x559858fdfd80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d05f4c10ff93add server_cookie=ef45f8faf8a63120 in_seq=0 out_seq=0 2024-07-29T00:47:27.536 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.529+0000 7feea7400640 10 client.27493 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.529+0000 7feea7400640 1 -- 172.21.15.175:0/1846270381 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559858f36020 con 0x559858fdd990 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.529+0000 7feea7400640 10 client.27493 handle_command_reply: tid=0 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.529+0000 7feecabcc740 1 client.27493 shutdown 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 msgr2=0x559858fcae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fcae20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feeac02a940 tx=0x7feeac004270 comp rx=0 tx=0).stop 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 shutdown_connections 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fdd990 0x559858fdfd80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fcd910 0x559858fcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fcae20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feea7400640 0 client.27493 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 --2- 172.21.15.175:0/1846270381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 >> 172.21.15.175:0/1846270381 conn(0x559858f8b950 msgr2=0x559858fc4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 shutdown_connections 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 1 -- 172.21.15.175:0/1846270381 wait complete. 2024-07-29T00:47:27.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.530+0000 7feecabcc740 20 client.27493 trim_cache size 0 max 16384 2024-07-29T00:47:27.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 Processor -- start 2024-07-29T00:47:27.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 -- start start 2024-07-29T00:47:27.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858f8b950 0x559858fbe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fe5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc40e0 0x559858fe5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b3b70 con 0x559858fc2b70 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b56b0 con 0x559858fc40e0 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feecabcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b4240 con 0x559858f8b950 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feec6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858f8b950 0x559858fbe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feec6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858f8b950 0x559858fbe250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:49230/0 (socket says 172.21.15.175:49230) 2024-07-29T00:47:27.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feec6200640 1 -- 172.21.15.175:0/4262384179 learned_addr learned my addr 172.21.15.175:0/4262384179 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.545+0000 7feec4e00640 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fe5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec6200640 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc40e0 0x559858fe5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec4e00640 1 -- 172.21.15.175:0/4262384179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858f8b950 msgr2=0x559858fbe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec4e00640 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858f8b950 0x559858fbe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec4e00640 1 -- 172.21.15.175:0/4262384179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc40e0 msgr2=0x559858fe5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec4e00640 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc40e0 0x559858fe5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec4e00640 1 -- 172.21.15.175:0/4262384179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559858fe03e0 con 0x559858fc2b70 2024-07-29T00:47:27.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec6200640 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc40e0 0x559858fe5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:27.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.546+0000 7feec4e00640 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fe5610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feeac003e80 tx=0x7feeac02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908c58c39a0012d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feeb2000640 1 -- 172.21.15.175:0/4262384179 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac035070 con 0x559858fc2b70 2024-07-29T00:47:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feeb2000640 1 -- 172.21.15.175:0/4262384179 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac040070 con 0x559858fc2b70 2024-07-29T00:47:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feeb2000640 1 -- 172.21.15.175:0/4262384179 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac031880 con 0x559858fc2b70 2024-07-29T00:47:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 -- 172.21.15.175:0/4262384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 msgr2=0x559858fe5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fe5610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feeac003e80 tx=0x7feeac02a420 comp rx=0 tx=0).stop 2024-07-29T00:47:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 -- 172.21.15.175:0/4262384179 shutdown_connections 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc40e0 0x559858fe5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858fc2b70 0x559858fe5610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 --2- 172.21.15.175:0/4262384179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858f8b950 0x559858fbe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.547+0000 7feecabcc740 1 -- 172.21.15.175:0/4262384179 >> 172.21.15.175:0/4262384179 conn(0x559858fc62d0 msgr2=0x559858fbdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.548+0000 7feecabcc740 1 -- 172.21.15.175:0/4262384179 shutdown_connections 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.548+0000 7feecabcc740 1 -- 172.21.15.175:0/4262384179 wait complete. 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.548+0000 7feecabcc740 1 -- start start 2024-07-29T00:47:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.548+0000 7feecabcc740 1 Processor -- start 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 0x559858fd1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc2b70 0x559858fd2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fd25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b4fe0 con 0x559858f8b950 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b4910 con 0x559858fc2b70 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b1b70 con 0x559858fc40e0 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feecabcc740 10 client.0 _ll_register_callbacks cb 0x7feeb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:27.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 0x559858fd1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 0x559858fd1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58828/0 (socket says 172.21.15.175:58828) 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 -- 172.21.15.175:0/3919250675 learned_addr learned my addr 172.21.15.175:0/3919250675 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec4e00640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc2b70 0x559858fd2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fd25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 -- 172.21.15.175:0/3919250675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 msgr2=0x559858fd25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fd25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 -- 172.21.15.175:0/3919250675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc2b70 msgr2=0x559858fd2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc2b70 0x559858fd2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.549+0000 7feec6200640 1 -- 172.21.15.175:0/3919250675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598590478b0 con 0x559858f8b950 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feec6200640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 0x559858fd1b10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009710 tx=0x7feeb8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aafa426b7eb467 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feea7400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feea7400640 1 -- 172.21.15.175:0/3919250675 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8039020 con 0x559858f8b950 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feea7400640 1 -- 172.21.15.175:0/3919250675 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7feeb8004810 con 0x559858f8b950 2024-07-29T00:47:27.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeac040070 con 0x559858f8b950 2024-07-29T00:47:27.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feea7400640 1 -- 172.21.15.175:0/3919250675 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feeb800f4e0 con 0x559858f8b950 2024-07-29T00:47:27.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feecabcc740 10 client.17406 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:27.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.550+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559859048bb0 con 0x559858f8b950 2024-07-29T00:47:27.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feea7400640 1 -- 172.21.15.175:0/3919250675 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac040070 con 0x559858f8b950 2024-07-29T00:47:27.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feea7400640 1 -- 172.21.15.175:0/3919250675 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7feeb8025070 con 0x559858f8b950 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feecabcc740 10 client.17406 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feecabcc740 10 client.17406 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feecabcc740 20 client.17406 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feecabcc740 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fcd910 0x55985904b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feecabcc740 4 client.17406 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 --> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559858f30530 con 0x559858fcd910 2024-07-29T00:47:27.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.551+0000 7feec4e00640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fcd910 0x55985904b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:27.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feec4e00640 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fcd910 0x55985904b0a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a98f4283d2753fa0 server_cookie=f9c3bb4c5b701bf2 in_seq=0 out_seq=0 2024-07-29T00:47:27.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feea7400640 10 client.17406 ms_handle_connect on v2:172.21.15.175:6838/1522099567 2024-07-29T00:47:27.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feea7400640 1 -- 172.21.15.175:0/3919250675 <== mds.1 v2:172.21.15.175:6838/1522099567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559858f30530 con 0x559858fcd910 2024-07-29T00:47:27.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feea7400640 10 client.17406 handle_command_reply: tid=0 2024-07-29T00:47:27.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 client.17406 shutdown 2024-07-29T00:47:27.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 msgr2=0x559858fd1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 0x559858fd1b10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009710 tx=0x7feeb8005bd0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 shutdown_connections 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:6838/1522099567,v1:172.21.15.175:6839/1522099567] conn(0x559858fcd910 0x55985904b0a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559858fc40e0 0x559858fd25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559858fc2b70 0x559858fd2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 --2- 172.21.15.175:0/3919250675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559858f8b950 0x559858fd1b10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.552+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 >> 172.21.15.175:0/3919250675 conn(0x559858fc62d0 msgr2=0x559858fcc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feea7400640 0 client.17406 ms_handle_reset on v2:172.21.15.175:6838/1522099567 2024-07-29T00:47:27.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 shutdown_connections 2024-07-29T00:47:27.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feecabcc740 1 -- 172.21.15.175:0/3919250675 wait complete. 2024-07-29T00:47:27.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feecabcc740 20 client.17406 trim_cache size 0 max 16384 2024-07-29T00:47:27.562 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:47:27.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feea80a2d70 msgr2=0x7feea80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feec6c00640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feea80a2d70 0x7feea80a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559858fc0060 tx=0x7feeb40d5530 comp rx=0 tx=0).stop 2024-07-29T00:47:27.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 msgr2=0x7feec019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:27.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.553+0000 7feec6c00640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec019e560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feeb400e9f0 tx=0x7feeb400eeb0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 shutdown_connections 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feec010dd50 0x7feec01a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7feea80a2d70 0x7feea80a5230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feec0108670 0x7feec019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 --2- 172.21.15.175:0/1720508660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec0107d10 0x7feec019e560 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 >> 172.21.15.175:0/1720508660 conn(0x7feec0077670 msgr2=0x7feec0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 shutdown_connections 2024-07-29T00:47:27.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:27.554+0000 7feec6c00640 1 -- 172.21.15.175:0/1720508660 wait complete. 2024-07-29T00:47:28.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:27 smithi204 ceph-mon[37368]: pgmap v341: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:47:28.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:27 smithi181 ceph-mon[36706]: pgmap v341: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:47:28.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:27 smithi175 ceph-mon[32138]: pgmap v341: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-07-29T00:47:28.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-29T00:47:28.971 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:29.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 -- 172.21.15.175:0/754080427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83410dd50 msgr2=0x7fa83410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 --2- 172.21.15.175:0/754080427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83410dd50 0x7fa83410e130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa824009930 tx=0x7fa82402a420 comp rx=0 tx=0).stop 2024-07-29T00:47:29.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 -- 172.21.15.175:0/754080427 shutdown_connections 2024-07-29T00:47:29.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 --2- 172.21.15.175:0/754080427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 0x7fa834114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 --2- 172.21.15.175:0/754080427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834107d10 0x7fa834108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 --2- 172.21.15.175:0/754080427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83410dd50 0x7fa83410e130 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 -- 172.21.15.175:0/754080427 >> 172.21.15.175:0/754080427 conn(0x7fa834077670 msgr2=0x7fa834079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:29.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 -- 172.21.15.175:0/754080427 shutdown_connections 2024-07-29T00:47:29.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.663+0000 7fa833200640 1 -- 172.21.15.175:0/754080427 wait complete. 2024-07-29T00:47:29.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.664+0000 7fa833200640 1 Processor -- start 2024-07-29T00:47:29.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.664+0000 7fa833200640 1 -- start start 2024-07-29T00:47:29.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.664+0000 7fa833200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa834107d10 0x7fa83419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.664+0000 7fa833200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 0x7fa83419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.664+0000 7fa833200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa83410dd50 0x7fa8341a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa833200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341180d0 con 0x7fa834107d10 2024-07-29T00:47:29.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa833200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa834117f50 con 0x7fa834108670 2024-07-29T00:47:29.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa833200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa834118250 con 0x7fa83410dd50 2024-07-29T00:47:29.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa834107d10 0x7fa83419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa834107d10 0x7fa83419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50416/0 (socket says 172.21.15.175:50416) 2024-07-29T00:47:29.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831e00640 1 -- 172.21.15.175:0/1535592509 learned_addr learned my addr 172.21.15.175:0/1535592509 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:29.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831400640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 0x7fa83419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa832800640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa83410dd50 0x7fa8341a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831400640 1 -- 172.21.15.175:0/1535592509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa83410dd50 msgr2=0x7fa8341a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831400640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa83410dd50 0x7fa8341a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831400640 1 -- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa834107d10 msgr2=0x7fa83419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831400640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa834107d10 0x7fa83419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.665+0000 7fa831400640 1 -- 172.21.15.175:0/1535592509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8341a3540 con 0x7fa834108670 2024-07-29T00:47:29.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.666+0000 7fa831400640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 0x7fa83419ead0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa82000e9f0 tx=0x7fa82000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:29.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.666+0000 7fa82aa00640 1 -- 172.21.15.175:0/1535592509 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82000ccc0 con 0x7fa834108670 2024-07-29T00:47:29.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.666+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8341a3830 con 0x7fa834108670 2024-07-29T00:47:29.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.666+0000 7fa82aa00640 1 -- 172.21.15.175:0/1535592509 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820005020 con 0x7fa834108670 2024-07-29T00:47:29.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.666+0000 7fa82aa00640 1 -- 172.21.15.175:0/1535592509 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8200049e0 con 0x7fa834108670 2024-07-29T00:47:29.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.666+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8341ab110 con 0x7fa834108670 2024-07-29T00:47:29.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.669+0000 7fa83b3fd740 1 Processor -- start 2024-07-29T00:47:29.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.669+0000 7fa82aa00640 1 -- 172.21.15.175:0/1535592509 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa820005560 con 0x7fa834108670 2024-07-29T00:47:29.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 -- start start 2024-07-29T00:47:29.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6ef3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6ef3b70 0x564da6ef3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6efe910 0x564da6f00d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564da6e59ab0 con 0x564da6efe910 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564da6df1a80 con 0x564da6ef50e0 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564da6e55a70 con 0x564da6ef3b70 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa832800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6ef3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa832800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6ef3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:38456/0 (socket says 172.21.15.175:38456) 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.670+0000 7fa831e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6ef3b70 0x564da6ef3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 -- 172.21.15.175:0/3445339686 learned_addr learned my addr 172.21.15.175:0/3445339686 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa831400640 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6efe910 0x564da6f00d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 -- 172.21.15.175:0/3445339686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6ef3b70 msgr2=0x564da6ef3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6ef3b70 0x564da6ef3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 -- 172.21.15.175:0/3445339686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6efe910 msgr2=0x564da6f00d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6efe910 0x564da6f00d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 -- 172.21.15.175:0/3445339686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564da6bcb3c0 con 0x564da6ef50e0 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa832800640 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6ef3630 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009f40 tx=0x7fa82c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9779242ec68dbbb server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.671+0000 7fa82aa00640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8180a2d70 0x7fa8180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa82aa00640 1 -- 172.21.15.175:0/1535592509 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fa8200cf3d0 con 0x7fa834108670 2024-07-29T00:47:29.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa81f000640 1 -- 172.21.15.175:0/3445339686 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c004d20 con 0x564da6ef50e0 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa81f000640 1 -- 172.21.15.175:0/3445339686 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c004050 con 0x564da6ef50e0 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa81f000640 1 -- 172.21.15.175:0/3445339686 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c005280 con 0x564da6ef50e0 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa831e00640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8180a2d70 0x7fa8180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa83b3fd740 1 -- 172.21.15.175:0/3445339686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 msgr2=0x564da6ef3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.672+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6ef3630 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009f40 tx=0x7fa82c0118d0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 -- 172.21.15.175:0/3445339686 shutdown_connections 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6efe910 0x564da6f00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6ef3b70 0x564da6ef3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/3445339686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6ef3630 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 -- 172.21.15.175:0/3445339686 >> 172.21.15.175:0/3445339686 conn(0x564da6ebc950 msgr2=0x564da6ef0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa831e00640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8180a2d70 0x7fa8180a5230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa824005c40 tx=0x7fa824005b50 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 -- 172.21.15.175:0/3445339686 shutdown_connections 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 -- 172.21.15.175:0/3445339686 wait complete. 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 -- start start 2024-07-29T00:47:29.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.673+0000 7fa83b3fd740 1 Processor -- start 2024-07-29T00:47:29.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6efb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6efbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6efc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564da6f016b0 con 0x564da6ef3b70 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564da6e14c10 con 0x564da6ef50e0 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564da6f01830 con 0x564da6efe910 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa83b3fd740 10 client.0 _ll_register_callbacks cb 0x564da6ddda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:29.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa831e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6efb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa831400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6efbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa831e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6efb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50426/0 (socket says 172.21.15.175:50426) 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa831e00640 1 -- 172.21.15.175:0/2576349160 learned_addr learned my addr 172.21.15.175:0/2576349160 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.674+0000 7fa832800640 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6efc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa831400640 1 -- 172.21.15.175:0/2576349160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 msgr2=0x564da6efc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa831400640 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6efc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa831400640 1 -- 172.21.15.175:0/2576349160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 msgr2=0x564da6efb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa831400640 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6efb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa831400640 1 -- 172.21.15.175:0/2576349160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564da6e181e0 con 0x564da6ef50e0 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa831400640 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6efbdb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa8200cca40 tx=0x7fa82009b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9572cb0e71198288 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa813400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa813400640 1 -- 172.21.15.175:0/2576349160 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82001c070 con 0x564da6ef50e0 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa813400640 1 -- 172.21.15.175:0/2576349160 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fa8200cccf0 con 0x564da6ef50e0 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.675+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c004050 con 0x564da6ef50e0 2024-07-29T00:47:29.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa813400640 1 -- 172.21.15.175:0/2576349160 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa8200ccef0 con 0x564da6ef50e0 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 10 client.27572 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564da6f0d480 con 0x564da6ef50e0 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa813400640 1 -- 172.21.15.175:0/2576349160 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c004050 con 0x564da6ef50e0 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa813400640 1 -- 172.21.15.175:0/2576349160 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fa8200a4020 con 0x564da6ef50e0 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 10 client.27572 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 10 client.27572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 20 client.27572 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f0e7d0 0x564da6f10be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 4 client.27572 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.676+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564da6e67020 con 0x564da6f0e7d0 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.677+0000 7fa831e00640 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f0e7d0 0x564da6f10be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.677+0000 7fa831e00640 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f0e7d0 0x564da6f10be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0abe2aae6b07d60 server_cookie=58891a9c40412827 in_seq=0 out_seq=0 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.677+0000 7fa813400640 10 client.27572 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.678+0000 7fa813400640 1 -- 172.21.15.175:0/2576349160 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564da6e67020 con 0x564da6f0e7d0 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.678+0000 7fa813400640 10 client.27572 handle_command_reply: tid=0 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.678+0000 7fa83b3fd740 1 client.27572 shutdown 2024-07-29T00:47:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 msgr2=0x564da6efbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6efbdb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa8200cca40 tx=0x7fa82009b6d0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 shutdown_connections 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f0e7d0 0x564da6f10be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6efc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6efbdb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa813400640 0 client.27572 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2576349160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6efb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 >> 172.21.15.175:0/2576349160 conn(0x564da6ebc950 msgr2=0x564da6ef5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:29.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 shutdown_connections 2024-07-29T00:47:29.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2576349160 wait complete. 2024-07-29T00:47:29.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.679+0000 7fa83b3fd740 20 client.27572 trim_cache size 0 max 16384 2024-07-29T00:47:29.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 Processor -- start 2024-07-29T00:47:29.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 -- start start 2024-07-29T00:47:29.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef3b70 0x564da6efd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 0x564da6f164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341b3b90 con 0x564da6ef50e0 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341b56d0 con 0x564da6ef3b70 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341b4260 con 0x564da6efe910 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa831400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 0x564da6f164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa831400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 0x564da6f164b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50434/0 (socket says 172.21.15.175:50434) 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa831400640 1 -- 172.21.15.175:0/1850888778 learned_addr learned my addr 172.21.15.175:0/1850888778 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.694+0000 7fa832800640 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831e00640 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef3b70 0x564da6efd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831400640 1 -- 172.21.15.175:0/1850888778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 msgr2=0x564da6f169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831400640 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831400640 1 -- 172.21.15.175:0/1850888778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef3b70 msgr2=0x564da6efd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831400640 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef3b70 0x564da6efd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831400640 1 -- 172.21.15.175:0/1850888778 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564da6ef42f0 con 0x564da6ef50e0 2024-07-29T00:47:29.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa831400640 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 0x564da6f164b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c6a0 tx=0x7fa820009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc11747e8be06c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:29.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa81e600640 1 -- 172.21.15.175:0/1850888778 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8200d29b0 con 0x564da6ef50e0 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa81e600640 1 -- 172.21.15.175:0/1850888778 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82009b840 con 0x564da6ef50e0 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.695+0000 7fa81e600640 1 -- 172.21.15.175:0/1850888778 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820004c10 con 0x564da6ef50e0 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 -- 172.21.15.175:0/1850888778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 msgr2=0x564da6f164b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 0x564da6f164b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c6a0 tx=0x7fa820009f60 comp rx=0 tx=0).stop 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 -- 172.21.15.175:0/1850888778 shutdown_connections 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef50e0 0x564da6f164b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/1850888778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef3b70 0x564da6efd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 -- 172.21.15.175:0/1850888778 >> 172.21.15.175:0/1850888778 conn(0x564da6ebc950 msgr2=0x564da6efd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 -- 172.21.15.175:0/1850888778 shutdown_connections 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.696+0000 7fa83b3fd740 1 -- 172.21.15.175:0/1850888778 wait complete. 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 -- start start 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 Processor -- start 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6f02940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6f02e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341b5000 con 0x564da6ef3b70 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341b4930 con 0x564da6ef50e0 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.697+0000 7fa83b3fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341b1b90 con 0x564da6efe910 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa83b3fd740 10 client.0 _ll_register_callbacks cb 0x7fa8200053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa831e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6f02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa831e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6f02940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50438/0 (socket says 172.21.15.175:50438) 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa831e00640 1 -- 172.21.15.175:0/2192016476 learned_addr learned my addr 172.21.15.175:0/2192016476 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa832800640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa831400640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6f02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa832800640 1 -- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 msgr2=0x564da6f02e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa832800640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6f02e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa832800640 1 -- 172.21.15.175:0/2192016476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 msgr2=0x564da6f02940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa832800640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6f02940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.698+0000 7fa832800640 1 -- 172.21.15.175:0/2192016476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564da6f785e0 con 0x564da6efe910 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa832800640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f033c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005c10 tx=0x7fa82c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87ca7c87de7da11 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa813400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa813400640 1 -- 172.21.15.175:0/2192016476 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0055c0 con 0x564da6efe910 2024-07-29T00:47:29.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82009b840 con 0x564da6efe910 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa813400640 1 -- 172.21.15.175:0/2192016476 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fa82c005760 con 0x564da6efe910 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa813400640 1 -- 172.21.15.175:0/2192016476 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa82c005980 con 0x564da6efe910 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa83b3fd740 10 client.27517 fetch_fsmap learned FSMap version 43 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.699+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564da6f79880 con 0x564da6efe910 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa813400640 1 -- 172.21.15.175:0/2192016476 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82009b840 con 0x564da6efe910 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa813400640 1 -- 172.21.15.175:0/2192016476 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fa82c02a070 con 0x564da6efe910 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa83b3fd740 10 client.27517 fetch_fsmap finished waiting for FSMap version 43 2024-07-29T00:47:29.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa83b3fd740 10 client.27517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-29T00:47:29.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa83b3fd740 20 client.27517 populate_metadata read hostname 'smithi175' 2024-07-29T00:47:29.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f79db0 0x564da6f7c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:29.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa83b3fd740 4 client.27517 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-29T00:47:29.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 --> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564da6e61530 con 0x564da6f79db0 2024-07-29T00:47:29.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.700+0000 7fa831e00640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f79db0 0x564da6f7c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:29.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa831e00640 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f79db0 0x564da6f7c1a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f08ea4308a30de server_cookie=a44d6ddf1d9d952f in_seq=0 out_seq=0 2024-07-29T00:47:29.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa813400640 10 client.27517 ms_handle_connect on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa813400640 1 -- 172.21.15.175:0/2192016476 <== mds.0 v2:172.21.15.181:6838/4150501134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564da6e61530 con 0x564da6f79db0 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa813400640 10 client.27517 handle_command_reply: tid=0 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 client.27517 shutdown 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 msgr2=0x564da6f033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f033c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005c10 tx=0x7fa82c005040 comp rx=0 tx=0).stop 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 shutdown_connections 2024-07-29T00:47:29.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:6838/4150501134,v1:172.21.15.181:6839/4150501134] conn(0x564da6f79db0 0x564da6f7c1a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564da6efe910 0x564da6f033c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564da6ef50e0 0x564da6f02e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 --2- 172.21.15.175:0/2192016476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564da6ef3b70 0x564da6f02940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa813400640 0 client.27517 ms_handle_reset on v2:172.21.15.181:6838/4150501134 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 >> 172.21.15.175:0/2192016476 conn(0x564da6ebc950 msgr2=0x564da6ef5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.701+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 shutdown_connections 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.702+0000 7fa83b3fd740 1 -- 172.21.15.175:0/2192016476 wait complete. 2024-07-29T00:47:29.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.702+0000 7fa83b3fd740 20 client.27517 trim_cache size 0 max 16384 2024-07-29T00:47:29.710 INFO:teuthology.orchestra.run.smithi175.stdout:[] 2024-07-29T00:47:29.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.702+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8180a2d70 msgr2=0x7fa8180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.702+0000 7fa833200640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8180a2d70 0x7fa8180a5230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa824005c40 tx=0x7fa824005b50 comp rx=0 tx=0).stop 2024-07-29T00:47:29.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.702+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 msgr2=0x7fa83419ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:29.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.702+0000 7fa833200640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 0x7fa83419ead0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa82000e9f0 tx=0x7fa82000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 shutdown_connections 2024-07-29T00:47:29.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa83410dd50 0x7fa8341a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa834108670 0x7fa83419ead0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fa8180a2d70 0x7fa8180a5230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 --2- 172.21.15.175:0/1535592509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa834107d10 0x7fa83419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:29.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 >> 172.21.15.175:0/1535592509 conn(0x7fa834077670 msgr2=0x7fa834078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:29.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 shutdown_connections 2024-07-29T00:47:29.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:29.703+0000 7fa833200640 1 -- 172.21.15.175:0/1535592509 wait complete. 2024-07-29T00:47:30.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:29 smithi175 ceph-mon[32138]: pgmap v342: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:30.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:29 smithi204 ceph-mon[37368]: pgmap v342: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:30.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:29 smithi181 ceph-mon[36706]: pgmap v342: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:30.543 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ba82afec3628bf830c0830558bf3d3c3aea66701/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-07-29T00:47:30.550 INFO:tasks.fwd_scrub:done joining 2024-07-29T00:47:30.551 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-07-29T00:47:30.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-29T00:47:30.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 -- 172.21.15.175:0/1643773878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f268810d160 msgr2=0x7f2688114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:30.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 --2- 172.21.15.175:0/1643773878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f268810d160 0x7f2688114cb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2680009ca0 tx=0x7f268002a8c0 comp rx=0 tx=0).stop 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 -- 172.21.15.175:0/1643773878 shutdown_connections 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 --2- 172.21.15.175:0/1643773878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f268810d160 0x7f2688114cb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 --2- 172.21.15.175:0/1643773878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26881089d0 0x7f268810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 --2- 172.21.15.175:0/1643773878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2688108020 0x7f2688108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.780+0000 7f268e400640 1 -- 172.21.15.175:0/1643773878 >> 172.21.15.175:0/1643773878 conn(0x7f2688077730 msgr2=0x7f2688079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.781+0000 7f268e400640 1 -- 172.21.15.175:0/1643773878 shutdown_connections 2024-07-29T00:47:30.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.781+0000 7f268e400640 1 -- 172.21.15.175:0/1643773878 wait complete. 2024-07-29T00:47:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 Processor -- start 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 -- start start 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 0x7f268819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268810d160 0x7f268819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26881a2fc0 0x7f26881a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26881181b0 con 0x7f2688108020 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2688118030 con 0x7f26881a2fc0 2024-07-29T00:47:30.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f268e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2688118330 con 0x7f268810d160 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.782+0000 7f2687400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 0x7f268819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 0x7f268819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50460/0 (socket says 172.21.15.175:50460) 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 -- 172.21.15.175:0/3403065176 learned_addr learned my addr 172.21.15.175:0/3403065176 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f267ea00640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268810d160 0x7f268819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687e00640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26881a2fc0 0x7f26881a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 -- 172.21.15.175:0/3403065176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268810d160 msgr2=0x7f268819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268810d160 0x7f268819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:30.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 -- 172.21.15.175:0/3403065176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26881a2fc0 msgr2=0x7f26881a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26881a2fc0 0x7f26881a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 -- 172.21.15.175:0/3403065176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26881a3ab0 con 0x7f2688108020 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.783+0000 7f2687400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 0x7f268819e700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f267800ea60 tx=0x7f267800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.784+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267800cde0 con 0x7f2688108020 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.784+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26780035f0 con 0x7f2688108020 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.784+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26881a3da0 con 0x7f2688108020 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.784+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678010670 con 0x7f2688108020 2024-07-29T00:47:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.784+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881ab5e0 con 0x7f2688108020 2024-07-29T00:47:30.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.785+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2688108400 con 0x7f2688108020 2024-07-29T00:47:30.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.787+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f26780037a0 con 0x7f2688108020 2024-07-29T00:47:30.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.788+0000 7f2684c00640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f266c0a2d80 0x7f266c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:30.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.788+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f26780cf790 con 0x7f2688108020 2024-07-29T00:47:30.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.789+0000 7f267ea00640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f266c0a2d80 0x7f266c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:30.789 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.789+0000 7f267ea00640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f266c0a2d80 0x7f266c0a5240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2688069930 tx=0x7f2670005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:30.792 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.793+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f267809c300 con 0x7f2688108020 2024-07-29T00:47:30.997 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:30.997+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2688110370 con 0x7f2688108020 2024-07-29T00:47:31.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.001+0000 7f2684c00640 1 -- 172.21.15.175:0/3403065176 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+23312 (secure 0 0 0) 0x7f267809bc00 con 0x7f2688108020 2024-07-29T00:47:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:47:31.003 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":102,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:47:15.764784+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1189171033":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/3246788131":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/257505941":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/2019123994":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/4008532755":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/3246788131":"2024-07-30T00:36:29.909303+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:47:31.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f266c0a2d80 msgr2=0x7f266c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f266c0a2d80 0x7f266c0a5240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2688069930 tx=0x7f2670005f90 comp rx=0 tx=0).stop 2024-07-29T00:47:31.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 msgr2=0x7f268819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 0x7f268819e700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f267800ea60 tx=0x7f267800ef20 comp rx=0 tx=0).stop 2024-07-29T00:47:31.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 shutdown_connections 2024-07-29T00:47:31.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26881a2fc0 0x7f26881a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268810d160 0x7f268819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f266c0a2d80 0x7f266c0a5240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 --2- 172.21.15.175:0/3403065176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108020 0x7f268819e700 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.004+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 >> 172.21.15.175:0/3403065176 conn(0x7f2688077730 msgr2=0x7f2688078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:31.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.005+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 shutdown_connections 2024-07-29T00:47:31.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.005+0000 7f268e400640 1 -- 172.21.15.175:0/3403065176 wait complete. 2024-07-29T00:47:31.027 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-29T00:29:04.051354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-29T00:33:52.192870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-29T00:33:55.363149+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-29T00:47:31.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-29T00:47:31.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 -- 172.21.15.175:0/529320037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 msgr2=0x7fb4e010b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 --2- 172.21.15.175:0/529320037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e010b750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009860 tx=0x7fb4c802a3d0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 -- 172.21.15.175:0/529320037 shutdown_connections 2024-07-29T00:47:31.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 --2- 172.21.15.175:0/529320037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4e010bd70 0x7fb4e010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.235 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 --2- 172.21.15.175:0/529320037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e010b750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 --2- 172.21.15.175:0/529320037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e0102950 0x7fb4e0102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 -- 172.21.15.175:0/529320037 >> 172.21.15.175:0/529320037 conn(0x7fb4e00fc3a0 msgr2=0x7fb4e00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:31.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 -- 172.21.15.175:0/529320037 shutdown_connections 2024-07-29T00:47:31.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.234+0000 7fb4e7400640 1 -- 172.21.15.175:0/529320037 wait complete. 2024-07-29T00:47:31.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.235+0000 7fb4e7400640 1 Processor -- start 2024-07-29T00:47:31.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.235+0000 7fb4e7400640 1 -- start start 2024-07-29T00:47:31.238 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4e0102950 0x7fb4e01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 0x7fb4e019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e7400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0111be0 con 0x7fb4e010bd70 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e7400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0111a60 con 0x7fb4e0103270 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e7400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0111d60 con 0x7fb4e0102950 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 0x7fb4e019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 0x7fb4e019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50480/0 (socket says 172.21.15.175:50480) 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 -- 172.21.15.175:0/1123557859 learned_addr learned my addr 172.21.15.175:0/1123557859 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:31.239 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e4a00640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4e0102950 0x7fb4e01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4dfe00640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 -- 172.21.15.175:0/1123557859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4e0102950 msgr2=0x7fb4e01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4e0102950 0x7fb4e01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 -- 172.21.15.175:0/1123557859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 msgr2=0x7fb4e01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4e5400640 1 -- 172.21.15.175:0/1123557859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e019d090 con 0x7fb4e010bd70 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.236+0000 7fb4dfe00640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-29T00:47:31.240 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.237+0000 7fb4e5400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 0x7fb4e019c960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800eb10 tx=0x7fb4d800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:31.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.237+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800ce20 con 0x7fb4e010bd70 2024-07-29T00:47:31.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.237+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8005020 con 0x7fb4e010bd70 2024-07-29T00:47:31.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.237+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e019d380 con 0x7fb4e010bd70 2024-07-29T00:47:31.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.237+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8010690 con 0x7fb4e010bd70 2024-07-29T00:47:31.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.237+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e01a4c10 con 0x7fb4e010bd70 2024-07-29T00:47:31.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.238+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e006a930 con 0x7fb4e010bd70 2024-07-29T00:47:31.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.240+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb4d8005540 con 0x7fb4e010bd70 2024-07-29T00:47:31.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.241+0000 7fb4dd600640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb4c40a2d00 0x7fb4c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.242+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fb4d80cfbb0 con 0x7fb4e010bd70 2024-07-29T00:47:31.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.242+0000 7fb4e4a00640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb4c40a2d00 0x7fb4c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.243+0000 7fb4e4a00640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb4c40a2d00 0x7fb4c40a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005fd0 tx=0x7fb4d0009260 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:31.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.245+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb4d809c720 con 0x7fb4e010bd70 2024-07-29T00:47:31.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.444+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4e01067e0 con 0x7fb4e010bd70 2024-07-29T00:47:31.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.445+0000 7fb4dd600640 1 -- 172.21.15.175:0/1123557859 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7fb4d809c020 con 0x7fb4e010bd70 2024-07-29T00:47:31.446 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 1 2024-07-29T00:47:31.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.448+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb4c40a2d00 msgr2=0x7fb4c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.448+0000 7fb4e7400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb4c40a2d00 0x7fb4c40a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005fd0 tx=0x7fb4d0009260 comp rx=0 tx=0).stop 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 msgr2=0x7fb4e019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 0x7fb4e019c960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800eb10 tx=0x7fb4d800ef10 comp rx=0 tx=0).stop 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 shutdown_connections 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb4c40a2d00 0x7fb4c40a51c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4e010bd70 0x7fb4e019c960 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4e0103270 0x7fb4e01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 --2- 172.21.15.175:0/1123557859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4e0102950 0x7fb4e01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 >> 172.21.15.175:0/1123557859 conn(0x7fb4e00fc3a0 msgr2=0x7fb4e00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:31.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 shutdown_connections 2024-07-29T00:47:31.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.449+0000 7fb4e7400640 1 -- 172.21.15.175:0/1123557859 wait complete. 2024-07-29T00:47:31.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-29T00:47:31.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 -- 172.21.15.175:0/2398543601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 msgr2=0x7fb3e0108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 --2- 172.21.15.175:0/2398543601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 0x7fb3e0108150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009930 tx=0x7fb3c802a440 comp rx=0 tx=0).stop 2024-07-29T00:47:31.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 -- 172.21.15.175:0/2398543601 shutdown_connections 2024-07-29T00:47:31.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 --2- 172.21.15.175:0/2398543601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e0114b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 --2- 172.21.15.175:0/2398543601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 0x7fb3e0108150 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 --2- 172.21.15.175:0/2398543601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 0x7fb3e010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 -- 172.21.15.175:0/2398543601 >> 172.21.15.175:0/2398543601 conn(0x7fb3e00775e0 msgr2=0x7fb3e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:31.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.681+0000 7fb3e5000640 1 -- 172.21.15.175:0/2398543601 shutdown_connections 2024-07-29T00:47:31.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.682+0000 7fb3e5000640 1 -- 172.21.15.175:0/2398543601 wait complete. 2024-07-29T00:47:31.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 Processor -- start 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 -- start start 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 0x7fb3e019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 0x7fb3e01a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0118040 con 0x7fb3e0107d10 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0117ec0 con 0x7fb3e010dd30 2024-07-29T00:47:31.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.683+0000 7fb3e5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01181c0 con 0x7fb3e0108690 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 0x7fb3e01a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e019eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:40620/0 (socket says 172.21.15.175:40620) 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 -- 172.21.15.175:0/259376681 learned_addr learned my addr 172.21.15.175:0/259376681 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3df400640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 0x7fb3e019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 -- 172.21.15.175:0/259376681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 msgr2=0x7fb3e01a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 0x7fb3e01a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 -- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 msgr2=0x7fb3e019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 0x7fb3e019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 -- 172.21.15.175:0/259376681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e01a35a0 con 0x7fb3e0108690 2024-07-29T00:47:31.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3dfe00640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 0x7fb3e01a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:31.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.684+0000 7fb3d7e00640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e019eaf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009230 tx=0x7fb3c8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:31.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.685+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c803f030 con 0x7fb3e0108690 2024-07-29T00:47:31.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.685+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c8004440 con 0x7fb3e0108690 2024-07-29T00:47:31.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.685+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8035070 con 0x7fb3e0108690 2024-07-29T00:47:31.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.685+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e01a3830 con 0x7fb3e0108690 2024-07-29T00:47:31.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.685+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e01ab120 con 0x7fb3e0108690 2024-07-29T00:47:31.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.686+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e0108150 con 0x7fb3e0108690 2024-07-29T00:47:31.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.688+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb3c8042030 con 0x7fb3e0108690 2024-07-29T00:47:31.689 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.690+0000 7fb3dcc00640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb3c00a2d80 0x7fb3c00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:31.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.690+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fb3c80efb80 con 0x7fb3e0108690 2024-07-29T00:47:31.690 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.690+0000 7fb3df400640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb3c00a2d80 0x7fb3c00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:31.691 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.691+0000 7fb3df400640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb3c00a2d80 0x7fb3c00a5240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009b50 tx=0x7fb3d0005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:31.692 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.692+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb3c80bc0a0 con 0x7fb3e0108690 2024-07-29T00:47:31.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.893+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3e01101b0 con 0x7fb3e0108690 2024-07-29T00:47:31.894 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.894+0000 7fb3dcc00640 1 -- 172.21.15.175:0/259376681 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7fb3c80c03a0 con 0x7fb3e0108690 2024-07-29T00:47:31.895 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 32 2024-07-29T00:47:31.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb3c00a2d80 msgr2=0x7fb3c00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb3c00a2d80 0x7fb3c00a5240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009b50 tx=0x7fb3d0005f90 comp rx=0 tx=0).stop 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 msgr2=0x7fb3e019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e019eaf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009230 tx=0x7fb3c8004040 comp rx=0 tx=0).stop 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 shutdown_connections 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e010dd30 0x7fb3e01a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e0108690 0x7fb3e019eaf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb3c00a2d80 0x7fb3c00a5240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 --2- 172.21.15.175:0/259376681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3e0107d10 0x7fb3e019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:31.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 >> 172.21.15.175:0/259376681 conn(0x7fb3e00775e0 msgr2=0x7fb3e0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:31.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 shutdown_connections 2024-07-29T00:47:31.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:31.897+0000 7fb3e5000640 1 -- 172.21.15.175:0/259376681 wait complete. 2024-07-29T00:47:31.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-29T00:47:32.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:31 smithi204 ceph-mon[37368]: pgmap v343: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:32.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:31 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3403065176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:47:32.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:31 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1123557859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:47:32.118 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 -- 172.21.15.175:0/2211854044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4109890 msgr2=0x7f26a4109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 --2- 172.21.15.175:0/2211854044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4109890 0x7f26a4109c70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2694009930 tx=0x7f269402a420 comp rx=0 tx=0).stop 2024-07-29T00:47:32.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 -- 172.21.15.175:0/2211854044 shutdown_connections 2024-07-29T00:47:32.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 --2- 172.21.15.175:0/2211854044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4104220 0x7f26a4110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 --2- 172.21.15.175:0/2211854044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26a4103880 0x7f26a4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 --2- 172.21.15.175:0/2211854044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4109890 0x7f26a4109c70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.119 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 -- 172.21.15.175:0/2211854044 >> 172.21.15.175:0/2211854044 conn(0x7f26a40ff580 msgr2=0x7f26a41019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:32.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 -- 172.21.15.175:0/2211854044 shutdown_connections 2024-07-29T00:47:32.120 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.118+0000 7f26a9400640 1 -- 172.21.15.175:0/2211854044 wait complete. 2024-07-29T00:47:32.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 Processor -- start 2024-07-29T00:47:32.122 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 -- start start 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 0x7f26a419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26a4104220 0x7f26a419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4109890 0x7f26a419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4113d20 con 0x7f26a4103880 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4113ba0 con 0x7f26a4109890 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.119+0000 7f26a9400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4113ea0 con 0x7f26a4104220 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 0x7f26a419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4109890 0x7f26a419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 0x7f26a419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50514/0 (socket says 172.21.15.175:50514) 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a3e00640 1 -- 172.21.15.175:0/640360805 learned_addr learned my addr 172.21.15.175:0/640360805 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f269b400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26a4104220 0x7f26a419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 -- 172.21.15.175:0/640360805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26a4104220 msgr2=0x7f26a419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26a4104220 0x7f26a419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 -- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 msgr2=0x7f26a419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 0x7f26a419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 -- 172.21.15.175:0/640360805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a419f1e0 con 0x7f26a4109890 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a3e00640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 0x7f26a419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.120+0000 7f26a8a00640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4109890 0x7f26a419eab0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f269c009fd0 tx=0x7f269c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.121+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c00ce20 con 0x7f26a4109890 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.121+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c005020 con 0x7f26a4109890 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.121+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c010720 con 0x7f26a4109890 2024-07-29T00:47:32.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.121+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a419f4d0 con 0x7f26a4109890 2024-07-29T00:47:32.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.121+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a41a6d60 con 0x7f26a4109890 2024-07-29T00:47:32.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.122+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a4103ce0 con 0x7f26a4109890 2024-07-29T00:47:32.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.124+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f269c005540 con 0x7f26a4109890 2024-07-29T00:47:32.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.125+0000 7f26a1600640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26880a2d00 0x7f26880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.126+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f269c0cfc60 con 0x7f26a4109890 2024-07-29T00:47:32.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.126+0000 7f26a3e00640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26880a2d00 0x7f26880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.126+0000 7f26a3e00640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26880a2d00 0x7f26880a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2694000c00 tx=0x7f2694031510 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:32.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.128+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f269c09c7d0 con 0x7f26a4109890 2024-07-29T00:47:32.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:31 smithi181 ceph-mon[36706]: pgmap v343: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:32.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:31 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3403065176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:47:32.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:31 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1123557859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:47:32.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:31 smithi175 ceph-mon[32138]: pgmap v343: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:32.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:31 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3403065176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:47:32.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:31 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1123557859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-29T00:47:32.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.328+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26a410be20 con 0x7f26a4109890 2024-07-29T00:47:32.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.329+0000 7f26a1600640 1 -- 172.21.15.175:0/640360805 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f269c09c0d0 con 0x7f26a4109890 2024-07-29T00:47:32.329 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 64 2024-07-29T00:47:32.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26880a2d00 msgr2=0x7f26880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26880a2d00 0x7f26880a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2694000c00 tx=0x7f2694031510 comp rx=0 tx=0).stop 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4109890 msgr2=0x7f26a419eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4109890 0x7f26a419eab0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f269c009fd0 tx=0x7f269c00ef10 comp rx=0 tx=0).stop 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 shutdown_connections 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4109890 0x7f26a419eab0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26a4104220 0x7f26a419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f26880a2d00 0x7f26880a51c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 --2- 172.21.15.175:0/640360805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4103880 0x7f26a419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 >> 172.21.15.175:0/640360805 conn(0x7f26a40ff580 msgr2=0x7f26a40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:32.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 shutdown_connections 2024-07-29T00:47:32.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.331+0000 7f26a9400640 1 -- 172.21.15.175:0/640360805 wait complete. 2024-07-29T00:47:32.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-07-29T00:47:32.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 -- 172.21.15.175:0/83831138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0074290 msgr2=0x7ff7b0074670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/83831138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0074290 0x7ff7b0074670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009900 tx=0x7ff7a402a490 comp rx=0 tx=0).stop 2024-07-29T00:47:32.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 -- 172.21.15.175:0/83831138 shutdown_connections 2024-07-29T00:47:32.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/83831138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b010dea0 0x7ff7b0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/83831138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b0074bb0 0x7ff7b010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/83831138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0074290 0x7ff7b0074670 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 -- 172.21.15.175:0/83831138 >> 172.21.15.175:0/83831138 conn(0x7ff7b00fd300 msgr2=0x7ff7b00ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:32.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 -- 172.21.15.175:0/83831138 shutdown_connections 2024-07-29T00:47:32.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.549+0000 7ff7b5e00640 1 -- 172.21.15.175:0/83831138 wait complete. 2024-07-29T00:47:32.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 Processor -- start 2024-07-29T00:47:32.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 -- start start 2024-07-29T00:47:32.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b0074290 0x7ff7b019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 0x7ff7b019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b010dea0 0x7ff7b019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0113c20 con 0x7ff7b010dea0 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0113aa0 con 0x7ff7b0074290 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.550+0000 7ff7b5e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0113da0 con 0x7ff7b0074bb0 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b0074290 0x7ff7b019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 0x7ff7b019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b010dea0 0x7ff7b019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 0x7ff7b019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:40652/0 (socket says 172.21.15.175:40652) 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 -- 172.21.15.175:0/1512658408 learned_addr learned my addr 172.21.15.175:0/1512658408 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:32.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 -- 172.21.15.175:0/1512658408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b0074290 msgr2=0x7ff7b019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b0074290 0x7ff7b019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 -- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b010dea0 msgr2=0x7ff7b019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b010dea0 0x7ff7b019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 -- 172.21.15.175:0/1512658408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019f0e0 con 0x7ff7b0074bb0 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.551+0000 7ff7afe00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 0x7ff7b019a630 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.552+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00cd80 con 0x7ff7b0074bb0 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.552+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019f3d0 con 0x7ff7b0074bb0 2024-07-29T00:47:32.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.552+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c010910 con 0x7ff7b0074bb0 2024-07-29T00:47:32.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.552+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0188f0 con 0x7ff7b0074bb0 2024-07-29T00:47:32.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.552+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a6c60 con 0x7ff7b0074bb0 2024-07-29T00:47:32.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.553+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b0074670 con 0x7ff7b0074bb0 2024-07-29T00:47:32.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.555+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff79c010ac0 con 0x7ff7b0074bb0 2024-07-29T00:47:32.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.556+0000 7ff7ad600640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff7980a2d00 0x7ff7980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:32.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.556+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff79c0cfd00 con 0x7ff7b0074bb0 2024-07-29T00:47:32.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.557+0000 7ff7b4a00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff7980a2d00 0x7ff7980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:32.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.557+0000 7ff7b4a00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff7980a2d00 0x7ff7980a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4000c00 tx=0x7ff7a4005b50 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:32.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.558+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff79c09c220 con 0x7ff7b0074bb0 2024-07-29T00:47:32.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.813+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ff7b0109020 con 0x7ff7b0074bb0 2024-07-29T00:47:32.823 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.823+0000 7ff7ad600640 1 -- 172.21.15.175:0/1512658408 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740911 (secure 0 0 0) 0x7ff79c0a0520 con 0x7ff7b0074bb0 2024-07-29T00:47:32.836 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:47:32.836 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:47:32.836 INFO:teuthology.orchestra.run.smithi175.stdout: "epoch": 26, 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: "active_gid": 15033, 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: "active_name": "x", 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: "active_addrs": { 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:6800", 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 1625031332 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:6801", 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 1625031332 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:32.838 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:32.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.839 INFO:teuthology.orchestra.run.smithi175.stdout: "active_addr": "172.21.15.175:6801/1625031332", 2024-07-29T00:47:32.839 INFO:teuthology.orchestra.run.smithi175.stdout: "active_change": "2024-07-29T00:36:29.909658+0000", 2024-07-29T00:47:32.839 INFO:teuthology.orchestra.run.smithi175.stdout: "active_mgr_features": 4540701547738038271, 2024-07-29T00:47:32.839 INFO:teuthology.orchestra.run.smithi175.stdout: "available": true, 2024-07-29T00:47:32.839 INFO:teuthology.orchestra.run.smithi175.stdout: "standbys": [ 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "gid": 25120, 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "z", 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "mgr_features": 4540701547738038271, 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "available_modules": [ 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "alerts", 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:32.840 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:32.841 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How frequently to reexamine health status", 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.842 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.843 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.844 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.845 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:32.847 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:32.848 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:32.849 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.850 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_destination": { 2024-07-29T00:47:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_destination", 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.852 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Email address to send alerts to", 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_from_name": { 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_from_name", 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.853 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.854 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.854 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "Ceph", 2024-07-29T00:47:32.854 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.854 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.854 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Email From: name", 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_host": { 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_host", 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.855 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.856 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.856 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.856 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.856 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.856 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.856 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP server", 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_password": { 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_password", 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.857 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Password to authenticate with", 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.858 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_port": { 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_port", 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.859 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "465", 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP port", 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.861 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_sender": { 2024-07-29T00:47:32.861 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_sender", 2024-07-29T00:47:32.861 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.861 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.861 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.861 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.862 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.862 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.862 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.862 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP envelope sender", 2024-07-29T00:47:32.862 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_ssl": { 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_ssl", 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.863 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.864 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_user": { 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_user", 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.865 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.866 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.866 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.866 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.866 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "User to authenticate as", 2024-07-29T00:47:32.866 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.866 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "balancer", 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:32.867 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:32.868 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:32.868 INFO:teuthology.orchestra.run.smithi175.stdout: "active": { 2024-07-29T00:47:32.868 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "active", 2024-07-29T00:47:32.868 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.868 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.868 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "automatically balance PGs across cluster", 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.869 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "begin_time": { 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "begin_time", 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.870 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0000", 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "beginning time of day to automatically balance", 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.872 INFO:teuthology.orchestra.run.smithi175.stdout: "begin_weekday": { 2024-07-29T00:47:32.872 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "begin_weekday", 2024-07-29T00:47:32.872 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:32.872 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.872 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.873 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:32.873 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0", 2024-07-29T00:47:32.873 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "6", 2024-07-29T00:47:32.873 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.873 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-29T00:47:32.873 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_max_iterations": { 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_max_iterations", 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:32.874 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "25", 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "250", 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.875 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_metrics": { 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_metrics", 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "pgs,objects,bytes", 2024-07-29T00:47:32.876 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_step": { 2024-07-29T00:47:32.877 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_step", 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0.5", 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0.001", 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "0.999", 2024-07-29T00:47:32.878 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "aggressiveness of optimization", 2024-07-29T00:47:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-29T00:47:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: "end_time": { 2024-07-29T00:47:32.880 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "end_time", 2024-07-29T00:47:32.880 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.880 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.880 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.880 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "2359", 2024-07-29T00:47:32.881 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.881 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.881 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.881 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "ending time of day to automatically balance", 2024-07-29T00:47:32.882 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-29T00:47:32.882 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.882 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.882 INFO:teuthology.orchestra.run.smithi175.stdout: "end_weekday": { 2024-07-29T00:47:32.883 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "end_weekday", 2024-07-29T00:47:32.883 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:32.883 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.883 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.883 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:32.884 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0", 2024-07-29T00:47:32.884 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "6", 2024-07-29T00:47:32.884 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.884 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-29T00:47:32.884 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-29T00:47:32.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.885 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.885 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:32.885 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:32.886 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.886 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.886 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.886 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.887 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.887 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.887 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.887 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:32.887 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:32.888 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:32.888 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:32.888 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:32.888 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:32.888 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.889 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.889 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.889 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.889 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.889 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:32.890 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:32.890 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.890 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.890 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.891 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.891 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.891 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.891 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:32.892 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:32.893 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.894 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.895 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.896 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "min_score": { 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "min_score", 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.897 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.898 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.899 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": { 2024-07-29T00:47:32.899 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mode", 2024-07-29T00:47:32.899 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.899 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.899 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "upmap", 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "crush-compat", 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "none", 2024-07-29T00:47:32.900 INFO:teuthology.orchestra.run.smithi175.stdout: "read", 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap", 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap-read" 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Balancer mode", 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "pool_ids": { 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pool_ids", 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.902 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:32.903 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-29T00:47:32.904 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap_max_deviation": { 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "upmap_max_deviation", 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:32.905 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-29T00:47:32.906 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap_max_optimizations": { 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "upmap_max_optimizations", 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.907 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.908 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cephadm", 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:32.909 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:32.910 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:32.910 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_down_multiplier": { 2024-07-29T00:47:32.910 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_down_multiplier", 2024-07-29T00:47:32.910 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:32.910 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3.0", 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.911 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_refresh_rate": { 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_refresh_rate", 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "20", 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.912 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_starting_port": { 2024-07-29T00:47:32.913 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_starting_port", 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "4721", 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.914 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.915 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-29T00:47:32.915 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.915 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.915 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.915 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_ptrace": { 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_ptrace", 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.916 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "autotune_interval": { 2024-07-29T00:47:32.917 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "autotune_interval", 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.918 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "autotune_memory_target_ratio": { 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "autotune_memory_target_ratio", 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.919 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0.7", 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm_log_destination": { 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cephadm_log_destination", 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.921 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: "file", 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: "file,syslog", 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: "syslog" 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-29T00:47:32.922 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: "cgroups_split": { 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cgroups_split", 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.923 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.924 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "config_checks_enabled": { 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_checks_enabled", 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.925 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.926 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.926 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.926 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.926 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-29T00:47:32.926 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.926 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: "config_dashboard": { 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_dashboard", 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.927 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.928 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_alertmanager": { 2024-07-29T00:47:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_alertmanager", 2024-07-29T00:47:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_base": { 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_base", 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.931 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Container image name, without the tag", 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.932 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_elasticsearch": { 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_elasticsearch", 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.933 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "elasticsearch container image", 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.934 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_grafana": { 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_grafana", 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.935 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.937 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_haproxy": { 2024-07-29T00:47:32.937 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_haproxy", 2024-07-29T00:47:32.937 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.937 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.937 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.937 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "HAproxy container image", 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.938 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_agent": { 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_agent", 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.939 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-29T00:47:32.940 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.940 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.940 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.940 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger agent container image", 2024-07-29T00:47:32.940 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.940 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_collector": { 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_collector", 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.941 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger collector container image", 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.942 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_query": { 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_query", 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.943 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger query container image", 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.944 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_keepalived": { 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_keepalived", 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-29T00:47:32.945 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Keepalived container image", 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_loki": { 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_loki", 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-29T00:47:32.947 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Loki container image", 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_node_exporter": { 2024-07-29T00:47:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_node_exporter", 2024-07-29T00:47:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-07-29T00:47:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.950 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_nvmeof": { 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_nvmeof", 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.951 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Nvme-of container image", 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_prometheus": { 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_prometheus", 2024-07-29T00:47:32.952 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:32.953 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.954 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.954 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.954 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_promtail": { 2024-07-29T00:47:32.954 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_promtail", 2024-07-29T00:47:32.954 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.957 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.957 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Promtail container image", 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.958 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_samba": { 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_samba", 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-29T00:47:32.959 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Samba/SMB container image", 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_snmp_gateway": { 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_snmp_gateway", 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.961 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SNMP Gateway container image", 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.962 INFO:teuthology.orchestra.run.smithi175.stdout: "container_init": { 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_init", 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.963 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Run podman/docker with `--init`", 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "daemon_cache_timeout": { 2024-07-29T00:47:32.964 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "daemon_cache_timeout", 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.965 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: "default_cephadm_command_timeout": { 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "default_cephadm_command_timeout", 2024-07-29T00:47:32.966 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "900", 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-29T00:47:32.967 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: "default_registry": { 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "default_registry", 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.968 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io", 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-29T00:47:32.969 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: "device_cache_timeout": { 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_cache_timeout", 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.970 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1800", 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache device inventory", 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.971 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: "device_enhanced_scan": { 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_enhanced_scan", 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.972 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.973 INFO:teuthology.orchestra.run.smithi175.stdout: "facts_cache_timeout": { 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "facts_cache_timeout", 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.974 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache host facts data", 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "grafana_dashboards_path": { 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "grafana_dashboards_path", 2024-07-29T00:47:32.975 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.976 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "host_check_interval": { 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "host_check_interval", 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.977 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to perform a host check", 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "hw_monitoring": { 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "hw_monitoring", 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.979 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "inventory_list_all": { 2024-07-29T00:47:32.980 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "inventory_list_all", 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.981 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.982 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:32.983 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.984 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "log_refresh_metadata": { 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_refresh_metadata", 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.985 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.987 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:32.988 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:32.989 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.990 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.991 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.993 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-29T00:47:32.994 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "*", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "max_count_per_host": { 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_count_per_host", 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:32.996 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.997 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.997 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.997 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "max number of daemons per service per host", 2024-07-29T00:47:32.997 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.997 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.997 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: "max_osd_draining_count": { 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_osd_draining_count", 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:32.998 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:32.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "migration_current": { 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "migration_current", 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.000 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "internal - do not modify", 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": { 2024-07-29T00:47:33.001 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mode", 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "root", 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.002 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm-package", 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "root" 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "mode for remote execution of cephadm", 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "oob_default_addr": { 2024-07-29T00:47:33.003 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "oob_default_addr", 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "169.254.1.1", 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.004 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "prometheus_alerts_path": { 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prometheus_alerts_path", 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.005 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_insecure": { 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_insecure", 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.007 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_password": { 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_password", 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.009 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.010 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.010 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-29T00:47:33.010 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.010 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.010 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_url": { 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_url", 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.011 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_username": { 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_username", 2024-07-29T00:47:33.012 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-29T00:47:33.013 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: "secure_monitoring_stack": { 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "secure_monitoring_stack", 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.014 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.016 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.016 INFO:teuthology.orchestra.run.smithi175.stdout: "service_discovery_port": { 2024-07-29T00:47:33.016 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "service_discovery_port", 2024-07-29T00:47:33.016 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.016 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8765", 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "cephadm service discovery port", 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.017 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: "ssh_config_file": { 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssh_config_file", 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.018 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.019 INFO:teuthology.orchestra.run.smithi175.stdout: "use_agent": { 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "use_agent", 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-29T00:47:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: "use_repo_digest": { 2024-07-29T00:47:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "use_repo_digest", 2024-07-29T00:47:33.022 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.022 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.022 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.022 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.022 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.022 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_failed_host_check": { 2024-07-29T00:47:33.023 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_failed_host_check", 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.024 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if the host check fails", 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_stray_daemons": { 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_stray_daemons", 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.025 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-29T00:47:33.026 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_stray_hosts": { 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_stray_hosts", 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-29T00:47:33.028 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crash", 2024-07-29T00:47:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.032 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.033 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.035 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.036 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.038 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "retain_interval": { 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "retain_interval", 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.039 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.040 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.040 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "31536000", 2024-07-29T00:47:33.040 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.040 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.040 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.040 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_recent_interval": { 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_recent_interval", 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1209600", 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-29T00:47:33.042 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "dashboard", 2024-07-29T00:47:33.043 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.044 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.045 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.045 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.045 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.045 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.045 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.045 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "ALERTMANAGER_API_HOST": { 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.046 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.048 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: "AUDIT_API_ENABLED": { 2024-07-29T00:47:33.049 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "AUDIT_API_ENABLED", 2024-07-29T00:47:33.050 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.050 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.050 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.050 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.050 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.052 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "ENABLE_BROWSABLE_API": { 2024-07-29T00:47:33.053 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.054 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.055 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.055 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.055 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.055 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.056 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-29T00:47:33.057 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-29T00:47:33.058 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.058 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.058 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.058 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.058 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.058 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-29T00:47:33.059 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-29T00:47:33.060 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.060 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.060 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.060 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.060 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.060 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.061 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-29T00:47:33.063 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.064 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.064 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.064 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.066 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-29T00:47:33.066 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-29T00:47:33.066 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.066 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.066 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.066 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.067 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.068 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.069 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.070 INFO:teuthology.orchestra.run.smithi175.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-29T00:47:33.070 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-29T00:47:33.070 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.070 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.070 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.071 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_PASSWORD": { 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.072 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:33.073 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.073 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.073 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.073 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.073 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.073 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_URL": { 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_URL", 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.076 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_USERNAME": { 2024-07-29T00:47:33.077 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.078 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-29T00:47:33.079 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.080 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.081 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.081 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.081 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-29T00:47:33.081 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-29T00:47:33.081 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.082 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.082 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.082 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.082 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.082 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.083 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.083 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.083 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.083 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.084 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.085 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.086 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.087 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "PROMETHEUS_API_HOST": { 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.088 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.089 INFO:teuthology.orchestra.run.smithi175.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-29T00:47:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-29T00:47:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-29T00:47:33.093 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-29T00:47:33.095 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.096 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-29T00:47:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-29T00:47:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.098 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.098 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.098 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.098 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.098 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-29T00:47:33.099 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.100 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.100 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.100 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.100 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.100 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.100 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-29T00:47:33.101 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.102 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-29T00:47:33.103 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.104 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_ENABLED": { 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.105 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.106 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.107 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.109 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-29T00:47:33.109 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-29T00:47:33.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.109 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.109 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.110 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.111 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-29T00:47:33.111 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-29T00:47:33.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.111 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8", 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "45", 2024-07-29T00:47:33.113 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_ACCESS_KEY": { 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.115 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-29T00:47:33.116 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-29T00:47:33.117 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.117 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.117 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.117 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:33.117 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.117 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_SECRET_KEY": { 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.119 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_SSL_VERIFY": { 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.120 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.121 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.122 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-29T00:47:33.125 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.126 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: "cross_origin_url": { 2024-07-29T00:47:33.127 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cross_origin_url", 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.128 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "crt_file": { 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crt_file", 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.129 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.130 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.130 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.130 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.130 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.130 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.130 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.131 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.131 INFO:teuthology.orchestra.run.smithi175.stdout: "debug": { 2024-07-29T00:47:33.131 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "debug", 2024-07-29T00:47:33.131 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable/disable debug options", 2024-07-29T00:47:33.132 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "jwt_token_ttl": { 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "jwt_token_ttl", 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.133 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "28800", 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.134 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "key_file": { 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "key_file", 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.135 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.136 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.137 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.137 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.137 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.137 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.137 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.137 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.138 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.138 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.138 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.138 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.138 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.138 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.139 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.140 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.140 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.140 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.140 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.140 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.141 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.141 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.141 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.141 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.141 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.144 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.144 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.144 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.144 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.145 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.145 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.145 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.145 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.145 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.146 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.146 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.146 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.146 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.146 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.147 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.147 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.147 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.147 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.148 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.148 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.148 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.148 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.149 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.149 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.149 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.149 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.149 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.150 INFO:teuthology.orchestra.run.smithi175.stdout: "motd": { 2024-07-29T00:47:33.150 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "motd", 2024-07-29T00:47:33.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.150 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.151 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.151 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.151 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.151 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.151 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "The message of the day", 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "redirect_resolve_ip_addr": { 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "redirect_resolve_ip_addr", 2024-07-29T00:47:33.152 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.153 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.154 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "::", 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.155 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8080", 2024-07-29T00:47:33.156 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.157 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl": { 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl", 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.158 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.162 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.162 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.162 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.162 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl_server_port": { 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl_server_port", 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8443", 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.163 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_behaviour": { 2024-07-29T00:47:33.164 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_behaviour", 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "redirect", 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: "redirect" 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_error_status_code": { 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_error_status_code", 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "500", 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "400", 2024-07-29T00:47:33.167 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "599", 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.168 INFO:teuthology.orchestra.run.smithi175.stdout: "url_prefix": { 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "url_prefix", 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.169 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.170 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.170 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.170 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.170 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.170 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.170 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "devicehealth", 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.171 INFO:teuthology.orchestra.run.smithi175.stdout: "enable_monitoring": { 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enable_monitoring", 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.172 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "monitor device health metrics", 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.173 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.174 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.176 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.177 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.178 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.179 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.179 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.179 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.179 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.179 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.180 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.180 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.180 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.180 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.181 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.181 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.181 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.181 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.182 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.183 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.183 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.183 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "mark_out_threshold": { 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mark_out_threshold", 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "2419200", 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.184 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "pool_name": { 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pool_name", 2024-07-29T00:47:33.185 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "device_health_metrics", 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-29T00:47:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "retention_period": { 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "retention_period", 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.187 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.188 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15552000", 2024-07-29T00:47:33.188 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.188 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.188 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.188 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long to retain device health metrics", 2024-07-29T00:47:33.188 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "scrape_frequency": { 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "scrape_frequency", 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.189 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.190 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "86400", 2024-07-29T00:47:33.190 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.190 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.190 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.190 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-29T00:47:33.190 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: "self_heal": { 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "self_heal", 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.191 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.192 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.193 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to wake up and check device health", 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.194 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_threshold": { 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_threshold", 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.195 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "7257600", 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.196 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "diskprediction_local", 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.197 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.198 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.198 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.198 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.198 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.198 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.198 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.206 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "predict_interval": { 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "predict_interval", 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.207 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "86400", 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "predictor_model": { 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "predictor_model", 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "prophetstor", 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.209 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.210 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.211 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "influx", 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": false, 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "influxdb python module not found", 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.212 INFO:teuthology.orchestra.run.smithi175.stdout: "batch_size": { 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "batch_size", 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5000", 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: "database": { 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "database", 2024-07-29T00:47:33.214 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "ceph", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.215 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": { 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "hostname", 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.216 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB server hostname", 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.217 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:33.218 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:33.218 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.218 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.218 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.218 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "30", 2024-07-29T00:47:33.218 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "5", 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.220 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.221 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.222 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.223 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.227 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.228 INFO:teuthology.orchestra.run.smithi175.stdout: "password": { 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "password", 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.229 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "password of InfluxDB server user", 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: "port": { 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "port", 2024-07-29T00:47:33.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8086", 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.231 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB server port", 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl": { 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl", 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.232 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "false", 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.233 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "threads": { 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "threads", 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:33.234 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "32", 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.235 INFO:teuthology.orchestra.run.smithi175.stdout: "username": { 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "username", 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.236 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "username of InfluxDB server user", 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "verify_ssl": { 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "verify_ssl", 2024-07-29T00:47:33.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "true", 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-29T00:47:33.238 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "insights", 2024-07-29T00:47:33.239 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.240 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.241 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.242 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.243 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.244 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.245 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.246 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.247 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "iostat", 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.249 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.250 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.251 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.252 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.254 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.256 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.257 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "k8sevents", 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.258 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_event_retention_days": { 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ceph_event_retention_days", 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "7", 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.259 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "config_check_secs": { 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_check_secs", 2024-07-29T00:47:33.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "10", 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-29T00:47:33.261 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.262 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.262 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.262 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.262 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.262 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.263 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.264 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.265 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.266 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.267 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.268 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.269 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.270 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.270 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.270 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.270 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.270 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.270 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "localpool", 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "failure_domain": { 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "failure_domain", 2024-07-29T00:47:33.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "host", 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "failure domain for any created local pool", 2024-07-29T00:47:33.273 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.275 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.276 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.277 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.277 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.277 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.277 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.277 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.277 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.278 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.279 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.280 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.281 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.282 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: "min_size": { 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "min_size", 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.283 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.284 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.284 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.284 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.284 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.284 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default min_size for any created local pool", 2024-07-29T00:47:33.284 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "num_rep": { 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "num_rep", 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.285 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.286 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3", 2024-07-29T00:47:33.286 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.286 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.286 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.286 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default replica count for any created local pool", 2024-07-29T00:47:33.286 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_num": { 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pg_num", 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.287 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "128", 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default pg_num for any created local pool", 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.288 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: "prefix": { 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prefix", 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.289 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "name prefix for any created local pool", 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.290 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "subtree": { 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "subtree", 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "rack", 2024-07-29T00:47:33.291 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mds_autoscaler", 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.293 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.294 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.295 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.296 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.297 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.298 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.298 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.298 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.298 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.298 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.299 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.300 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.301 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.302 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mirroring", 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.303 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.307 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.308 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.309 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.310 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.311 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.312 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.313 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "nfs", 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.315 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.316 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.317 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.318 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.319 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.319 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.319 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.319 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.319 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.322 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.322 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.322 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.322 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.322 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.322 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "orchestrator", 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "fail_fs": { 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "fail_fs", 2024-07-29T00:47:33.324 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-29T00:47:33.325 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.326 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.327 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.328 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.329 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.330 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.331 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.331 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.331 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.331 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.331 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.331 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.332 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.333 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.334 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator": { 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "orchestrator", 2024-07-29T00:47:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm", 2024-07-29T00:47:33.336 INFO:teuthology.orchestra.run.smithi175.stdout: "rook", 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: "test_orchestrator" 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Orchestrator backend", 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.337 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "osd_perf_query", 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.338 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.339 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.340 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.341 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.341 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.341 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.341 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.341 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.342 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.343 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.344 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.345 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.346 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.347 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "osd_support", 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.348 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.349 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.350 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.351 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.351 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.351 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.351 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.352 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.353 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.354 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.355 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.356 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.357 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.358 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.358 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.358 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pg_autoscaler", 2024-07-29T00:47:33.359 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.359 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.359 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.359 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.359 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.359 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.360 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.361 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.362 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.363 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.363 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.363 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.363 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.363 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.364 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.364 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.364 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.364 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.367 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.367 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.367 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.368 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.369 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.371 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.371 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.371 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.371 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.371 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.371 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:33.372 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.373 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: "threshold": { 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "threshold", 2024-07-29T00:47:33.374 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3.0", 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1.0", 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.375 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "scaling threshold", 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "progress", 2024-07-29T00:47:33.376 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_pg_recovery_event": { 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_pg_recovery_event", 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.377 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.378 INFO:teuthology.orchestra.run.smithi175.stdout: "enabled": { 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enabled", 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.379 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.380 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.381 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.381 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.381 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.381 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.381 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.381 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.383 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.385 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.386 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.387 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.388 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "max_completed_events": { 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_completed_events", 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.389 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "50", 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "number of past completed events to remember", 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.390 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long the module is going to sleep", 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prometheus", 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.393 INFO:teuthology.orchestra.run.smithi175.stdout: "cache": { 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cache", 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.394 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "exclude_perf_counters": { 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "exclude_perf_counters", 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.395 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.396 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.397 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.398 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.399 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.400 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.401 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.402 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.403 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.404 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_stats_pools": { 2024-07-29T00:47:33.405 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_stats_pools", 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.406 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.407 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "300", 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "scrape_interval": { 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "scrape_interval", 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15.0", 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.409 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:33.410 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.411 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.411 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.411 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "::", 2024-07-29T00:47:33.411 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.411 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.411 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.412 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "9283", 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.413 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: "stale_cache_strategy": { 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "stale_cache_strategy", 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.414 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "log", 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.415 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_behaviour": { 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_behaviour", 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "default", 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.416 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "default", 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "error" 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.417 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_error_status_code": { 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_error_status_code", 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "500", 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "400", 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "599", 2024-07-29T00:47:33.418 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_support", 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.421 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.422 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.422 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.422 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.422 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.422 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.422 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.423 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.424 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.425 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.426 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.427 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.428 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "max_concurrent_snap_create": { 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_concurrent_snap_create", 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.429 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "mirror_snapshot_schedule": { 2024-07-29T00:47:33.430 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mirror_snapshot_schedule", 2024-07-29T00:47:33.431 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.431 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.431 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.431 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.431 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.431 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.432 INFO:teuthology.orchestra.run.smithi175.stdout: "trash_purge_schedule": { 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "trash_purge_schedule", 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.433 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.434 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "restful", 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "enable_auth": { 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enable_auth", 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.435 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.436 INFO:teuthology.orchestra.run.smithi175.stdout: "key_file": { 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "key_file", 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.437 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.438 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.438 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.439 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.439 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.439 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.440 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.440 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.440 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.440 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.440 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.440 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.441 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.441 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.441 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.441 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.441 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.441 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.442 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.442 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.442 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.442 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.443 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.443 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.443 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.443 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.443 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.444 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.446 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.447 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.448 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.449 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.450 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.451 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.451 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.451 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.451 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.451 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.451 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.452 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:33.452 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:33.452 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.453 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.453 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.453 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.453 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.454 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.454 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.454 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.454 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.454 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.454 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rgw", 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.455 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.456 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.457 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.458 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.458 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.458 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.458 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.458 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.459 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.459 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.459 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.459 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.459 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.459 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.460 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.461 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.461 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.461 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.461 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.462 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.463 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.464 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rook", 2024-07-29T00:47:33.465 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.467 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.467 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.467 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.467 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.467 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.467 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.468 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.468 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.468 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.468 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.468 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.468 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.469 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.470 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.471 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.471 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.471 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.471 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.471 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.472 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.473 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.474 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.475 INFO:teuthology.orchestra.run.smithi175.stdout: "storage_class": { 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "storage_class", 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "local", 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.476 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "selftest", 2024-07-29T00:47:33.477 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.479 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.479 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.479 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.479 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.479 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.480 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.480 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.480 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.480 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.481 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.481 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.481 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.481 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.482 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.482 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.482 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.483 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.483 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.483 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.483 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.484 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.484 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.484 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.485 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.485 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.485 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.485 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.486 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.486 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.486 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.487 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.487 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.487 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.487 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.488 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.488 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.488 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.489 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.489 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.489 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.489 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.490 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.490 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.490 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.491 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.491 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.491 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.491 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.492 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.492 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.492 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.493 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "roption1": { 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "roption1", 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.494 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "roption2": { 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "roption2", 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "xyz", 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.496 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption1": { 2024-07-29T00:47:33.497 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption1", 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.498 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.499 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.499 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.499 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.499 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.499 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption2": { 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption2", 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.500 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.501 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption3": { 2024-07-29T00:47:33.502 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption3", 2024-07-29T00:47:33.502 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.502 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.502 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.502 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.502 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.503 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption4": { 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption4", 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.504 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption5": { 2024-07-29T00:47:33.505 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption5", 2024-07-29T00:47:33.506 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.506 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.506 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.506 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.506 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.506 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption6": { 2024-07-29T00:47:33.507 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption6", 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.508 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption7": { 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption7", 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.509 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "42", 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.510 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "testkey": { 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testkey", 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.511 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.512 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.512 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.512 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.512 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: "testlkey": { 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testlkey", 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.513 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.514 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.515 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.515 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.515 INFO:teuthology.orchestra.run.smithi175.stdout: "testnewline": { 2024-07-29T00:47:33.515 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testnewline", 2024-07-29T00:47:33.516 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.516 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.516 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.516 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.516 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.517 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snap_schedule", 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.518 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_m_granularity": { 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_m_granularity", 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.519 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow minute scheduled snapshots", 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "dump_on_update": { 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "dump_on_update", 2024-07-29T00:47:33.520 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "dump database to debug log on update", 2024-07-29T00:47:33.521 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.523 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.525 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.527 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.528 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.530 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "stats", 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.532 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.533 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.534 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.535 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.536 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.537 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.538 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.540 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.541 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "status", 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.542 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.543 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.543 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.543 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.543 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.543 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.543 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.544 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.544 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.544 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.544 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.545 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.545 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.545 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.546 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.546 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.546 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.546 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.547 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.547 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.547 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.548 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.548 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.548 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.548 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.549 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.550 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.551 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.552 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.553 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.554 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.554 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.554 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.554 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.554 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.554 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "telegraf", 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "address": { 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "address", 2024-07-29T00:47:33.555 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.556 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.557 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15", 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.558 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.559 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.560 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.561 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.562 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.562 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.562 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.562 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.562 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.564 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.565 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.566 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.566 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.566 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.566 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.566 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.567 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.569 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.570 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.570 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.570 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.570 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.572 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "telemetry", 2024-07-29T00:47:33.572 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.572 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.572 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_basic": { 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_basic", 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.573 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.574 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_crash": { 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_crash", 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.575 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_device": { 2024-07-29T00:47:33.576 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_device", 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.577 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_ident": { 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_ident", 2024-07-29T00:47:33.578 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.579 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.579 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.579 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.579 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_perf": { 2024-07-29T00:47:33.580 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_perf", 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.581 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.582 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-29T00:47:33.582 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.582 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.582 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "contact": { 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "contact", 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.583 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.584 INFO:teuthology.orchestra.run.smithi175.stdout: "description": { 2024-07-29T00:47:33.585 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "description", 2024-07-29T00:47:33.585 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.585 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.585 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.585 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.585 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "device_url": { 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_url", 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-29T00:47:33.587 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.588 INFO:teuthology.orchestra.run.smithi175.stdout: "enabled": { 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enabled", 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.589 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:33.590 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "24", 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "8", 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.591 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "last_opt_revision": { 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "last_opt_revision", 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.592 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1", 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.593 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: "leaderboard": { 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "leaderboard", 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.594 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.595 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "leaderboard_description": { 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "leaderboard_description", 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.596 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.597 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.598 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.599 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.599 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.599 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.599 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.599 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.599 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.600 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.601 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.602 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.602 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.602 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.602 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.602 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.603 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.605 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.606 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.606 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.606 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.606 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "organization": { 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "organization", 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.607 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "proxy": { 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "proxy", 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.608 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.609 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: "url": { 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "url", 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.610 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.611 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "test_orchestrator", 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.613 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.614 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.615 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.616 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.617 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.618 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.620 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.620 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.620 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.620 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.620 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.620 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.621 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "volumes", 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.622 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.623 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.623 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.623 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.623 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.623 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.623 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.624 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.625 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.626 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.628 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.629 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.631 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: "max_concurrent_clones": { 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_concurrent_clones", 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.632 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "4", 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.633 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "periodic_async_work": { 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "periodic_async_work", 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.634 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Periodically check for async work", 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.635 INFO:teuthology.orchestra.run.smithi175.stdout: "snapshot_clone_delay": { 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snapshot_clone_delay", 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.636 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.637 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-29T00:47:33.637 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.637 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.637 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.638 INFO:teuthology.orchestra.run.smithi175.stdout: "snapshot_clone_no_wait": { 2024-07-29T00:47:33.638 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snapshot_clone_no_wait", 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.639 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix", 2024-07-29T00:47:33.640 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "discovery_interval": { 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "discovery_interval", 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "100", 2024-07-29T00:47:33.641 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "identifier": { 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "identifier", 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.643 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:33.644 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.645 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.646 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.647 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.648 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.649 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.650 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.651 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.652 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.653 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.654 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_host": { 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_host", 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.655 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_port": { 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_port", 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.656 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10051", 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_sender": { 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_sender", 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-29T00:47:33.658 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: "gid": 25130, 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "y", 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: "mgr_features": 4540701547738038271, 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: "available_modules": [ 2024-07-29T00:47:33.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "alerts", 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.661 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How frequently to reexamine health status", 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.663 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.664 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.665 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.666 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.667 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.668 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.669 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.670 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.670 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.670 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.670 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.671 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_destination": { 2024-07-29T00:47:33.672 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_destination", 2024-07-29T00:47:33.672 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.672 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.672 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.672 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.672 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Email address to send alerts to", 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_from_name": { 2024-07-29T00:47:33.673 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_from_name", 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "Ceph", 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.674 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Email From: name", 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_host": { 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_host", 2024-07-29T00:47:33.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP server", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.676 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_password": { 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_password", 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Password to authenticate with", 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.678 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_port": { 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_port", 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "465", 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.679 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP port", 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_sender": { 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_sender", 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.680 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP envelope sender", 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.681 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_ssl": { 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_ssl", 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.682 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.683 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_user": { 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_user", 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.684 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "User to authenticate as", 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "balancer", 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "active": { 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "active", 2024-07-29T00:47:33.686 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "automatically balance PGs across cluster", 2024-07-29T00:47:33.687 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "begin_time": { 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "begin_time", 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0000", 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "beginning time of day to automatically balance", 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: "begin_weekday": { 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "begin_weekday", 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.690 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0", 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "6", 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_max_iterations": { 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_max_iterations", 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "25", 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "250", 2024-07-29T00:47:33.692 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_metrics": { 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_metrics", 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.693 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "pgs,objects,bytes", 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-29T00:47:33.694 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_step": { 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_step", 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.695 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0.5", 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0.001", 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "0.999", 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "aggressiveness of optimization", 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.696 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "end_time": { 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "end_time", 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "2359", 2024-07-29T00:47:33.697 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.698 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.698 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.698 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "ending time of day to automatically balance", 2024-07-29T00:47:33.698 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-29T00:47:33.698 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.698 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.699 INFO:teuthology.orchestra.run.smithi175.stdout: "end_weekday": { 2024-07-29T00:47:33.699 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "end_weekday", 2024-07-29T00:47:33.699 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:33.700 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.700 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:33.700 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0", 2024-07-29T00:47:33.701 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "6", 2024-07-29T00:47:33.701 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.701 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-29T00:47:33.701 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-29T00:47:33.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.702 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.702 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.705 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.705 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.705 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.705 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.706 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.706 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.706 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.706 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.707 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.708 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.708 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.708 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.708 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.708 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.710 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.710 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.710 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.710 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.710 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.710 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.711 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.712 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.713 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: "min_score": { 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "min_score", 2024-07-29T00:47:33.714 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.715 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.715 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.715 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:33.715 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.715 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.715 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.716 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-29T00:47:33.716 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.716 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.716 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.717 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": { 2024-07-29T00:47:33.717 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mode", 2024-07-29T00:47:33.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.718 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.718 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "upmap", 2024-07-29T00:47:33.719 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.719 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.719 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.719 INFO:teuthology.orchestra.run.smithi175.stdout: "crush-compat", 2024-07-29T00:47:33.719 INFO:teuthology.orchestra.run.smithi175.stdout: "none", 2024-07-29T00:47:33.720 INFO:teuthology.orchestra.run.smithi175.stdout: "read", 2024-07-29T00:47:33.720 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap", 2024-07-29T00:47:33.720 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap-read" 2024-07-29T00:47:33.720 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.721 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Balancer mode", 2024-07-29T00:47:33.721 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.721 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.721 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.722 INFO:teuthology.orchestra.run.smithi175.stdout: "pool_ids": { 2024-07-29T00:47:33.722 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pool_ids", 2024-07-29T00:47:33.723 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.723 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.723 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.723 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.724 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.724 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.724 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.724 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-29T00:47:33.724 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.724 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.725 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.726 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap_max_deviation": { 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "upmap_max_deviation", 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:33.727 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap_max_optimizations": { 2024-07-29T00:47:33.728 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "upmap_max_optimizations", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-29T00:47:33.729 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.730 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.730 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.731 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cephadm", 2024-07-29T00:47:33.731 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.731 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.731 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.731 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_down_multiplier": { 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_down_multiplier", 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3.0", 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.732 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_refresh_rate": { 2024-07-29T00:47:33.733 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_refresh_rate", 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "20", 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.734 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_starting_port": { 2024-07-29T00:47:33.735 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_starting_port", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "4721", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-29T00:47:33.736 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_ptrace": { 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_ptrace", 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.737 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.738 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.738 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.738 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.738 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-29T00:47:33.738 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-29T00:47:33.738 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: "autotune_interval": { 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "autotune_interval", 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "autotune_memory_target_ratio": { 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "autotune_memory_target_ratio", 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0.7", 2024-07-29T00:47:33.741 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm_log_destination": { 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cephadm_log_destination", 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.743 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "file", 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "file,syslog", 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "syslog" 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-29T00:47:33.744 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "cgroups_split": { 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cgroups_split", 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.745 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "config_checks_enabled": { 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_checks_enabled", 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.747 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "config_dashboard": { 2024-07-29T00:47:33.748 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_dashboard", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-29T00:47:33.749 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_alertmanager": { 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_alertmanager", 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.750 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.751 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_base": { 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_base", 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.752 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Container image name, without the tag", 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_elasticsearch": { 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_elasticsearch", 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-29T00:47:33.754 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "elasticsearch container image", 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.755 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_grafana": { 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_grafana", 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.756 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_haproxy": { 2024-07-29T00:47:33.757 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_haproxy", 2024-07-29T00:47:33.758 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.758 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.758 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.758 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-29T00:47:33.758 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.758 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "HAproxy container image", 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_agent": { 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_agent", 2024-07-29T00:47:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger agent container image", 2024-07-29T00:47:33.760 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_collector": { 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_collector", 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger collector container image", 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.763 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_query": { 2024-07-29T00:47:33.763 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_query", 2024-07-29T00:47:33.763 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.763 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.763 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger query container image", 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.764 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_keepalived": { 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_keepalived", 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-29T00:47:33.765 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Keepalived container image", 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_loki": { 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_loki", 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.767 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Loki container image", 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_node_exporter": { 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_node_exporter", 2024-07-29T00:47:33.768 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.769 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.770 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.770 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_nvmeof": { 2024-07-29T00:47:33.770 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_nvmeof", 2024-07-29T00:47:33.770 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Nvme-of container image", 2024-07-29T00:47:33.771 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.772 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.772 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.772 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_prometheus": { 2024-07-29T00:47:33.772 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_prometheus", 2024-07-29T00:47:33.772 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.774 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.774 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.774 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-07-29T00:47:33.774 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.774 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.774 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_promtail": { 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_promtail", 2024-07-29T00:47:33.775 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Promtail container image", 2024-07-29T00:47:33.776 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_samba": { 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_samba", 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.777 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Samba/SMB container image", 2024-07-29T00:47:33.778 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_snmp_gateway": { 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_snmp_gateway", 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-29T00:47:33.779 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SNMP Gateway container image", 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.780 INFO:teuthology.orchestra.run.smithi175.stdout: "container_init": { 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_init", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Run podman/docker with `--init`", 2024-07-29T00:47:33.781 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: "daemon_cache_timeout": { 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "daemon_cache_timeout", 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.782 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.783 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "default_cephadm_command_timeout": { 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "default_cephadm_command_timeout", 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "900", 2024-07-29T00:47:33.784 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "default_registry": { 2024-07-29T00:47:33.785 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "default_registry", 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io", 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.786 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "device_cache_timeout": { 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_cache_timeout", 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.787 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1800", 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache device inventory", 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.788 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "device_enhanced_scan": { 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_enhanced_scan", 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.789 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.790 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: "facts_cache_timeout": { 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "facts_cache_timeout", 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.791 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache host facts data", 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "grafana_dashboards_path": { 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "grafana_dashboards_path", 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-29T00:47:33.793 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "host_check_interval": { 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "host_check_interval", 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:33.795 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to perform a host check", 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "hw_monitoring": { 2024-07-29T00:47:33.796 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "hw_monitoring", 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.797 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "inventory_list_all": { 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "inventory_list_all", 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.798 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.799 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.800 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.801 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "log_refresh_metadata": { 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_refresh_metadata", 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.802 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.804 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.806 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.807 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.807 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.807 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.807 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.807 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.807 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.808 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.810 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.811 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "*", 2024-07-29T00:47:33.812 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.813 INFO:teuthology.orchestra.run.smithi175.stdout: "max_count_per_host": { 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_count_per_host", 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.814 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "max number of daemons per service per host", 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "max_osd_draining_count": { 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_osd_draining_count", 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.815 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.816 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "migration_current": { 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "migration_current", 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.817 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "internal - do not modify", 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.818 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": { 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mode", 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "root", 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.819 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm-package", 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "root" 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "mode for remote execution of cephadm", 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "oob_default_addr": { 2024-07-29T00:47:33.820 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "oob_default_addr", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "169.254.1.1", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-29T00:47:33.821 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "prometheus_alerts_path": { 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prometheus_alerts_path", 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.822 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.823 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_insecure": { 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_insecure", 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_password": { 2024-07-29T00:47:33.825 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_password", 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.826 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_url": { 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_url", 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.827 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.828 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_username": { 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_username", 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.829 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.830 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.830 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-29T00:47:33.830 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.830 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.830 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "secure_monitoring_stack": { 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "secure_monitoring_stack", 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "service_discovery_port": { 2024-07-29T00:47:33.832 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "service_discovery_port", 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8765", 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.833 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "cephadm service discovery port", 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: "ssh_config_file": { 2024-07-29T00:47:33.834 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssh_config_file", 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.835 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "use_agent": { 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "use_agent", 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.836 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.837 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "use_repo_digest": { 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "use_repo_digest", 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.838 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.840 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_failed_host_check": { 2024-07-29T00:47:33.840 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_failed_host_check", 2024-07-29T00:47:33.840 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.840 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if the host check fails", 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_stray_daemons": { 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_stray_daemons", 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.842 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.843 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.843 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-29T00:47:33.843 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.843 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.843 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_stray_hosts": { 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_stray_hosts", 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.844 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crash", 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.846 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.847 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.848 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.849 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.851 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.852 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.853 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.854 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: "retain_interval": { 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "retain_interval", 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.855 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "31536000", 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_recent_interval": { 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_recent_interval", 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1209600", 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.857 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.858 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-29T00:47:33.858 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.858 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.858 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.858 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.858 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "dashboard", 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.859 INFO:teuthology.orchestra.run.smithi175.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.860 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "ALERTMANAGER_API_HOST": { 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-29T00:47:33.861 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.862 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.863 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.864 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "AUDIT_API_ENABLED": { 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "AUDIT_API_ENABLED", 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.865 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-29T00:47:33.866 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.867 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "ENABLE_BROWSABLE_API": { 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.868 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.869 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.870 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.871 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.872 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.873 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.874 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.875 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.877 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-29T00:47:33.878 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.879 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.881 INFO:teuthology.orchestra.run.smithi175.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.882 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_PASSWORD": { 2024-07-29T00:47:33.883 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.884 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-29T00:47:33.885 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.887 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.887 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.887 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_URL": { 2024-07-29T00:47:33.889 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_URL", 2024-07-29T00:47:33.890 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.890 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.890 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.891 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.891 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.891 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.891 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.891 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.892 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.892 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.892 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.893 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_USERNAME": { 2024-07-29T00:47:33.893 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-29T00:47:33.893 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.893 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.894 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.895 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.896 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-29T00:47:33.898 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.899 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.900 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.901 INFO:teuthology.orchestra.run.smithi175.stdout: "PROMETHEUS_API_HOST": { 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.902 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-29T00:47:33.903 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.904 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.905 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.906 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.906 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.906 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.906 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.906 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.906 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.907 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.908 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.909 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.909 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.909 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-29T00:47:33.909 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-29T00:47:33.909 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.910 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-29T00:47:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-29T00:47:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-29T00:47:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_ENABLED": { 2024-07-29T00:47:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-29T00:47:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-29T00:47:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-29T00:47:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-29T00:47:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-29T00:47:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-29T00:47:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-29T00:47:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8", 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "45", 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_ACCESS_KEY": { 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-29T00:47:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-29T00:47:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_SECRET_KEY": { 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_SSL_VERIFY": { 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-29T00:47:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-29T00:47:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "cross_origin_url": { 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cross_origin_url", 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "crt_file": { 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crt_file", 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "debug": { 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "debug", 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable/disable debug options", 2024-07-29T00:47:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "jwt_token_ttl": { 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "jwt_token_ttl", 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "28800", 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "key_file": { 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "key_file", 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "motd": { 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "motd", 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "The message of the day", 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "redirect_resolve_ip_addr": { 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "redirect_resolve_ip_addr", 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "::", 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8080", 2024-07-29T00:47:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl": { 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl", 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl_server_port": { 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl_server_port", 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8443", 2024-07-29T00:47:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_behaviour": { 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_behaviour", 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "redirect", 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "redirect" 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_error_status_code": { 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_error_status_code", 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "500", 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "400", 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "599", 2024-07-29T00:47:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "url_prefix": { 2024-07-29T00:47:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "url_prefix", 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "devicehealth", 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "enable_monitoring": { 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enable_monitoring", 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "monitor device health metrics", 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.990 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.990 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:33.990 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:33.991 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:33.991 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.991 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.991 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.992 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.992 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:33.992 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:33.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:33.993 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.993 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:33.994 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.994 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.994 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:33.995 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:33.995 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:33.995 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:33.995 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:33.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:33.996 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:33.996 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:33.997 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:33.997 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:33.997 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:33.997 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:33.998 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:33.998 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:33.998 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:33.998 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.000 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.000 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.000 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.000 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.001 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.001 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.002 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.002 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.002 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.002 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.003 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.003 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.003 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.004 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.004 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.004 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.004 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.005 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.005 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.005 INFO:teuthology.orchestra.run.smithi175.stdout: "mark_out_threshold": { 2024-07-29T00:47:34.006 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mark_out_threshold", 2024-07-29T00:47:34.006 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.006 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.006 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.007 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "2419200", 2024-07-29T00:47:34.007 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.007 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.008 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.008 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-29T00:47:34.008 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.008 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.009 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.009 INFO:teuthology.orchestra.run.smithi175.stdout: "pool_name": { 2024-07-29T00:47:34.009 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pool_name", 2024-07-29T00:47:34.010 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.010 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.010 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.010 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "device_health_metrics", 2024-07-29T00:47:34.011 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.011 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.011 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.012 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-29T00:47:34.012 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.012 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.012 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.013 INFO:teuthology.orchestra.run.smithi175.stdout: "retention_period": { 2024-07-29T00:47:34.013 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "retention_period", 2024-07-29T00:47:34.013 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.014 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.014 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.014 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15552000", 2024-07-29T00:47:34.015 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.015 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.015 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.016 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long to retain device health metrics", 2024-07-29T00:47:34.016 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.016 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.016 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.017 INFO:teuthology.orchestra.run.smithi175.stdout: "scrape_frequency": { 2024-07-29T00:47:34.017 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "scrape_frequency", 2024-07-29T00:47:34.017 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.018 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.018 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.018 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "86400", 2024-07-29T00:47:34.019 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.019 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.019 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.019 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-29T00:47:34.020 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.020 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.020 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.021 INFO:teuthology.orchestra.run.smithi175.stdout: "self_heal": { 2024-07-29T00:47:34.021 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "self_heal", 2024-07-29T00:47:34.021 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.021 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.022 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.022 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.022 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.023 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.023 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.023 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-29T00:47:34.023 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.024 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.024 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.024 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.025 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.025 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.025 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.026 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.026 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.026 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.026 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to wake up and check device health", 2024-07-29T00:47:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_threshold": { 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_threshold", 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "7257600", 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.028 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "diskprediction_local", 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.030 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.032 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.033 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.034 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.034 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.034 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.034 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.034 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.034 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.035 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.036 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.037 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.038 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "predict_interval": { 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "predict_interval", 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "86400", 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.040 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "predictor_model": { 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "predictor_model", 2024-07-29T00:47:34.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "prophetstor", 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.042 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.043 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "influx", 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": false, 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "influxdb python module not found", 2024-07-29T00:47:34.045 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "batch_size": { 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "batch_size", 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5000", 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.046 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "database": { 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "database", 2024-07-29T00:47:34.047 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "ceph", 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-29T00:47:34.048 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": { 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "hostname", 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB server hostname", 2024-07-29T00:47:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.051 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "30", 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "5", 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.052 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.053 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.054 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.055 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.056 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.057 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.058 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.058 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.058 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.058 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.058 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.058 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.059 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.060 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "password": { 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "password", 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.062 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.062 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.062 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "password of InfluxDB server user", 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "port": { 2024-07-29T00:47:34.063 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "port", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8086", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB server port", 2024-07-29T00:47:34.064 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl": { 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl", 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.065 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "false", 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "threads": { 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "threads", 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "32", 2024-07-29T00:47:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "username": { 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "username", 2024-07-29T00:47:34.068 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.069 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "username of InfluxDB server user", 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "verify_ssl": { 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "verify_ssl", 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.070 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.071 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.071 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "true", 2024-07-29T00:47:34.071 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.071 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.071 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.071 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.072 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "insights", 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.074 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.075 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.075 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.075 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.075 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.075 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.076 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.078 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.079 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.080 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.081 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.082 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.082 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.082 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.082 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.083 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "iostat", 2024-07-29T00:47:34.084 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.084 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.084 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.084 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.085 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.085 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.085 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.086 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.086 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.086 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.086 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.087 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.087 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.087 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.087 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.087 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.088 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.088 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.088 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.088 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.089 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.089 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.090 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.090 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.090 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.090 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.091 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.091 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.091 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.091 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.091 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.091 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.093 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.094 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.095 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "k8sevents", 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_event_retention_days": { 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ceph_event_retention_days", 2024-07-29T00:47:34.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "7", 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-29T00:47:34.098 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "config_check_secs": { 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_check_secs", 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.099 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "10", 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.100 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.102 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.103 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.104 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.106 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.107 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.109 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "localpool", 2024-07-29T00:47:34.110 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "failure_domain": { 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "failure_domain", 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.111 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "host", 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "failure domain for any created local pool", 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.112 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.113 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.114 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.116 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.118 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.120 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.121 INFO:teuthology.orchestra.run.smithi175.stdout: "min_size": { 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "min_size", 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default min_size for any created local pool", 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "num_rep": { 2024-07-29T00:47:34.123 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "num_rep", 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3", 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.124 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default replica count for any created local pool", 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_num": { 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pg_num", 2024-07-29T00:47:34.125 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.126 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.126 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.126 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "128", 2024-07-29T00:47:34.126 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.126 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default pg_num for any created local pool", 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "prefix": { 2024-07-29T00:47:34.127 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prefix", 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.128 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "name prefix for any created local pool", 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "subtree": { 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "subtree", 2024-07-29T00:47:34.129 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "rack", 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.130 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mds_autoscaler", 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.132 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.133 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.134 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.135 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.136 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.137 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.138 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.139 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.140 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mirroring", 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.141 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.142 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.143 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.144 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.145 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.146 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.147 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.148 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.149 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.150 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "nfs", 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.151 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.152 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.155 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.156 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "orchestrator", 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "fail_fs": { 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "fail_fs", 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.160 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.162 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.163 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.164 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.165 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.166 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.168 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator": { 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "orchestrator", 2024-07-29T00:47:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm", 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: "rook", 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: "test_orchestrator" 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Orchestrator backend", 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "osd_perf_query", 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.173 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.174 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.175 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.176 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.177 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.179 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.181 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "osd_support", 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pg_autoscaler", 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.194 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.195 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.196 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.197 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.198 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.199 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.201 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.202 INFO:teuthology.orchestra.run.smithi175.stdout: "threshold": { 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "threshold", 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3.0", 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1.0", 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.203 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "scaling threshold", 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.205 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "progress", 2024-07-29T00:47:34.205 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.205 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.205 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.205 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_pg_recovery_event": { 2024-07-29T00:47:34.206 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_pg_recovery_event", 2024-07-29T00:47:34.206 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.206 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.206 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.207 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.207 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.207 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.207 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.208 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-29T00:47:34.208 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.208 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.208 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "enabled": { 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enabled", 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.209 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.211 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.213 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.215 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.216 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.218 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "max_completed_events": { 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_completed_events", 2024-07-29T00:47:34.219 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "50", 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "number of past completed events to remember", 2024-07-29T00:47:34.220 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long the module is going to sleep", 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.222 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prometheus", 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.223 INFO:teuthology.orchestra.run.smithi175.stdout: "cache": { 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cache", 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.224 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "exclude_perf_counters": { 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "exclude_perf_counters", 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.225 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-29T00:47:34.226 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.227 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.228 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.229 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.230 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.231 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.234 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.235 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.235 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.235 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.235 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.235 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.235 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_stats_pools": { 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_stats_pools", 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.236 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.237 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.238 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "300", 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.239 INFO:teuthology.orchestra.run.smithi175.stdout: "scrape_interval": { 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "scrape_interval", 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15.0", 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.240 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.241 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "::", 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.242 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.243 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "9283", 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "stale_cache_strategy": { 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "stale_cache_strategy", 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "log", 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.245 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_behaviour": { 2024-07-29T00:47:34.246 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_behaviour", 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "default", 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.247 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: "default", 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: "error" 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.248 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_error_status_code": { 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_error_status_code", 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "500", 2024-07-29T00:47:34.249 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "400", 2024-07-29T00:47:34.250 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "599", 2024-07-29T00:47:34.250 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.250 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.250 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.251 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.252 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_support", 2024-07-29T00:47:34.252 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.253 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.253 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.253 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.253 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.253 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.256 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.258 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.259 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.261 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: "max_concurrent_snap_create": { 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_concurrent_snap_create", 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.262 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.263 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "mirror_snapshot_schedule": { 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mirror_snapshot_schedule", 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.264 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.265 INFO:teuthology.orchestra.run.smithi175.stdout: "trash_purge_schedule": { 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "trash_purge_schedule", 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.266 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.267 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "restful", 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "enable_auth": { 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enable_auth", 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.269 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: "key_file": { 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "key_file", 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.270 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.271 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.272 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.273 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.274 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.275 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.275 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.275 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.275 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.275 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.275 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.276 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.277 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.278 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.279 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.279 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.279 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.279 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.279 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.280 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.280 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.281 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.281 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.281 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.281 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.282 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.282 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.282 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.282 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.283 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.283 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.283 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.284 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:34.284 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:34.284 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.284 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.285 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.285 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.285 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.286 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.286 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.286 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.287 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.287 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.288 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:34.288 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:34.288 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.288 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.289 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.290 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.290 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.290 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.291 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rgw", 2024-07-29T00:47:34.291 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.291 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.293 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.294 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.295 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.296 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.297 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.298 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.300 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rook", 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.301 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.302 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.302 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.302 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.302 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.302 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.302 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.303 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.304 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.305 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.306 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.308 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.308 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.308 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.308 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.308 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.308 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.309 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.310 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.311 INFO:teuthology.orchestra.run.smithi175.stdout: "storage_class": { 2024-07-29T00:47:34.311 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "storage_class", 2024-07-29T00:47:34.311 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.311 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.311 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "local", 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.312 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "selftest", 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.313 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.315 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.316 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.318 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.319 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.321 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.323 INFO:teuthology.orchestra.run.smithi175.stdout: "roption1": { 2024-07-29T00:47:34.323 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "roption1", 2024-07-29T00:47:34.323 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.323 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.324 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.324 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.324 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.324 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.325 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.325 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.325 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.325 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.326 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.326 INFO:teuthology.orchestra.run.smithi175.stdout: "roption2": { 2024-07-29T00:47:34.327 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "roption2", 2024-07-29T00:47:34.327 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.327 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.327 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.328 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "xyz", 2024-07-29T00:47:34.328 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.328 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.328 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.329 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.329 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.329 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.329 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.330 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption1": { 2024-07-29T00:47:34.330 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption1", 2024-07-29T00:47:34.330 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.330 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.330 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.331 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.331 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.331 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.331 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.332 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.332 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.332 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.332 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.332 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption2": { 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption2", 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.333 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption3": { 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption3", 2024-07-29T00:47:34.334 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.335 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption4": { 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption4", 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.336 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.337 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.337 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.337 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.337 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.338 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption5": { 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption5", 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption6": { 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption6", 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.340 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.341 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption7": { 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption7", 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.342 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "42", 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "testkey": { 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testkey", 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.344 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "testlkey": { 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testlkey", 2024-07-29T00:47:34.345 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.346 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "testnewline": { 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testnewline", 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.347 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snap_schedule", 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_m_granularity": { 2024-07-29T00:47:34.349 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_m_granularity", 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.350 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow minute scheduled snapshots", 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "dump_on_update": { 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "dump_on_update", 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.351 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "dump database to debug log on update", 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.352 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.353 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.354 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.355 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.357 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.358 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.359 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.360 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.360 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.360 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.360 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.360 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.361 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "stats", 2024-07-29T00:47:34.362 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.363 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.364 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.365 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.366 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.367 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.368 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.369 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.370 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.371 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "status", 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.373 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.374 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.375 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.376 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.377 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.378 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.379 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.381 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.382 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.382 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.382 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.382 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.382 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "telegraf", 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.386 INFO:teuthology.orchestra.run.smithi175.stdout: "address": { 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "address", 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.387 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.388 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15", 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.390 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.392 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.393 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.394 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.397 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "telemetry", 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_basic": { 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_basic", 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.399 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_crash": { 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_crash", 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.401 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.402 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_device": { 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_device", 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.403 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_ident": { 2024-07-29T00:47:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_ident", 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_perf": { 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_perf", 2024-07-29T00:47:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-29T00:47:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "contact": { 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "contact", 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "description": { 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "description", 2024-07-29T00:47:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "device_url": { 2024-07-29T00:47:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_url", 2024-07-29T00:47:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "enabled": { 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enabled", 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "24", 2024-07-29T00:47:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "8", 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "last_opt_revision": { 2024-07-29T00:47:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "last_opt_revision", 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1", 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "leaderboard": { 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "leaderboard", 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "leaderboard_description": { 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "leaderboard_description", 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "organization": { 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "organization", 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "proxy": { 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "proxy", 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "url": { 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "url", 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-29T00:47:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "test_orchestrator", 2024-07-29T00:47:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "volumes", 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "max_concurrent_clones": { 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_concurrent_clones", 2024-07-29T00:47:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "4", 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-29T00:47:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "periodic_async_work": { 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "periodic_async_work", 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Periodically check for async work", 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "snapshot_clone_delay": { 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snapshot_clone_delay", 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "snapshot_clone_no_wait": { 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snapshot_clone_no_wait", 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-29T00:47:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix", 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "discovery_interval": { 2024-07-29T00:47:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "discovery_interval", 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "100", 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "identifier": { 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "identifier", 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_host": { 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_host", 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_port": { 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_port", 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10051", 2024-07-29T00:47:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_sender": { 2024-07-29T00:47:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_sender", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "modules": [ 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm", 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "dashboard", 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "iostat", 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "nfs", 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "restful", 2024-07-29T00:47:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "snap_schedule" 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "available_modules": [ 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "alerts", 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How frequently to reexamine health status", 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_destination": { 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_destination", 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Email address to send alerts to", 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_from_name": { 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_from_name", 2024-07-29T00:47:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "Ceph", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Email From: name", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_host": { 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_host", 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP server", 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_password": { 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_password", 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Password to authenticate with", 2024-07-29T00:47:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_port": { 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_port", 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "465", 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP port", 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_sender": { 2024-07-29T00:47:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_sender", 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SMTP envelope sender", 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_ssl": { 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_ssl", 2024-07-29T00:47:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "smtp_user": { 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "smtp_user", 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.515 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "User to authenticate as", 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "balancer", 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "active": { 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "active", 2024-07-29T00:47:34.517 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.518 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "automatically balance PGs across cluster", 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "begin_time": { 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "begin_time", 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.519 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0000", 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "beginning time of day to automatically balance", 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.520 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "begin_weekday": { 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "begin_weekday", 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0", 2024-07-29T00:47:34.521 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "6", 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_max_iterations": { 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_max_iterations", 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "25", 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "250", 2024-07-29T00:47:34.523 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_metrics": { 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_metrics", 2024-07-29T00:47:34.524 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "pgs,objects,bytes", 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-29T00:47:34.525 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: "crush_compat_step": { 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crush_compat_step", 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.526 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0.5", 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0.001", 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "0.999", 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "aggressiveness of optimization", 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.527 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "end_time": { 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "end_time", 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "2359", 2024-07-29T00:47:34.528 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "ending time of day to automatically balance", 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "end_weekday": { 2024-07-29T00:47:34.529 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "end_weekday", 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "0", 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "6", 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.531 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.532 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.533 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.534 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.535 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.536 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.537 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.538 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.539 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "min_score": { 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "min_score", 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.540 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": { 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mode", 2024-07-29T00:47:34.541 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "upmap", 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "crush-compat", 2024-07-29T00:47:34.542 INFO:teuthology.orchestra.run.smithi175.stdout: "none", 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: "read", 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap", 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap-read" 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Balancer mode", 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.543 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "pool_ids": { 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pool_ids", 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.544 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.545 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.546 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap_max_deviation": { 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "upmap_max_deviation", 2024-07-29T00:47:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-29T00:47:34.548 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: "upmap_max_optimizations": { 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "upmap_max_optimizations", 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.549 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.550 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cephadm", 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.551 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_down_multiplier": { 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_down_multiplier", 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3.0", 2024-07-29T00:47:34.552 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_refresh_rate": { 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_refresh_rate", 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "20", 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.554 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.555 INFO:teuthology.orchestra.run.smithi175.stdout: "agent_starting_port": { 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "agent_starting_port", 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "4721", 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.556 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_ptrace": { 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_ptrace", 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.557 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-29T00:47:34.558 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "autotune_interval": { 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "autotune_interval", 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.559 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "autotune_memory_target_ratio": { 2024-07-29T00:47:34.560 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "autotune_memory_target_ratio", 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0.7", 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.561 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm_log_destination": { 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cephadm_log_destination", 2024-07-29T00:47:34.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "file", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "file,syslog", 2024-07-29T00:47:34.563 INFO:teuthology.orchestra.run.smithi175.stdout: "syslog" 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: "cgroups_split": { 2024-07-29T00:47:34.564 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cgroups_split", 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.565 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "config_checks_enabled": { 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_checks_enabled", 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.566 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.567 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "config_dashboard": { 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_dashboard", 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.568 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_alertmanager": { 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_alertmanager", 2024-07-29T00:47:34.569 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:34.570 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_base": { 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_base", 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.571 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Container image name, without the tag", 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.572 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_elasticsearch": { 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_elasticsearch", 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.573 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "elasticsearch container image", 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_grafana": { 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_grafana", 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.575 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_haproxy": { 2024-07-29T00:47:34.576 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_haproxy", 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.577 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "HAproxy container image", 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_agent": { 2024-07-29T00:47:34.578 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_agent", 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.579 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger agent container image", 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_collector": { 2024-07-29T00:47:34.580 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_collector", 2024-07-29T00:47:34.581 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.581 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.581 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.581 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-29T00:47:34.581 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.581 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger collector container image", 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_jaeger_query": { 2024-07-29T00:47:34.582 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_jaeger_query", 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.583 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Jaeger query container image", 2024-07-29T00:47:34.584 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.584 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.584 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.584 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_keepalived": { 2024-07-29T00:47:34.584 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_keepalived", 2024-07-29T00:47:34.585 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.585 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.585 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.585 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-29T00:47:34.585 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.585 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Keepalived container image", 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.586 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_loki": { 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_loki", 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.587 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Loki container image", 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_node_exporter": { 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_node_exporter", 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.588 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.589 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_nvmeof": { 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_nvmeof", 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.590 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Nvme-of container image", 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_prometheus": { 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_prometheus", 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.592 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Prometheus container image", 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_promtail": { 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_promtail", 2024-07-29T00:47:34.593 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.594 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.594 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.594 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-29T00:47:34.594 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.594 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.594 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Promtail container image", 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_samba": { 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_samba", 2024-07-29T00:47:34.595 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Samba/SMB container image", 2024-07-29T00:47:34.596 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "container_image_snmp_gateway": { 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_image_snmp_gateway", 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.597 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "SNMP Gateway container image", 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.598 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: "container_init": { 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "container_init", 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.599 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Run podman/docker with `--init`", 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "daemon_cache_timeout": { 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "daemon_cache_timeout", 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.601 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "default_cephadm_command_timeout": { 2024-07-29T00:47:34.602 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "default_cephadm_command_timeout", 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "900", 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.603 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "default_registry": { 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "default_registry", 2024-07-29T00:47:34.604 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "docker.io", 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.605 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: "device_cache_timeout": { 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_cache_timeout", 2024-07-29T00:47:34.606 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1800", 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache device inventory", 2024-07-29T00:47:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "device_enhanced_scan": { 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_enhanced_scan", 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: "facts_cache_timeout": { 2024-07-29T00:47:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "facts_cache_timeout", 2024-07-29T00:47:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "seconds to cache host facts data", 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "grafana_dashboards_path": { 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "grafana_dashboards_path", 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: "host_check_interval": { 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "host_check_interval", 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to perform a host check", 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "hw_monitoring": { 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "hw_monitoring", 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "inventory_list_all": { 2024-07-29T00:47:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "inventory_list_all", 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: "log_refresh_metadata": { 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_refresh_metadata", 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.622 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-29T00:47:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.624 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.625 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.626 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.627 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.628 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.629 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.630 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.631 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.632 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "*", 2024-07-29T00:47:34.633 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.633 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.633 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.633 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-29T00:47:34.633 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: "max_count_per_host": { 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_count_per_host", 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.634 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "max number of daemons per service per host", 2024-07-29T00:47:34.635 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.636 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.636 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "max_osd_draining_count": { 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_osd_draining_count", 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.637 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "migration_current": { 2024-07-29T00:47:34.638 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "migration_current", 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.639 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "internal - do not modify", 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: "mode": { 2024-07-29T00:47:34.640 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mode", 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "root", 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.641 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm-package", 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: "root" 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "mode for remote execution of cephadm", 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "oob_default_addr": { 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "oob_default_addr", 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "169.254.1.1", 2024-07-29T00:47:34.643 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.644 INFO:teuthology.orchestra.run.smithi175.stdout: "prometheus_alerts_path": { 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prometheus_alerts_path", 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.645 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_insecure": { 2024-07-29T00:47:34.646 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_insecure", 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.647 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_password": { 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_password", 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.648 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.649 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_url": { 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_url", 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.652 INFO:teuthology.orchestra.run.smithi175.stdout: "registry_username": { 2024-07-29T00:47:34.652 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "registry_username", 2024-07-29T00:47:34.652 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.652 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.652 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.652 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "secure_monitoring_stack": { 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "secure_monitoring_stack", 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: "service_discovery_port": { 2024-07-29T00:47:34.655 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "service_discovery_port", 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8765", 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.656 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "cephadm service discovery port", 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "ssh_config_file": { 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssh_config_file", 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.657 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-29T00:47:34.658 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "use_agent": { 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "use_agent", 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.659 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.660 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "use_repo_digest": { 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "use_repo_digest", 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.661 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.662 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.662 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.662 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-29T00:47:34.662 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.662 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.662 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_failed_host_check": { 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_failed_host_check", 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.663 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if the host check fails", 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_stray_daemons": { 2024-07-29T00:47:34.664 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_stray_daemons", 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.665 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_on_stray_hosts": { 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_on_stray_hosts", 2024-07-29T00:47:34.666 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-29T00:47:34.667 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crash", 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.668 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.669 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.670 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.671 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.672 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.673 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.674 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.675 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.676 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "retain_interval": { 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "retain_interval", 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "31536000", 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.678 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_recent_interval": { 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_recent_interval", 2024-07-29T00:47:34.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1209600", 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-29T00:47:34.680 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.681 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "dashboard", 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.682 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.683 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "ALERTMANAGER_API_HOST": { 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.686 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.687 INFO:teuthology.orchestra.run.smithi175.stdout: "AUDIT_API_ENABLED": { 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "AUDIT_API_ENABLED", 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.688 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.689 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.690 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: "ENABLE_BROWSABLE_API": { 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.691 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.692 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.693 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.694 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.695 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-29T00:47:34.695 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-29T00:47:34.695 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.695 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.695 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.696 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.696 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.696 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.696 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.697 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.697 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.697 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.697 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.698 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-29T00:47:34.699 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.700 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.701 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.702 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.704 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.704 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.704 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.704 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.704 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.704 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.705 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.706 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.706 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.706 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.706 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.707 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.707 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.707 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.707 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.708 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.708 INFO:teuthology.orchestra.run.smithi175.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-29T00:47:34.708 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.709 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.710 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.710 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.710 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.710 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_PASSWORD": { 2024-07-29T00:47:34.710 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.711 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-29T00:47:34.712 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.713 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.714 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.714 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.714 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.714 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_URL": { 2024-07-29T00:47:34.714 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_URL", 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.715 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_API_USERNAME": { 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.717 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.718 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.719 INFO:teuthology.orchestra.run.smithi175.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.720 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-29T00:47:34.721 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.722 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.723 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.724 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: "PROMETHEUS_API_HOST": { 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.725 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.726 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.727 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.728 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.729 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.730 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.731 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-29T00:47:34.732 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.733 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.734 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.734 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.734 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.734 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.735 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.736 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-29T00:47:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-29T00:47:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.739 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-29T00:47:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.741 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_ENABLED": { 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.742 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.743 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-29T00:47:34.744 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-29T00:47:34.745 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.746 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.747 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8", 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.748 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "45", 2024-07-29T00:47:34.749 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_ACCESS_KEY": { 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.751 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-29T00:47:34.752 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "admin", 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.753 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_SECRET_KEY": { 2024-07-29T00:47:34.754 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "RGW_API_SSL_VERIFY": { 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.756 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.757 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.758 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.761 INFO:teuthology.orchestra.run.smithi175.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.762 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "cross_origin_url": { 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cross_origin_url", 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.763 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.764 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "crt_file": { 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "crt_file", 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.765 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "debug": { 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "debug", 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.767 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Enable/disable debug options", 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "jwt_token_ttl": { 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "jwt_token_ttl", 2024-07-29T00:47:34.768 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "28800", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.769 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: "key_file": { 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "key_file", 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.770 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.772 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.772 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.772 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.772 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.772 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.772 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.773 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.775 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.776 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.777 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.778 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.778 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.778 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.778 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.778 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.778 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.779 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: "motd": { 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "motd", 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.781 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "The message of the day", 2024-07-29T00:47:34.782 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: "redirect_resolve_ip_addr": { 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "redirect_resolve_ip_addr", 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.783 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.784 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.785 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "::", 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8080", 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.787 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl": { 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl", 2024-07-29T00:47:34.788 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.789 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl_server_port": { 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl_server_port", 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.790 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.791 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8443", 2024-07-29T00:47:34.791 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_behaviour": { 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_behaviour", 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "redirect", 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.795 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "redirect" 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.796 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_error_status_code": { 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_error_status_code", 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "500", 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "400", 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "599", 2024-07-29T00:47:34.797 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "url_prefix": { 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "url_prefix", 2024-07-29T00:47:34.798 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.799 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "devicehealth", 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.800 INFO:teuthology.orchestra.run.smithi175.stdout: "enable_monitoring": { 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enable_monitoring", 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.801 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "monitor device health metrics", 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.802 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.803 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.804 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.805 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.806 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.806 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.806 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.806 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.806 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.806 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.807 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.808 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.809 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.809 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.809 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.809 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.809 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.809 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.810 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.810 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.810 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.810 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.810 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.811 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "mark_out_threshold": { 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mark_out_threshold", 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.812 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "2419200", 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.813 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.814 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pool_name": { 2024-07-29T00:47:34.814 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pool_name", 2024-07-29T00:47:34.814 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.814 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "device_health_metrics", 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.815 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "retention_period": { 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "retention_period", 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15552000", 2024-07-29T00:47:34.816 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long to retain device health metrics", 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "scrape_frequency": { 2024-07-29T00:47:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "scrape_frequency", 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "86400", 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "self_heal": { 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "self_heal", 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.819 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.821 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how frequently to wake up and check device health", 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.822 INFO:teuthology.orchestra.run.smithi175.stdout: "warn_threshold": { 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "warn_threshold", 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "7257600", 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.823 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.824 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "diskprediction_local", 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.825 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.826 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.828 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.829 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.830 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.831 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.832 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.833 INFO:teuthology.orchestra.run.smithi175.stdout: "predict_interval": { 2024-07-29T00:47:34.834 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "predict_interval", 2024-07-29T00:47:34.834 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.834 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.834 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.834 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "86400", 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "predictor_model": { 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "predictor_model", 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "prophetstor", 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.836 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.837 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.838 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.838 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.838 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "600", 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.839 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "influx", 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": false, 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "influxdb python module not found", 2024-07-29T00:47:34.840 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "batch_size": { 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "batch_size", 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5000", 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.841 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "database": { 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "database", 2024-07-29T00:47:34.842 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "ceph", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.843 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": { 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "hostname", 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.844 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB server hostname", 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.845 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "30", 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "5", 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.846 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.847 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.848 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.849 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.849 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.849 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.849 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.850 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.850 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.850 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.850 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.851 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.851 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.851 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.852 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.852 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.852 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.852 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.853 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.853 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.853 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.853 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.853 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.854 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.854 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.854 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.855 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.855 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.855 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.855 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.856 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.856 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.856 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.856 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.857 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.857 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.857 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.857 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.857 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.858 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.859 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.860 INFO:teuthology.orchestra.run.smithi175.stdout: "password": { 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "password", 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.861 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "password of InfluxDB server user", 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "port": { 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "port", 2024-07-29T00:47:34.862 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "8086", 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.863 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "InfluxDB server port", 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "ssl": { 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ssl", 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.864 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.865 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.865 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "false", 2024-07-29T00:47:34.865 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.865 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.865 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.865 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: "threads": { 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "threads", 2024-07-29T00:47:34.866 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "32", 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-29T00:47:34.867 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "username": { 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "username", 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.868 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "username of InfluxDB server user", 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "verify_ssl": { 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "verify_ssl", 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "true", 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.870 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "insights", 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.872 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.874 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.875 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.876 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.877 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.878 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.878 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.878 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.878 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.878 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.878 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.880 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "iostat", 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.882 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.883 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.883 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.883 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.883 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.883 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.883 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.884 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.885 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.886 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.887 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.887 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.887 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.887 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.888 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.889 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.890 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.891 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "k8sevents", 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.892 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.893 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_event_retention_days": { 2024-07-29T00:47:34.893 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "ceph_event_retention_days", 2024-07-29T00:47:34.893 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.893 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.893 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.893 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "7", 2024-07-29T00:47:34.894 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.894 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.894 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.894 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "config_check_secs": { 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "config_check_secs", 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.895 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "10", 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.897 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.899 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.901 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.902 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.903 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.904 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "localpool", 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.905 INFO:teuthology.orchestra.run.smithi175.stdout: "failure_domain": { 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "failure_domain", 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "host", 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "failure domain for any created local pool", 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "min_size": { 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "min_size", 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default min_size for any created local pool", 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "num_rep": { 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "num_rep", 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3", 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default replica count for any created local pool", 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_num": { 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pg_num", 2024-07-29T00:47:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "128", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "default pg_num for any created local pool", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "prefix": { 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prefix", 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "name prefix for any created local pool", 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "subtree": { 2024-07-29T00:47:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "subtree", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "rack", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-29T00:47:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mds_autoscaler", 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mirroring", 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.938 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.939 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "nfs", 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "orchestrator", 2024-07-29T00:47:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "fail_fs": { 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "fail_fs", 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator": { 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "orchestrator", 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "cephadm", 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "rook", 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "test_orchestrator" 2024-07-29T00:47:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Orchestrator backend", 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "osd_perf_query", 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "osd_support", 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "pg_autoscaler", 2024-07-29T00:47:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "threshold": { 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "threshold", 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "3.0", 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1.0", 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "scaling threshold", 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "progress", 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_pg_recovery_event": { 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_pg_recovery_event", 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "enabled": { 2024-07-29T00:47:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enabled", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "max_completed_events": { 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_completed_events", 2024-07-29T00:47:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "50", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "number of past completed events to remember", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "sleep_interval": { 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "sleep_interval", 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "5", 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "how long the module is going to sleep", 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "prometheus", 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "cache": { 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "cache", 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "exclude_perf_counters": { 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "exclude_perf_counters", 2024-07-29T00:47:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-29T00:47:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.022 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.023 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.024 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.025 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.026 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.026 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.026 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.026 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.026 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.026 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.027 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.028 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_stats_pools": { 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_stats_pools", 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.029 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-29T00:47:35.030 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "300", 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.031 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "scrape_interval": { 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "scrape_interval", 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15.0", 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.033 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.034 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "::", 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.035 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "9283", 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.036 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "stale_cache_strategy": { 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "stale_cache_strategy", 2024-07-29T00:47:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "log", 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.038 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_behaviour": { 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_behaviour", 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.039 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "default", 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: "default", 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: "error" 2024-07-29T00:47:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "standby_error_status_code": { 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "standby_error_status_code", 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.041 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "500", 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "400", 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "599", 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_support", 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.043 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.044 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.045 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.046 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.047 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.048 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.049 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.050 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "max_concurrent_snap_create": { 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_concurrent_snap_create", 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.052 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10", 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.053 INFO:teuthology.orchestra.run.smithi175.stdout: "mirror_snapshot_schedule": { 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "mirror_snapshot_schedule", 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.054 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.055 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.055 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.055 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.055 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.056 INFO:teuthology.orchestra.run.smithi175.stdout: "trash_purge_schedule": { 2024-07-29T00:47:35.056 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "trash_purge_schedule", 2024-07-29T00:47:35.056 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.057 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.057 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.057 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.057 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.058 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.058 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.058 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.058 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.058 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.059 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.059 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.059 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.060 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "restful", 2024-07-29T00:47:35.060 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.060 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.060 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.061 INFO:teuthology.orchestra.run.smithi175.stdout: "enable_auth": { 2024-07-29T00:47:35.061 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enable_auth", 2024-07-29T00:47:35.061 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.061 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.062 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.062 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.062 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "key_file": { 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "key_file", 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.064 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.065 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.066 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.066 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.066 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.066 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.066 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.066 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.068 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.069 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.070 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.071 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.072 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.073 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.074 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "server_addr": { 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_addr", 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.075 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.076 INFO:teuthology.orchestra.run.smithi175.stdout: "server_port": { 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "server_port", 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.077 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rgw", 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.079 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.080 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.081 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.082 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.083 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.084 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.085 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.087 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rook", 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.088 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.089 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.090 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.091 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.093 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.094 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.095 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.096 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "storage_class": { 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "storage_class", 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.097 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "local", 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.098 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "selftest", 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.099 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.101 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.103 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.105 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.106 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.107 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "roption1": { 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "roption1", 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.108 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.109 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "roption2": { 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "roption2", 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "xyz", 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.110 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.111 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.111 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.111 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.111 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.111 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption1": { 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption1", 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.112 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.113 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption2": { 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption2", 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.114 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption3": { 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption3", 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "float", 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption4": { 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption4", 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.117 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.118 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption5": { 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption5", 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.119 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption6": { 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption6", 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.120 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.121 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.121 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.121 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.121 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.121 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.121 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.122 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.122 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.122 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "rwoption7": { 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rwoption7", 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "1", 2024-07-29T00:47:35.123 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "42", 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "testkey": { 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testkey", 2024-07-29T00:47:35.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.125 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: "testlkey": { 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testlkey", 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.126 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "testnewline": { 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "testnewline", 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.128 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snap_schedule", 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "allow_m_granularity": { 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "allow_m_granularity", 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "allow minute scheduled snapshots", 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "dump_on_update": { 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "dump_on_update", 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.132 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "dump database to debug log on update", 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.133 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.134 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.138 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.139 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.140 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.140 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.140 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "stats", 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.143 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.144 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.146 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.147 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.148 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.149 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.150 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.151 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "status", 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.152 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "telegraf", 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "address": { 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "address", 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "15", 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.168 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.169 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.172 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.173 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.174 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "telemetry", 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_basic": { 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_basic", 2024-07-29T00:47:35.176 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-29T00:47:35.177 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_crash": { 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_crash", 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.178 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.179 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.179 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.179 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.179 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.179 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-29T00:47:35.179 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_device": { 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_device", 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.180 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.181 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_ident": { 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_ident", 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.182 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "channel_perf": { 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "channel_perf", 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.184 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "contact": { 2024-07-29T00:47:35.185 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "contact", 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.186 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "description": { 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "description", 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.187 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.188 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "device_url": { 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "device_url", 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.189 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "enabled": { 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "enabled", 2024-07-29T00:47:35.190 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "24", 2024-07-29T00:47:35.192 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "8", 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.193 INFO:teuthology.orchestra.run.smithi175.stdout: "last_opt_revision": { 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "last_opt_revision", 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "1", 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.194 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "leaderboard": { 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "leaderboard", 2024-07-29T00:47:35.195 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.196 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.197 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.197 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.197 INFO:teuthology.orchestra.run.smithi175.stdout: "leaderboard_description": { 2024-07-29T00:47:35.197 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "leaderboard_description", 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.198 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.200 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.203 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.204 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.205 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.206 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.206 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.206 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.206 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.206 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.207 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "organization": { 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "organization", 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.209 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "proxy": { 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "proxy", 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.210 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.212 INFO:teuthology.orchestra.run.smithi175.stdout: "url": { 2024-07-29T00:47:35.212 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "url", 2024-07-29T00:47:35.212 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.212 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.212 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.212 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.213 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "test_orchestrator", 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.214 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.215 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.216 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.217 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.218 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.219 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.220 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.221 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.222 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.222 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.222 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.222 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.222 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.223 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "volumes", 2024-07-29T00:47:35.224 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.225 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.226 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.228 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.229 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.230 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.232 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.233 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "max_concurrent_clones": { 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "max_concurrent_clones", 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.234 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "4", 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "periodic_async_work": { 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "periodic_async_work", 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Periodically check for async work", 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "snapshot_clone_delay": { 2024-07-29T00:47:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snapshot_clone_delay", 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "0", 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: "snapshot_clone_no_wait": { 2024-07-29T00:47:35.239 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snapshot_clone_no_wait", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "True", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-29T00:47:35.240 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.241 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix", 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "can_run": true, 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "error_string": "", 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "module_options": { 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "discovery_interval": { 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "discovery_interval", 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "uint", 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "100", 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.243 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.244 INFO:teuthology.orchestra.run.smithi175.stdout: "identifier": { 2024-07-29T00:47:35.244 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "identifier", 2024-07-29T00:47:35.244 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.244 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.244 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "interval": { 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "interval", 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "secs", 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.246 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "60", 2024-07-29T00:47:35.247 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.247 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.247 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.247 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.247 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: "log_level": { 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_level", 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.248 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.249 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.250 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster": { 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster", 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.251 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_cluster_level": { 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_cluster_level", 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "info", 2024-07-29T00:47:35.253 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [ 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "", 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "critical", 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "debug", 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "error", 2024-07-29T00:47:35.254 INFO:teuthology.orchestra.run.smithi175.stdout: "info", 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: "warning" 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.255 INFO:teuthology.orchestra.run.smithi175.stdout: "log_to_file": { 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "log_to_file", 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "bool", 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 1, 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "False", 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.256 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_host": { 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_host", 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.257 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "", 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.258 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.259 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.259 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.259 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_port": { 2024-07-29T00:47:35.259 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_port", 2024-07-29T00:47:35.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "int", 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "10051", 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "zabbix_sender": { 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "zabbix_sender", 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "str", 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "level": "advanced", 2024-07-29T00:47:35.261 INFO:teuthology.orchestra.run.smithi175.stdout: "flags": 0, 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "min": "", 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "max": "", 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "enum_allowed": [], 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "desc": "", 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "long_desc": "", 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "tags": [], 2024-07-29T00:47:35.262 INFO:teuthology.orchestra.run.smithi175.stdout: "see_also": [] 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: "services": { 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: "dashboard": "https://172.21.15.175:8443/" 2024-07-29T00:47:35.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.264 INFO:teuthology.orchestra.run.smithi175.stdout: "always_on_modules": { 2024-07-29T00:47:35.264 INFO:teuthology.orchestra.run.smithi175.stdout: "octopus": [ 2024-07-29T00:47:35.264 INFO:teuthology.orchestra.run.smithi175.stdout: "balancer", 2024-07-29T00:47:35.264 INFO:teuthology.orchestra.run.smithi175.stdout: "crash", 2024-07-29T00:47:35.265 INFO:teuthology.orchestra.run.smithi175.stdout: "devicehealth", 2024-07-29T00:47:35.265 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator", 2024-07-29T00:47:35.265 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_autoscaler", 2024-07-29T00:47:35.265 INFO:teuthology.orchestra.run.smithi175.stdout: "progress", 2024-07-29T00:47:35.266 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_support", 2024-07-29T00:47:35.266 INFO:teuthology.orchestra.run.smithi175.stdout: "status", 2024-07-29T00:47:35.266 INFO:teuthology.orchestra.run.smithi175.stdout: "telemetry", 2024-07-29T00:47:35.266 INFO:teuthology.orchestra.run.smithi175.stdout: "volumes" 2024-07-29T00:47:35.267 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.267 INFO:teuthology.orchestra.run.smithi175.stdout: "pacific": [ 2024-07-29T00:47:35.267 INFO:teuthology.orchestra.run.smithi175.stdout: "balancer", 2024-07-29T00:47:35.267 INFO:teuthology.orchestra.run.smithi175.stdout: "crash", 2024-07-29T00:47:35.267 INFO:teuthology.orchestra.run.smithi175.stdout: "devicehealth", 2024-07-29T00:47:35.268 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator", 2024-07-29T00:47:35.268 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_autoscaler", 2024-07-29T00:47:35.268 INFO:teuthology.orchestra.run.smithi175.stdout: "progress", 2024-07-29T00:47:35.268 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_support", 2024-07-29T00:47:35.268 INFO:teuthology.orchestra.run.smithi175.stdout: "status", 2024-07-29T00:47:35.269 INFO:teuthology.orchestra.run.smithi175.stdout: "telemetry", 2024-07-29T00:47:35.269 INFO:teuthology.orchestra.run.smithi175.stdout: "volumes" 2024-07-29T00:47:35.269 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.269 INFO:teuthology.orchestra.run.smithi175.stdout: "quincy": [ 2024-07-29T00:47:35.270 INFO:teuthology.orchestra.run.smithi175.stdout: "balancer", 2024-07-29T00:47:35.270 INFO:teuthology.orchestra.run.smithi175.stdout: "crash", 2024-07-29T00:47:35.270 INFO:teuthology.orchestra.run.smithi175.stdout: "devicehealth", 2024-07-29T00:47:35.271 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator", 2024-07-29T00:47:35.271 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_autoscaler", 2024-07-29T00:47:35.271 INFO:teuthology.orchestra.run.smithi175.stdout: "progress", 2024-07-29T00:47:35.271 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_support", 2024-07-29T00:47:35.272 INFO:teuthology.orchestra.run.smithi175.stdout: "status", 2024-07-29T00:47:35.272 INFO:teuthology.orchestra.run.smithi175.stdout: "telemetry", 2024-07-29T00:47:35.272 INFO:teuthology.orchestra.run.smithi175.stdout: "volumes" 2024-07-29T00:47:35.272 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.272 INFO:teuthology.orchestra.run.smithi175.stdout: "reef": [ 2024-07-29T00:47:35.272 INFO:teuthology.orchestra.run.smithi175.stdout: "balancer", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "crash", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "devicehealth", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_autoscaler", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "progress", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_support", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "status", 2024-07-29T00:47:35.273 INFO:teuthology.orchestra.run.smithi175.stdout: "telemetry", 2024-07-29T00:47:35.274 INFO:teuthology.orchestra.run.smithi175.stdout: "volumes" 2024-07-29T00:47:35.274 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-07-29T00:47:35.274 INFO:teuthology.orchestra.run.smithi175.stdout: "squid": [ 2024-07-29T00:47:35.274 INFO:teuthology.orchestra.run.smithi175.stdout: "balancer", 2024-07-29T00:47:35.274 INFO:teuthology.orchestra.run.smithi175.stdout: "crash", 2024-07-29T00:47:35.274 INFO:teuthology.orchestra.run.smithi175.stdout: "devicehealth", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "orchestrator", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "pg_autoscaler", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "progress", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "rbd_support", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "status", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "telemetry", 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: "volumes" 2024-07-29T00:47:35.275 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: "last_failure_osd_epoch": 92, 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: "active_clients": [ 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "devicehealth", 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.276 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:0", 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 1890115345 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "libcephsqlite", 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:35.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:0", 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 761780209 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "rbd_support", 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:0", 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 942948944 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.280 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snap_schedule", 2024-07-29T00:47:35.280 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:35.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:35.280 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:0", 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3609281376 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "snap_schedule", 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.282 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:35.282 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:0", 2024-07-29T00:47:35.282 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3283118659 2024-07-29T00:47:35.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.282 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: "name": "volumes", 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: "addrvec": [ 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v2", 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "172.21.15.175:0", 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 3574021858 2024-07-29T00:47:35.283 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:35.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-29T00:47:35.284 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:47:35.286 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/259376681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-29T00:47:35.286 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:32 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/640360805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-29T00:47:35.286 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:33 smithi181 ceph-mon[36706]: pgmap v344: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:35.286 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:33 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1512658408' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-29T00:47:35.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/259376681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-29T00:47:35.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:32 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/640360805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-29T00:47:35.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:33 smithi204 ceph-mon[37368]: pgmap v344: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:35.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:33 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1512658408' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff7980a2d00 msgr2=0x7ff7980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff7980a2d00 0x7ff7980a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4000c00 tx=0x7ff7a4005b50 comp rx=0 tx=0).stop 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 msgr2=0x7ff7b019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 0x7ff7b019a630 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00eeb0 comp rx=0 tx=0).stop 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 shutdown_connections 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7ff7980a2d00 0x7ff7980a51c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b010dea0 0x7ff7b019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:35.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7b0074bb0 0x7ff7b019a630 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:35.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 --2- 172.21.15.175:0/1512658408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b0074290 0x7ff7b019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:35.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 >> 172.21.15.175:0/1512658408 conn(0x7ff7b00fd300 msgr2=0x7ff7b00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:35.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 shutdown_connections 2024-07-29T00:47:35.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:32.832+0000 7ff7b5e00640 1 -- 172.21.15.175:0/1512658408 wait complete. 2024-07-29T00:47:35.296 DEBUG:tasks.check_counter:Getting stats from a 2024-07-29T00:47:35.296 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.a perf dump 2024-07-29T00:47:35.297 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/259376681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-29T00:47:35.297 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:32 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/640360805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-29T00:47:35.297 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:33 smithi175 ceph-mon[32138]: pgmap v344: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:35.297 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:33 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1512658408' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-29T00:47:35.797 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:36.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:35 smithi204 ceph-mon[37368]: pgmap v345: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:36.188 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:35 smithi175 ceph-mon[32138]: pgmap v345: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:36.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:35 smithi181 ceph-mon[36706]: pgmap v345: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:36.540 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:47:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 4, 2024-07-29T00:47:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 4, 2024-07-29T00:47:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3024, 2024-07-29T00:47:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 448, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.006927381, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000091613, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.003577146, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-29T00:47:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.006265671, 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002088557 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 236, 2024-07-29T00:47:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 225, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1584656, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 138304, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.087019236, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.035892332, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.075676549, 2024-07-29T00:47:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 224, 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.022938339, 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000102403 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1584656, 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 138304 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 2, 2024-07-29T00:47:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 172, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 320, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 186800, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.036241003, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.025818518, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.005194223, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-29T00:47:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 169, 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.028117928, 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000166378 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 186800 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-29T00:47:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 148977, 2024-07-29T00:47:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 30, 2024-07-29T00:47:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-29T00:47:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-29T00:47:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2024-07-29T00:47:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2024-07-29T00:47:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-29T00:47:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 928, 2024-07-29T00:47:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2024-07-29T00:47:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-29T00:47:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-29T00:47:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-29T00:47:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 238, 2024-07-29T00:47:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1551392, 2024-07-29T00:47:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 238, 2024-07-29T00:47:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 238, 2024-07-29T00:47:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1551392, 2024-07-29T00:47:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:36.566 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:47:37.477 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-07-29T00:47:37.477 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-07-29T00:47:37.478 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-07-29T00:47:37.478 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-07-29T00:47:37.478 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-07-29T00:47:37.478 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-07-29T00:47:37.478 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-07-29T00:47:37.478 DEBUG:tasks.check_counter:Getting stats from d 2024-07-29T00:47:37.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.d perf dump 2024-07-29T00:47:37.946 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:38.097 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:37 smithi175 ceph-mon[32138]: pgmap v346: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:38.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:37 smithi181 ceph-mon[36706]: pgmap v346: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:38.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:37 smithi204 ceph-mon[37368]: pgmap v346: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 0, 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2, 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 0, 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 0, 2024-07-29T00:47:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.004146257, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000000000, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.001024534, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:38.653 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:38.654 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:38.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.654 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:38.654 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:38.654 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:38.654 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 242, 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 231, 2024-07-29T00:47:38.655 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1586288, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 142144, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.089492749, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.035945870, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.079887775, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:38.656 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 230, 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.023782832, 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000103403 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1586288, 2024-07-29T00:47:38.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 142144 2024-07-29T00:47:38.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.658 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:38.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 6, 2024-07-29T00:47:38.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 179, 2024-07-29T00:47:38.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3344, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 188128, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.035495521, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.024177111, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.006556959, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 177, 2024-07-29T00:47:38.659 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.037373036, 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000211147 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-07-29T00:47:38.661 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 188128 2024-07-29T00:47:38.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.661 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-29T00:47:38.661 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2024-07-29T00:47:38.661 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:38.662 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:38.663 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:38.664 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:38.665 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 150776, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 32, 2024-07-29T00:47:38.666 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2024-07-29T00:47:38.667 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 928, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-29T00:47:38.668 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 244, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1552172, 2024-07-29T00:47:38.669 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 244, 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 244, 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1552172, 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:38.670 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:38.671 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:38.671 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:38.671 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:38.671 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:38.671 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:47:39.517 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-07-29T00:47:39.517 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-07-29T00:47:39.517 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-07-29T00:47:39.517 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-07-29T00:47:39.518 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-07-29T00:47:39.518 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-07-29T00:47:39.518 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-07-29T00:47:39.518 DEBUG:tasks.check_counter:Getting stats from g 2024-07-29T00:47:39.518 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.g perf dump 2024-07-29T00:47:39.990 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:40.140 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:39 smithi175 ceph-mon[32138]: pgmap v347: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:40.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:39 smithi181 ceph-mon[36706]: pgmap v347: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:40.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:39 smithi204 ceph-mon[37368]: pgmap v347: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:40.739 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:47:40.739 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:40.739 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 19573, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 29004, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 7226583, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 32667297, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 21, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 7.057357451, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 4.141002583, 2024-07-29T00:47:40.740 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 3.332615464, 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.159534931, 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 29000, 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.208482923, 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000076154 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:40.741 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 38043, 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.019994072, 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000525 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 6282592, 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 19901232 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.742 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 40040, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 29490, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 13303298, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 6938163, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 9, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 6.692622471, 2024-07-29T00:47:40.743 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 3.023939581, 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 4.728552603, 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.042345519, 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 29492, 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.011301209, 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000068198 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:40.744 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 67342, 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.027741379, 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000411 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1752128, 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 157632 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.745 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1982, 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1765, 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 839906, 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1707085, 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 36, 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-07-29T00:47:40.746 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.389083639, 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.167986304, 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.178657990, 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.058018162, 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1762, 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.194762998, 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000110535 2024-07-29T00:47:40.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1445, 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000641360, 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000443 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 467104 2024-07-29T00:47:40.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-29T00:47:40.749 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4496, 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 5.741352727, 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001276991 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-29T00:47:40.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3401, 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.469065937, 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000137920 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 12090, 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 10807, 2024-07-29T00:47:40.751 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 10807, 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 28.658205796, 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002651818 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 1283, 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:40.752 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 93, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 24, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 34, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 35, 2024-07-29T00:47:40.753 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 58, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 50, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 4, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 4, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 10, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 24211, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 9418, 2024-07-29T00:47:40.754 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 1280, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 33, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 89, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 2, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 34, 2024-07-29T00:47:40.755 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 3, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 77, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 73, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 67, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 21176629, 2024-07-29T00:47:40.756 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-07-29T00:47:40.757 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 73, 2024-07-29T00:47:40.757 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:40.757 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:40.757 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:40.757 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:40.757 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 73, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 1022, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 7084, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:40.758 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 2546, 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 167, 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 134, 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 10353 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-07-29T00:47:40.759 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 173, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 406, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 212, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 417, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 180, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:40.760 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 1859, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 1859, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-07-29T00:47:40.761 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 5, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 143, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 2, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:40.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 8198, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 8198, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-07-29T00:47:40.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 10, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 10, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-07-29T00:47:40.764 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 15718502, 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2134, 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 4.934465431, 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002312308 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-07-29T00:47:40.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 62, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 1956, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 1894, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 34, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 233, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 199, 2024-07-29T00:47:40.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 93, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 2931, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 2838, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 4, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 1935, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 1931, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 56988, 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223520 2024-07-29T00:47:40.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 21420, 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 10, 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 12090, 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 18, 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 3829, 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-07-29T00:47:40.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1859, 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 8.990135729, 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004836006 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 51, 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.073856406, 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001448164 2024-07-29T00:47:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-07-29T00:47:40.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2388, 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 9.581746237, 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004012456 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.771 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-07-29T00:47:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4566, 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 3.764183081, 2024-07-29T00:47:40.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000824394 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 84, 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.355300030, 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004229762 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.779 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.780 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.780 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:40.780 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.780 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.780 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.781 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-07-29T00:47:40.781 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.781 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.781 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.782 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:40.782 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.782 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.782 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-07-29T00:47:40.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.786 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1859, 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 5.892984313, 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003169975 2024-07-29T00:47:40.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.788 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 3067, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 3067, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 570, 2024-07-29T00:47:40.789 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:40.790 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:40.791 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:40.792 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:40.793 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 312916, 2024-07-29T00:47:40.794 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 148, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:40.795 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 185, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 289590, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2240, 2024-07-29T00:47:40.796 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 4674, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 11900848, 2024-07-29T00:47:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 4674, 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4674, 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 14.227782468, 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003044027 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-07-29T00:47:40.798 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4674, 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 4923 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 4674, 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 224, 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 4450, 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-07-29T00:47:40.799 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 151, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 2285, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 107, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-07-29T00:47:40.800 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 1873, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 73, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:40.801 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:40.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-07-29T00:47:40.803 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-07-29T00:47:40.804 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 102, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 15, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 12, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 305, 2024-07-29T00:47:40.805 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 99 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-07-29T00:47:40.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 148 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 6, 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-07-29T00:47:40.807 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 6, 2024-07-29T00:47:40.808 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 1859, 2024-07-29T00:47:40.808 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 1859, 2024-07-29T00:47:40.808 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:40.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.808 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:40.809 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:40.809 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-29T00:47:40.809 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:40.809 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 61591, 2024-07-29T00:47:40.809 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 15010927, 2024-07-29T00:47:40.809 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 61591, 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 61591, 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 15010927, 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:40.810 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:40.811 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 4674, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 12088607, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 4674, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 12088607, 2024-07-29T00:47:40.812 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-07-29T00:47:40.813 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 4674, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 4674, 2024-07-29T00:47:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 4674, 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 4674, 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 1859, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 187759, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 1859, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 151, 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 187759, 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x56455dbe4960": { 2024-07-29T00:47:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 8198, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 11524198, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 8198, 2024-07-29T00:47:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2134, 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 11524198, 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:40.820 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:47:41.749 INFO:tasks.check_counter:Daemon mds.g mds.exported=2 2024-07-29T00:47:41.749 INFO:tasks.check_counter:Daemon mds.g mds.imported=3 2024-07-29T00:47:41.749 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=173 2024-07-29T00:47:41.749 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=406 2024-07-29T00:47:41.749 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-07-29T00:47:41.750 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:41.750 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:41.750 DEBUG:tasks.check_counter:Getting stats from j 2024-07-29T00:47:41.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.j perf dump 2024-07-29T00:47:42.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:41 smithi175 ceph-mon[32138]: pgmap v348: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:42.213 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:42.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:41 smithi181 ceph-mon[36706]: pgmap v348: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:42.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:41 smithi204 ceph-mon[37368]: pgmap v348: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:42.918 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-29T00:47:42.918 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:42.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 432, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 434, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 4307087, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 137440, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.097178842, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.026864737, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.038429864, 2024-07-29T00:47:42.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.020934437, 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 432, 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.032830166, 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000075995 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:42.920 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 774, 2024-07-29T00:47:42.921 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 951, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 4386462, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 712269, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.208872760, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.092230809, 2024-07-29T00:47:42.922 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.051783709, 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.044869239, 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 948, 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.086264983, 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000090996 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:42.923 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 466768 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.924 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1581, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1552, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 5206262, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 559790, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.387310244, 2024-07-29T00:47:42.925 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.150219529, 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.173242395, 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.070466109, 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1552, 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.220595211, 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000142136 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.926 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1722528, 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 147344 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-29T00:47:42.927 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4185, 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.213132472, 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000050927 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.929 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.930 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 1308, 2024-07-29T00:47:42.931 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 6, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 1302, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 1300, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-07-29T00:47:42.932 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 5, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-07-29T00:47:42.933 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:42.934 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:42.935 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-07-29T00:47:42.936 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-07-29T00:47:42.937 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:42.938 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-07-29T00:47:42.939 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-07-29T00:47:42.940 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 8324, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 11, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-07-29T00:47:42.941 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 15669418, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 15668844, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 8324 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 88, 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 1293, 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 1205, 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 23, 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 23, 2024-07-29T00:47:42.943 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 0, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 1308, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 1308, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 51084, 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215328 2024-07-29T00:47:42.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.945 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.947 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:42.948 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.949 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-07-29T00:47:42.951 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.952 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.952 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.952 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-07-29T00:47:42.952 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-07-29T00:47:42.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.956 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.958 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.959 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.960 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.961 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.964 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 4, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 3, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 3, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 4437, 2024-07-29T00:47:42.965 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:42.966 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:42.967 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:42.968 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:42.969 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 203788, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 133, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:42.970 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:42.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 185, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 1172526, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 7106, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:42.972 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 2501, 2024-07-29T00:47:42.973 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 2501, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2501, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.421962961, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000968397 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-07-29T00:47:42.974 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2501, 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2502 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 2501, 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 2501, 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-07-29T00:47:42.975 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 1518, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 981, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-07-29T00:47:42.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:42.977 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:42.978 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-07-29T00:47:42.979 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-07-29T00:47:42.980 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 102, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 15, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 6, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 6, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-07-29T00:47:42.981 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-07-29T00:47:42.982 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-07-29T00:47:42.983 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-29T00:47:42.984 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 2783, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 13632065, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 2783, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2783, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 13632065, 2024-07-29T00:47:42.985 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:42.986 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 2501, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 11475119, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2501, 2024-07-29T00:47:42.987 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 11475119, 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:42.988 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 2501, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 2501, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2501, 2024-07-29T00:47:42.989 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2501, 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:42.990 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-29T00:47:42.991 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x56055d8d8be0": { 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-29T00:47:42.992 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-29T00:47:42.993 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-29T00:47:42.994 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-29T00:47:43.973 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-07-29T00:47:43.973 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-07-29T00:47:43.973 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=0 2024-07-29T00:47:43.974 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=0 2024-07-29T00:47:43.974 INFO:tasks.check_counter:Daemon mds.j mds.root_rsnaps=0 2024-07-29T00:47:43.974 INFO:tasks.check_counter:Daemon mds.j mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:43.974 INFO:tasks.check_counter:Daemon mds.j mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:43.974 DEBUG:tasks.check_counter:Getting stats from b 2024-07-29T00:47:43.974 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.b perf dump 2024-07-29T00:47:44.238 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:47:44.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:43 smithi181 ceph-mon[36706]: pgmap v349: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:44.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:43 smithi175 ceph-mon[32138]: pgmap v349: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:44.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:43 smithi204 ceph-mon[37368]: pgmap v349: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:44.752 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-07-29T00:47:44.752 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:44.752 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 29403, 2024-07-29T00:47:44.752 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 41551, 2024-07-29T00:47:44.752 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 8349298, 2024-07-29T00:47:44.752 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 29286654, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 16, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 2.146090051, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 1.025781284, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 1.176580697, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.083912624, 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 41549, 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.977913015, 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000023536 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 63416, 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.009374180, 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000147 2024-07-29T00:47:44.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1858384, 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 530880 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 63427, 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 86079, 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 20048656, 2024-07-29T00:47:44.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 79671999, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 19, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 4.131997746, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 1.841939011, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 2.666034225, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.209588696, 2024-07-29T00:47:44.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 86078, 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.639615766, 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000019048 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 122783, 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.018413018, 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000149 2024-07-29T00:47:44.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 14162160, 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 38760416 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 59640, 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 81359, 2024-07-29T00:47:44.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 13552596, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 42232599, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 131, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 3.358711821, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 1.389999857, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 2.252445699, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.099414412, 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:44.759 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 81357, 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.423385527, 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000017495 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 131540, 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.019270440, 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000146 2024-07-29T00:47:44.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 1136 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.761 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 20540, 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 9.620818968, 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000468394 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3877, 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.180429136, 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000046538 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.763 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 22932, 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 20468, 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 20468, 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 72.924947283, 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003562876 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-07-29T00:47:44.764 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 2464, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 21, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 1, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 114, 2024-07-29T00:47:44.765 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 30, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 55, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 29, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 72, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 276, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 17, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 17, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 17, 2024-07-29T00:47:44.766 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 34824, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 25194, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 2460, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 105, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-07-29T00:47:44.767 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 12, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 207, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 576, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:44.768 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 67, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 21176629, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 6, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 576, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 246, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 13, 2024-07-29T00:47:44.769 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 233, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 330, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 14, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 6643, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 8154, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:44.770 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 11888, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 1977, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 863, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 9825 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 55, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-07-29T00:47:44.771 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 921, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 1919, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 1919, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-07-29T00:47:44.772 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 5, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 238, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 13, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 12, 2024-07-29T00:47:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 1, 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-07-29T00:47:44.774 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 16494, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 16494, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 24, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-07-29T00:47:44.775 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 24, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 3, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 66066646, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10109, 2024-07-29T00:47:44.776 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 24.461453139, 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002419769 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 102, 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 2205, 2024-07-29T00:47:44.777 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2103, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 46, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 53, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 7, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 114, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 4546, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 4432, 2024-07-29T00:47:44.778 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 17, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 2162, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 2145, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 68936, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 231712 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 32832, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 10679, 2024-07-29T00:47:44.779 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 22932, 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 308, 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 21354, 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2001, 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.346351501, 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001172589 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.780 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 6804, 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 43.466752629, 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006388411 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.781 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-07-29T00:47:44.782 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3550, 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 16.401933882, 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004620263 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000242612, 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000242612 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.784 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.786 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:44.786 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.786 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.786 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 11, 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001986002, 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000180545 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.787 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 20, 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.022756095, 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001137804 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 11, 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.091266276, 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.008296934 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-07-29T00:47:44.789 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000385990, 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000385990 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5766, 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 7.586370030, 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001315707 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.790 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 327, 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.414656006, 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001268061 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.792 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5, 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.033203456, 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006640691 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:44.793 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000793745, 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000793745 2024-07-29T00:47:44.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 44, 2024-07-29T00:47:44.795 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.010335620, 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000234900 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-07-29T00:47:44.796 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 7, 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.019120402, 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002731486 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1919, 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.528793037, 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001317766 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:44.798 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-07-29T00:47:44.799 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 5, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 3, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 324, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 162, 2024-07-29T00:47:44.800 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 657, 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:44.801 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:44.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:44.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:44.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:44.805 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 328276, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 152, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:44.806 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 185, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-07-29T00:47:44.807 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 535422, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 5007, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-07-29T00:47:44.808 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 13711, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 62424861, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 37, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 13711, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 13711, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 52.262953990, 2024-07-29T00:47:44.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003811753 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 13711, 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 13954 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 13711, 2024-07-29T00:47:44.810 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 718, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 12993, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 11, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 129, 2024-07-29T00:47:44.811 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 10225, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 107, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 1, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 2445, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-07-29T00:47:44.812 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 589, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 22, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-07-29T00:47:44.813 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-07-29T00:47:44.814 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-07-29T00:47:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-07-29T00:47:44.816 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-07-29T00:47:44.816 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-07-29T00:47:44.816 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-07-29T00:47:44.816 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 102, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 15, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-07-29T00:47:44.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 346, 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 79 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-07-29T00:47:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 243 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.819 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 6, 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 6, 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 1919, 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 1919, 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:44.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 152466, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 26488860, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:44.821 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 152466, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 152466, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 26488860, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.822 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:44.823 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 13711, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 62618816, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 13711, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 62618816, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.824 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:44.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 13711, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 13711, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 13711, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 13711, 2024-07-29T00:47:44.826 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:44.827 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1919, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 193955, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1919, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 116, 2024-07-29T00:47:44.828 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 193955, 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55fcffd24960": { 2024-07-29T00:47:44.829 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 16494, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 61872342, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 16494, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:44.830 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 10109, 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 61872342, 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.831 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:44.832 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-07-29T00:47:45.573 INFO:tasks.check_counter:Daemon mds.b mds.exported=12 2024-07-29T00:47:45.574 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-07-29T00:47:45.574 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=55 2024-07-29T00:47:45.574 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-07-29T00:47:45.574 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-07-29T00:47:45.574 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-07-29T00:47:45.574 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-07-29T00:47:45.574 DEBUG:tasks.check_counter:Getting stats from e 2024-07-29T00:47:45.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.e perf dump 2024-07-29T00:47:45.832 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:47:46.116 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:45 smithi181 ceph-mon[36706]: pgmap v350: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:46.123 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:45 smithi175 ceph-mon[32138]: pgmap v350: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:46.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:45 smithi204 ceph-mon[37368]: pgmap v350: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:46.425 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-07-29T00:47:46.425 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:46.425 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1094, 2024-07-29T00:47:46.425 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1065, 2024-07-29T00:47:46.425 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1994752, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 404602, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.130771827, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.051842628, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.063788843, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.021134572, 2024-07-29T00:47:46.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1065, 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.062622078, 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000058800 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.427 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1722256, 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 146704 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 364, 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 540, 2024-07-29T00:47:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 89436, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 583386, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.043039286, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.015180553, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.009771505, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.013616088, 2024-07-29T00:47:46.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 537, 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.033424577, 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000062243 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 468208 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1276, 2024-07-29T00:47:46.431 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1278, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 11854880, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 405775, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.095279419, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.022073489, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.034483588, 2024-07-29T00:47:46.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.032098822, 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1276, 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.033304331, 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000026100 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-07-29T00:47:46.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4144, 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.093620800, 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000022591 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.435 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-07-29T00:47:46.436 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-07-29T00:47:46.437 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 1947, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 24, 2024-07-29T00:47:46.438 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 1923, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 1913, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 3, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-07-29T00:47:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-07-29T00:47:46.440 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-07-29T00:47:46.441 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-07-29T00:47:46.442 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-07-29T00:47:46.443 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-07-29T00:47:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:46.445 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-07-29T00:47:46.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-07-29T00:47:46.447 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 8198, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-07-29T00:47:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 10, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 15718502, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 15717928, 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-07-29T00:47:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 8198 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-07-29T00:47:46.450 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 40, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 1925, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 1885, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 21, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 23, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 2, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 1947, 2024-07-29T00:47:46.451 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 1982, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 35, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 52588, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 215328 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-07-29T00:47:46.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.461 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-07-29T00:47:46.462 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.463 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.465 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.466 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.467 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.468 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.469 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-07-29T00:47:46.470 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 3, 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 2, 2024-07-29T00:47:46.472 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2178, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:46.473 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:46.474 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:46.475 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 191848, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 79, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-07-29T00:47:46.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 185, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 1509278, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 8523, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:46.480 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 2449, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 2449, 2024-07-29T00:47:46.481 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2449, 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.395972093, 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000570017 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-07-29T00:47:46.482 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2449, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2450 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 2449, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 2449, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-07-29T00:47:46.483 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 1485, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 962, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-07-29T00:47:46.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-07-29T00:47:46.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-07-29T00:47:46.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-07-29T00:47:46.487 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-07-29T00:47:46.488 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 102, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 15, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 5, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 5, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-07-29T00:47:46.489 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:46.490 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-07-29T00:47:46.491 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 0, 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:46.492 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2730, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 13676144, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2730, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2730, 2024-07-29T00:47:46.493 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 13676144, 2024-07-29T00:47:46.494 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:46.494 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.494 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.494 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.494 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:46.495 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2449, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 11527016, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2449, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 11527016, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.496 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:46.497 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2449, 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2449, 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2449, 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2449, 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:46.498 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:46.499 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-07-29T00:47:46.500 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x56086f8feb40": { 2024-07-29T00:47:46.501 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:46.504 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-07-29T00:47:47.133 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-07-29T00:47:47.133 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-07-29T00:47:47.133 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-07-29T00:47:47.133 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-07-29T00:47:47.133 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-07-29T00:47:47.134 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:47.134 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:47.134 DEBUG:tasks.check_counter:Getting stats from h 2024-07-29T00:47:47.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.h perf dump 2024-07-29T00:47:47.385 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:47:47.930 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 39781, 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 27555, 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 11557958, 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 10627781, 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 15, 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-07-29T00:47:47.931 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 1.966024074, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.870445741, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 1.423884201, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.036294121, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 27556, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.417218642, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000015140 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.932 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 63869, 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.008076000, 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000126 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2208, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2389, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 646371, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 5751620, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 20, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.165906850, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.055977407, 2024-07-29T00:47:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.052957638, 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.048968425, 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2385, 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.076574154, 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000032106 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 640, 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000100132, 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000156 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1408, 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 547952 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 15847, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 21173, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 7699821, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 18594880, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 33, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 8, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 1.092120650, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.491738967, 2024-07-29T00:47:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.783864769, 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.029358263, 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 21170, 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.328519557, 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000015518 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 29065, 2024-07-29T00:47:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.004032169, 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000138 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 7090592, 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 14514416 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-07-29T00:47:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 6076, 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.161825638, 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000355797 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2571, 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.100980789, 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000039276 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 8510, 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 7245, 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 7245, 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 81.854791063, 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.011298107 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 1265, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 115, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 14, 2024-07-29T00:47:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 101, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 96, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 4, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 4, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 8, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 19007, 2024-07-29T00:47:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 9329, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 1264, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 27, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 391, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-07-29T00:47:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 3, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 32, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 122, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 67, 2024-07-29T00:47:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 21176629, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 6, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 122, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 122, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 488, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 5138, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 4196, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 1808, 2024-07-29T00:47:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 261, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 7249 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 217, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 425, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 219, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 440, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 228, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-07-29T00:47:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 1205, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 1205, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 5, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 135, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:47.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 8324, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 8324, 2024-07-29T00:47:47.952 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 11, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 11, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-07-29T00:47:47.953 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 15669418, 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2928, 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 8.909005750, 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003042693 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.954 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 96, 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 1301, 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 1205, 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 32, 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 246, 2024-07-29T00:47:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 214, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 115, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 1457, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 1342, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 4, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 1306, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 1302, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 58072, 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 223520 2024-07-29T00:47:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 15619, 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 8510, 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 500, 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 2795, 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-07-29T00:47:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1199, 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 25.550133359, 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.021309535 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 45, 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.035653971, 2024-07-29T00:47:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000792310 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1760, 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.596616683, 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000907168 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000122021, 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000122021 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-07-29T00:47:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-07-29T00:47:47.965 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2471, 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.926110366, 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000374791 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 391, 2024-07-29T00:47:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 32.657475611, 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.083522955 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 19.242728854, 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 19.242728854 2024-07-29T00:47:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 172, 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.058475313, 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000339972 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1205, 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.787474885, 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001483381 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-07-29T00:47:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 4, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 3, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 2355, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 2355, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 640, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.977 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:47.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:47.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:47.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 289341, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 145, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:47.982 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:47.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 185, 2024-07-29T00:47:47.984 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:47.984 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:47.984 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:47.984 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-07-29T00:47:47.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 363219, 2024-07-29T00:47:47.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2509, 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-07-29T00:47:47.985 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 4859, 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 11759408, 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 4859, 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4859, 2024-07-29T00:47:47.986 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 15.127696471, 2024-07-29T00:47:47.987 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003113335 2024-07-29T00:47:47.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.987 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-07-29T00:47:47.987 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-07-29T00:47:47.987 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4859, 2024-07-29T00:47:47.987 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5123 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 4859, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 265, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 4594, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-07-29T00:47:47.988 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 143, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 3071, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 106, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-07-29T00:47:47.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 1219, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 122, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:47.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-07-29T00:47:47.991 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-07-29T00:47:47.992 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-07-29T00:47:47.993 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 102, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 15, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-07-29T00:47:47.994 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 328, 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 104 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-07-29T00:47:47.995 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 143 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 6, 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-07-29T00:47:47.996 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 6, 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 1205, 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 1205, 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:47.997 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 57832, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 14023828, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 57832, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:47.998 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 57832, 2024-07-29T00:47:48.001 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 14023828, 2024-07-29T00:47:48.001 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:48.001 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 4859, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 11881249, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4859, 2024-07-29T00:47:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 11881249, 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 4859, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 4859, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4859, 2024-07-29T00:47:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 4859, 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1205, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 121841, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1205, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 143, 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 121841, 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55727ca1a960": { 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 8324, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 11475114, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:48.009 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 8324, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2928, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 11475114, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:48.010 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:48.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:48.011 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-07-29T00:47:48.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:47 smithi175 ceph-mon[32138]: pgmap v351: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:48.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:47 smithi181 ceph-mon[36706]: pgmap v351: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:48.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:47 smithi204 ceph-mon[37368]: pgmap v351: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:48.804 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-07-29T00:47:48.804 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-07-29T00:47:48.805 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=217 2024-07-29T00:47:48.805 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=425 2024-07-29T00:47:48.805 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-07-29T00:47:48.805 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:48.805 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:48.805 DEBUG:tasks.check_counter:Getting stats from k 2024-07-29T00:47:48.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.k perf dump 2024-07-29T00:47:49.054 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.b/config 2024-07-29T00:47:49.583 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-07-29T00:47:49.583 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:49.583 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1153, 2024-07-29T00:47:49.583 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1153, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 360047, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 365831, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.112556667, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.041127182, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.027194069, 2024-07-29T00:47:49.584 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.033106718, 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1152, 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.055946427, 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000048564 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:49.585 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.586 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.586 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.586 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-07-29T00:47:49.586 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2024-07-29T00:47:49.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 751, 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 930, 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 5762909, 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 706986, 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-07-29T00:47:49.587 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:49.588 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.074513480, 2024-07-29T00:47:49.588 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.023552573, 2024-07-29T00:47:49.588 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.022718645, 2024-07-29T00:47:49.588 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.020976005, 2024-07-29T00:47:49.588 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:49.588 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 927, 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.042334745, 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000045668 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.589 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 468736 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 870, 2024-07-29T00:47:49.590 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 842, 2024-07-29T00:47:49.591 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 10260922, 2024-07-29T00:47:49.591 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 335164, 2024-07-29T00:47:49.591 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-07-29T00:47:49.591 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:49.591 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.095212813, 2024-07-29T00:47:49.591 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.029822815, 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.064555172, 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.013530957, 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 841, 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.030517444, 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000036287 2024-07-29T00:47:49.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1720320, 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 148816 2024-07-29T00:47:49.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4254, 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.093668894, 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000022019 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-07-29T00:47:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-07-29T00:47:49.596 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 1568, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 10, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 1558, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:49.598 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 1554, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 6, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-07-29T00:47:49.599 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-07-29T00:47:49.600 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-07-29T00:47:49.601 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:49.602 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-07-29T00:47:49.603 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-07-29T00:47:49.604 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-07-29T00:47:49.605 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 0, 2024-07-29T00:47:49.606 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-07-29T00:47:49.607 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 9556, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-07-29T00:47:49.608 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 11, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 18154147, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 18153573, 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-07-29T00:47:49.609 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 9556 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 104, 2024-07-29T00:47:49.610 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 1542, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 1438, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 25, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 25, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 0, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 1568, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 1568, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 0, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-07-29T00:47:49.611 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 50244, 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 215328 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:49.612 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.613 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-07-29T00:47:49.615 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.616 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.617 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.619 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.620 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-07-29T00:47:49.620 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.620 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.620 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.621 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.622 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-07-29T00:47:49.623 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.623 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.623 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.623 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-07-29T00:47:49.623 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.624 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.625 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.626 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.626 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:49.626 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.626 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.626 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.627 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.628 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.630 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:49.631 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-07-29T00:47:49.632 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 3, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 2, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2181, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.633 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:49.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:49.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:49.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:49.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:49.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-07-29T00:47:49.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:49.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:49.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:49.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:49.638 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 211185, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 153, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:49.639 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 185, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 1395181, 2024-07-29T00:47:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 8482, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-07-29T00:47:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 2484, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 2484, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2484, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.384952354, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000960125 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.642 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2484, 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2485 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 2484, 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 2484, 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-07-29T00:47:49.643 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 1490, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 992, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-07-29T00:47:49.644 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2024-07-29T00:47:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:49.646 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-07-29T00:47:49.647 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-07-29T00:47:49.648 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 102, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 15, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 7, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 7, 2024-07-29T00:47:49.649 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:49.650 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-07-29T00:47:49.651 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 0, 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2770, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 16117146, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2770, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:49.653 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2770, 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 16117146, 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.654 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2484, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 13961538, 2024-07-29T00:47:49.655 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2484, 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 13961538, 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:49.656 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2484, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2484, 2024-07-29T00:47:49.657 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2484, 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2484, 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.658 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:49.659 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-07-29T00:47:49.661 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55b0b0446b40": { 2024-07-29T00:47:49.661 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-07-29T00:47:49.661 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-07-29T00:47:49.661 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-07-29T00:47:49.661 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-07-29T00:47:49.661 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-07-29T00:47:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-07-29T00:47:49.663 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-07-29T00:47:49.663 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-07-29T00:47:49.663 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-07-29T00:47:49.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-07-29T00:47:49.663 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-07-29T00:47:50.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:49 smithi204 ceph-mon[37368]: pgmap v352: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:50.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:49 smithi181 ceph-mon[36706]: pgmap v352: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:50.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:49 smithi175 ceph-mon[32138]: pgmap v352: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:50.350 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-07-29T00:47:50.350 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-07-29T00:47:50.350 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-07-29T00:47:50.350 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-07-29T00:47:50.351 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-07-29T00:47:50.351 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:50.351 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:50.351 DEBUG:tasks.check_counter:Getting stats from c 2024-07-29T00:47:50.351 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.c perf dump 2024-07-29T00:47:50.617 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:47:51.130 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-07-29T00:47:51.131 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:51.131 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1468, 2024-07-29T00:47:51.131 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1995, 2024-07-29T00:47:51.131 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 504745, 2024-07-29T00:47:51.131 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 4614887, 2024-07-29T00:47:51.131 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 18, 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.124260223, 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.045373237, 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.045547025, 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.026355134, 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:51.132 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1993, 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.067147013, 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000033691 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1256, 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000193736, 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000154 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.133 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 334256 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 41385, 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 26372, 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 12739377, 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 11170178, 2024-07-29T00:47:51.134 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 10, 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 2.083111205, 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.839198222, 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 1.702259533, 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.008171519, 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:51.135 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 26372, 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.450582282, 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000017085 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 65333, 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.009319336, 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000142 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.136 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2156432, 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 6780992 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 997, 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 930, 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2011187, 2024-07-29T00:47:51.137 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 813613, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 36, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 11, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.100222936, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.035608635, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.075729667, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.007500208, 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:51.138 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 927, 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.043224513, 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000046628 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 644, 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000097280, 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000151 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.139 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1733920, 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 158800 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:51.140 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1430, 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.597908671, 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000418117 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 781, 2024-07-29T00:47:51.141 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.027237908, 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000034875 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 2610, 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 2528, 2024-07-29T00:47:51.142 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2528, 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 20.150929672, 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.007971095 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 82, 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:51.143 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 162, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 30, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 132, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:51.144 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 119, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 6, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 6, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 8, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 5613, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 3139, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 82, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 14, 2024-07-29T00:47:51.145 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 89, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 1000, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 1, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 4, 2024-07-29T00:47:51.146 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 79, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 162, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 67, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 21176629, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 162, 2024-07-29T00:47:51.147 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 162, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 6, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 591, 2024-07-29T00:47:51.148 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 1971, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 2616, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 207, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 116, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 2363 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.149 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 50, 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 523, 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 268, 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 512, 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 58, 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-07-29T00:47:51.150 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 295, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 295, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:51.151 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 5, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 123, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-07-29T00:47:51.152 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 2396, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 2396, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-07-29T00:47:51.153 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 4, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 4, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-07-29T00:47:51.154 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 7717031, 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 605, 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.013273392, 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003327724 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-07-29T00:47:51.155 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.156 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-07-29T00:47:51.156 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 151, 2024-07-29T00:47:51.156 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 446, 2024-07-29T00:47:51.156 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 295, 2024-07-29T00:47:51.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 32, 2024-07-29T00:47:51.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 285, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 253, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 162, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 623, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 461, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 6, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 450, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 444, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 55420, 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223520 2024-07-29T00:47:51.157 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.158 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-07-29T00:47:51.158 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 4535, 2024-07-29T00:47:51.158 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 2, 2024-07-29T00:47:51.158 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 2610, 2024-07-29T00:47:51.158 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 12, 2024-07-29T00:47:51.159 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 698, 2024-07-29T00:47:51.159 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-07-29T00:47:51.159 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 295, 2024-07-29T00:47:51.159 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 18.107171671, 2024-07-29T00:47:51.159 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.061380242 2024-07-29T00:47:51.159 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 78, 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.131767372, 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001689325 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.160 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 516, 2024-07-29T00:47:51.161 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.416964672, 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000808071 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.162 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.164 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:51.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.164 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.164 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.164 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:51.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.165 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.165 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.165 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-07-29T00:47:51.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.166 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.166 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.167 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-07-29T00:47:51.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.167 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.167 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.168 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-07-29T00:47:51.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.168 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.168 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.168 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001575751, 2024-07-29T00:47:51.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000525250 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1249, 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.568741692, 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000455357 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.170 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 92, 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.322952237, 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003510350 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-07-29T00:47:51.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.173 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.174 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:51.175 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.176 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.177 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 295, 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.601756277, 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002039851 2024-07-29T00:47:51.178 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.179 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.180 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-07-29T00:47:51.180 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-07-29T00:47:51.180 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 1, 2024-07-29T00:47:51.180 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 0, 2024-07-29T00:47:51.180 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 1390, 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 1390, 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 681, 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-07-29T00:47:51.181 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:51.182 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:51.183 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:51.183 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:51.183 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:51.183 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:51.183 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:51.183 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:51.184 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:51.184 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:51.184 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:51.184 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:51.184 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:51.185 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:51.186 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 307659, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 195, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:51.187 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 185, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 513785, 2024-07-29T00:47:51.188 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 3832, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-07-29T00:47:51.189 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 1411, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 3619711, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 1411, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1411, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.888945926, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002756162 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.190 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1411, 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1572 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 1411, 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 238, 2024-07-29T00:47:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 1173, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 76, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 681, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 62, 2024-07-29T00:47:51.192 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 307, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:51.193 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:51.194 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 162, 2024-07-29T00:47:51.194 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-07-29T00:47:51.194 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:51.194 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:51.194 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:51.194 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-07-29T00:47:51.195 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-07-29T00:47:51.196 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-07-29T00:47:51.197 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-07-29T00:47:51.197 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-07-29T00:47:51.197 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-07-29T00:47:51.197 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-07-29T00:47:51.197 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-07-29T00:47:51.197 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-07-29T00:47:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-07-29T00:47:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 102, 2024-07-29T00:47:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-07-29T00:47:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-07-29T00:47:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2024-07-29T00:47:51.198 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 204, 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 50 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.199 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-07-29T00:47:51.200 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-07-29T00:47:51.200 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:51.200 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-07-29T00:47:51.200 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-07-29T00:47:51.200 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.200 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-07-29T00:47:51.201 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 129 2024-07-29T00:47:51.201 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.201 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-07-29T00:47:51.201 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:51.201 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 6, 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 6, 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 295, 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 295, 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.202 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:51.203 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:51.203 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:51.203 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:51.203 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 43846, 2024-07-29T00:47:51.203 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 11045117, 2024-07-29T00:47:51.203 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:51.204 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 43846, 2024-07-29T00:47:51.204 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:51.204 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:51.205 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 43846, 2024-07-29T00:47:51.205 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 11045117, 2024-07-29T00:47:51.206 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:51.206 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.206 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.206 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.206 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.206 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:51.207 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 1411, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3649546, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1411, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3649546, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.208 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:51.209 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 1411, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1411, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1411, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1411, 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:51.210 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:51.211 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 295, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 29835, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 295, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 76, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 29835, 2024-07-29T00:47:51.212 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x558faf89c960": { 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:51.213 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2396, 2024-07-29T00:47:51.214 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3522727, 2024-07-29T00:47:51.214 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:51.214 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2396, 2024-07-29T00:47:51.214 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:51.214 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:51.214 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 605, 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3522727, 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:51.215 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-07-29T00:47:51.987 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-07-29T00:47:51.987 INFO:tasks.check_counter:Daemon mds.c mds.imported=4 2024-07-29T00:47:51.987 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=50 2024-07-29T00:47:51.987 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=523 2024-07-29T00:47:51.987 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-07-29T00:47:51.987 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:51.988 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:51.988 DEBUG:tasks.check_counter:Getting stats from f 2024-07-29T00:47:51.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.f perf dump 2024-07-29T00:47:52.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:51 smithi181 ceph-mon[36706]: pgmap v353: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:52.250 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:47:52.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:51 smithi204 ceph-mon[37368]: pgmap v353: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:52.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:51 smithi175 ceph-mon[32138]: pgmap v353: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:52.765 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-07-29T00:47:52.765 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:52.765 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1592, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1591, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 34030441, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 505124, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.160025595, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.046040891, 2024-07-29T00:47:52.766 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.054600470, 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.045580263, 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1590, 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.068524644, 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000043097 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:52.767 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:52.768 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 438, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 411, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 14343214, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 197048, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.066430699, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.019926708, 2024-07-29T00:47:52.769 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.057279199, 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.003683317, 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 410, 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.023750018, 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000057926 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.770 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1719664, 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 147056 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:52.771 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1015, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1192, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 15990139, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 792135, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 8, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.097124797, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.034774135, 2024-07-29T00:47:52.772 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.028351440, 2024-07-29T00:47:52.773 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.025480452, 2024-07-29T00:47:52.773 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:52.773 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1189, 2024-07-29T00:47:52.773 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.057409592, 2024-07-29T00:47:52.773 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000048283 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-29T00:47:52.774 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 470000 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:52.775 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4646, 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.113264917, 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000024379 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:52.776 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-07-29T00:47:52.777 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:52.778 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 2020, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 2020, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:52.779 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 2020, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 14, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-07-29T00:47:52.780 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-07-29T00:47:52.781 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 67, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 21176629, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-07-29T00:47:52.782 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-07-29T00:47:52.783 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-07-29T00:47:52.784 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-07-29T00:47:52.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-07-29T00:47:52.786 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:52.787 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-07-29T00:47:52.788 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 16494, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-07-29T00:47:52.789 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 24, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 3, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 66066646, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 66066534, 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-07-29T00:47:52.790 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 16494 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 94, 2024-07-29T00:47:52.791 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 2123, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 2029, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 39, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 40, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 1, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 2020, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 2180, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 160, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-07-29T00:47:52.792 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 54192, 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215328 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:52.793 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.794 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.795 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.796 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.797 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.799 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.800 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.801 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-07-29T00:47:52.801 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.801 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.803 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.804 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.805 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.806 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.807 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-07-29T00:47:52.808 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.809 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.810 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-07-29T00:47:52.811 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.812 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.813 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 5, 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-07-29T00:47:52.814 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3669, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:52.815 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:52.816 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:52.817 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:52.818 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:52.819 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 229758, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 228, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:52.820 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 185, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 1820212, 2024-07-29T00:47:52.821 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 11798, 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-07-29T00:47:52.822 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2757, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2757, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2757, 2024-07-29T00:47:52.823 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.635963411, 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000956098 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2757, 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2758 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.824 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2757, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2757, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1635, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1119, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-07-29T00:47:52.825 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:52.826 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-07-29T00:47:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-07-29T00:47:52.828 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-07-29T00:47:52.829 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 102, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-07-29T00:47:52.830 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 9, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 9, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.831 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-07-29T00:47:52.832 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-07-29T00:47:52.833 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-07-29T00:47:52.834 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:52.834 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.834 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:52.834 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:52.834 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:52.834 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:52.835 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 3041, 2024-07-29T00:47:52.835 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 64072782, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 3041, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3041, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 64072782, 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:52.836 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2757, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 61872352, 2024-07-29T00:47:52.838 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2757, 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 61872352, 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:52.840 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2757, 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2757, 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2757, 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2757, 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.841 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:52.842 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.843 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x564390170b40": { 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:52.844 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:52.845 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:52.846 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.846 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:52.846 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-07-29T00:47:53.580 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-07-29T00:47:53.581 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-07-29T00:47:53.581 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-07-29T00:47:53.581 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-07-29T00:47:53.581 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-07-29T00:47:53.581 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:53.581 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:53.581 DEBUG:tasks.check_counter:Getting stats from i 2024-07-29T00:47:53.582 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.i perf dump 2024-07-29T00:47:53.834 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:47:54.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:53 smithi204 ceph-mon[37368]: pgmap v354: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:54.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:53 smithi181 ceph-mon[36706]: pgmap v354: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:54.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:53 smithi175 ceph-mon[32138]: pgmap v354: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:54.357 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 40318, 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 29080, 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 11950634, 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 11367967, 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 10, 2024-07-29T00:47:54.358 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 2.064849524, 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.833521961, 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 1.564313343, 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.040890115, 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 29083, 2024-07-29T00:47:54.359 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.513288180, 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000017649 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 65816, 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.009363387, 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000142 2024-07-29T00:47:54.360 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2767, 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2898, 2024-07-29T00:47:54.361 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2427755, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 9745706, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 9, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.219329178, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.085914153, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.095007702, 2024-07-29T00:47:54.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.047691327, 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2899, 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.102370475, 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000035312 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:54.363 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 761, 2024-07-29T00:47:54.364 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000122801, 2024-07-29T00:47:54.364 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000161 2024-07-29T00:47:54.364 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1741456, 2024-07-29T00:47:54.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 498496 2024-07-29T00:47:54.364 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 16066, 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 23103, 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 6027871, 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 17418781, 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 48, 2024-07-29T00:47:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 10, 2024-07-29T00:47:54.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 1.172065959, 2024-07-29T00:47:54.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.532807338, 2024-07-29T00:47:54.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.887307817, 2024-07-29T00:47:54.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.025220569, 2024-07-29T00:47:54.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:54.366 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 23093, 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.403043213, 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000017453 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 31219, 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.004857676, 2024-07-29T00:47:54.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000155 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 5738496, 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 15270928 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-07-29T00:47:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 5151, 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.410143343, 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000467898 2024-07-29T00:47:54.369 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.370 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.370 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:54.370 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:54.370 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:54.370 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2958, 2024-07-29T00:47:54.370 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.126955749, 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000042919 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 9751, 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 8516, 2024-07-29T00:47:54.371 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 8516, 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 14.370176409, 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001687432 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 1235, 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:54.372 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 147, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 22, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 125, 2024-07-29T00:47:54.373 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 116, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 4, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 4, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 9, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 23203, 2024-07-29T00:47:54.374 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 10804, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 1235, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 23, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 522, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 1000, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 1, 2024-07-29T00:47:54.375 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 4, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 53, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 133, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:54.376 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 67, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 21176629, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 133, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:54.377 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 133, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 425, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 6436, 2024-07-29T00:47:54.378 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 3888, 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 1327, 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 579, 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 8348 2024-07-29T00:47:54.379 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 329, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 351, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 182, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 358, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 340, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-07-29T00:47:54.380 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 1438, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 1438, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:54.381 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 5, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 132, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:54.382 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-07-29T00:47:54.383 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 9556, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 9556, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 11, 2024-07-29T00:47:54.384 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 11, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-07-29T00:47:54.385 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 18154147, 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2434, 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 7.767245170, 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003191144 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.386 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 110, 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 1548, 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 1438, 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 33, 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 206, 2024-07-29T00:47:54.387 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 173, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 147, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 2304, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 2157, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 4, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 1517, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 1513, 2024-07-29T00:47:54.388 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 58128, 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223520 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 18395, 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 9751, 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 188, 2024-07-29T00:47:54.389 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 3391, 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1429, 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 7.300300013, 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005108677 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-07-29T00:47:54.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4, 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.006597232, 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001649308 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.391 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2644, 2024-07-29T00:47:54.392 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.271607751, 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000859155 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-07-29T00:47:54.393 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.394 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:54.395 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.396 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.396 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.396 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.396 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.397 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.398 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.398 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-07-29T00:47:54.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.398 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.398 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-07-29T00:47:54.399 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2477, 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.950776487, 2024-07-29T00:47:54.400 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000383841 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 522, 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.524687135, 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002920856 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.401 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.402 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.011005994, 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005502997 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:54.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.406 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.407 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1438, 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.305201797, 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001603061 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:54.409 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:54.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.410 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.410 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.410 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.410 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-07-29T00:47:54.411 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-07-29T00:47:54.411 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-07-29T00:47:54.411 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-07-29T00:47:54.411 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-07-29T00:47:54.411 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-07-29T00:47:54.411 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 2723, 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 2723, 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 581, 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:54.412 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:54.413 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:54.414 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:54.415 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:54.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 297412, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 158, 2024-07-29T00:47:54.417 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:54.418 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 185, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-07-29T00:47:54.419 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 420486, 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 3052, 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.420 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 4617, 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 14297311, 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 4617, 2024-07-29T00:47:54.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4617, 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 14.465798729, 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003133159 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-07-29T00:47:54.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4617, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 4888 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 4617, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 276, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 4341, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-07-29T00:47:54.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 143, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 2577, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 102, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-07-29T00:47:54.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 1453, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 133, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:54.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-07-29T00:47:54.426 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-07-29T00:47:54.427 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-07-29T00:47:54.428 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 102, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-07-29T00:47:54.429 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 354, 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 96 2024-07-29T00:47:54.430 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.431 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 137 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 6, 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-07-29T00:47:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 6, 2024-07-29T00:47:54.433 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 1438, 2024-07-29T00:47:54.433 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 1438, 2024-07-29T00:47:54.433 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:54.433 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.433 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:54.433 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:54.434 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:54.434 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:54.434 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 59147, 2024-07-29T00:47:54.434 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 14362412, 2024-07-29T00:47:54.434 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:54.434 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 59147, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 59147, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 14362412, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.435 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:54.436 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 4617, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 14442693, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4617, 2024-07-29T00:47:54.437 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 14442693, 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.438 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 4617, 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 4617, 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4617, 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 4617, 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.440 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.441 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.441 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.441 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.441 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:54.441 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:54.441 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 1438, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 145382, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 1438, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:54.442 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 143, 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 145382, 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:54.443 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x555f01ff4960": { 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 9556, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 13959843, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 9556, 2024-07-29T00:47:54.444 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2434, 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 13959843, 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:54.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:54.446 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.446 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:54.446 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-07-29T00:47:55.147 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-07-29T00:47:55.148 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-07-29T00:47:55.148 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=329 2024-07-29T00:47:55.148 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=351 2024-07-29T00:47:55.148 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-07-29T00:47:55.148 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:55.148 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:55.148 DEBUG:tasks.check_counter:Getting stats from l 2024-07-29T00:47:55.149 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph daemon mds.l perf dump 2024-07-29T00:47:55.399 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.c/config 2024-07-29T00:47:55.895 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 822, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 824, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 276137, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 261397, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.094451448, 2024-07-29T00:47:55.896 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.028634091, 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.025181558, 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.030609495, 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 822, 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.049082763, 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000059711 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.897 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-07-29T00:47:55.898 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1614, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1793, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3934753, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1060763, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 8, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.125121973, 2024-07-29T00:47:55.899 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.043328347, 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.034420763, 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.037225541, 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1790, 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.074415346, 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000041572 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.900 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 549680 2024-07-29T00:47:55.901 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 288, 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 261, 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1722212, 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 151061, 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-07-29T00:47:55.902 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.046910723, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016328280, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.045525584, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000072668, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 260, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.018183767, 2024-07-29T00:47:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000069937 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1720864, 2024-07-29T00:47:55.905 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 150096 2024-07-29T00:47:55.905 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.905 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-07-29T00:47:55.905 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-07-29T00:47:55.905 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-07-29T00:47:55.905 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4169, 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.108896444, 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000026120 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.906 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.907 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.908 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 444, 2024-07-29T00:47:55.909 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 26, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 418, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 407, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 6, 2024-07-29T00:47:55.910 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-07-29T00:47:55.911 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-07-29T00:47:55.912 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-07-29T00:47:55.913 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-07-29T00:47:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-07-29T00:47:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-07-29T00:47:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-07-29T00:47:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-07-29T00:47:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-07-29T00:47:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-29T00:47:55.915 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-29T00:47:55.915 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-07-29T00:47:55.915 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-07-29T00:47:55.915 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-07-29T00:47:55.915 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-07-29T00:47:55.915 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-07-29T00:47:55.916 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-07-29T00:47:55.917 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-07-29T00:47:55.918 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-07-29T00:47:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 2396, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 4, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-07-29T00:47:55.920 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 7717031, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 7716457, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 2396 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 137, 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 432, 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 295, 2024-07-29T00:47:55.922 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 18, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 18, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 444, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 444, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-07-29T00:47:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49944, 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215328 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-07-29T00:47:55.924 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.925 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.926 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-07-29T00:47:55.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.926 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.926 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.927 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-07-29T00:47:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.930 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-07-29T00:47:55.930 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.930 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.930 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.930 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.930 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.932 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-07-29T00:47:55.933 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.933 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.933 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.933 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-07-29T00:47:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.935 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.935 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.935 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-07-29T00:47:55.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.935 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.936 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.936 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-07-29T00:47:55.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.936 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-07-29T00:47:55.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.938 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.939 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-07-29T00:47:55.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.941 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-07-29T00:47:55.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.943 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.945 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 1, 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 0, 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-07-29T00:47:55.946 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2180, 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-07-29T00:47:55.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-29T00:47:55.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-07-29T00:47:55.949 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-29T00:47:55.950 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-07-29T00:47:55.950 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-29T00:47:55.950 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-07-29T00:47:55.950 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-07-29T00:47:55.951 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 202995, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 185, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-07-29T00:47:55.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8416, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 185, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-07-29T00:47:55.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 661597, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 4741, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-07-29T00:47:55.954 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2432, 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-07-29T00:47:55.955 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2432, 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2432, 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.521978914, 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000625813 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-07-29T00:47:55.956 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-07-29T00:47:55.957 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2432, 2024-07-29T00:47:55.957 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2433 2024-07-29T00:47:55.957 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.957 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2432, 2024-07-29T00:47:55.957 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2432, 2024-07-29T00:47:55.957 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1506, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 924, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-07-29T00:47:55.958 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-07-29T00:47:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-07-29T00:47:55.960 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-07-29T00:47:55.961 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-07-29T00:47:55.962 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 102, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 6, 2024-07-29T00:47:55.963 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 6, 2024-07-29T00:47:55.964 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-07-29T00:47:55.964 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-07-29T00:47:55.964 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-07-29T00:47:55.964 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-07-29T00:47:55.964 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-07-29T00:47:55.964 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-07-29T00:47:55.965 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.966 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-07-29T00:47:55.966 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-07-29T00:47:55.966 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-07-29T00:47:55.966 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-07-29T00:47:55.966 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-07-29T00:47:55.966 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:55.967 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2720, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 5670723, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2720, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2720, 2024-07-29T00:47:55.968 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 5670723, 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-07-29T00:47:55.969 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2432, 2024-07-29T00:47:55.970 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3522732, 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2432, 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3522732, 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:55.972 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2432, 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2432, 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2432, 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2432, 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.973 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:55.974 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:55.975 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55805d1acb40": { 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-07-29T00:47:55.976 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-07-29T00:47:55.977 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-07-29T00:47:55.978 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-07-29T00:47:55.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-07-29T00:47:55.978 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-07-29T00:47:55.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-07-29T00:47:55.978 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.979 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-07-29T00:47:55.979 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-07-29T00:47:56.085 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:55 smithi204 ceph-mon[37368]: pgmap v355: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:56.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:55 smithi181 ceph-mon[36706]: pgmap v355: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:56.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:55 smithi175 ceph-mon[32138]: pgmap v355: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-07-29T00:47:56.584 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-07-29T00:47:56.585 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-29T00:47:56.600 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-29T00:47:56.601 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:47:56.601 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-29T00:47:56.659 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-29T00:47:56.659 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-29T00:47:56.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd blocklist ls 2024-07-29T00:47:57.206 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:47:57.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 -- 172.21.15.175:0/24843309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1041126d0 msgr2=0x7fb104114ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:57.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 --2- 172.21.15.175:0/24843309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1041126d0 0x7fb104114ac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009be0 tx=0x7fb0fc02a750 comp rx=0 tx=0).stop 2024-07-29T00:47:57.868 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 -- 172.21.15.175:0/24843309 shutdown_connections 2024-07-29T00:47:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 --2- 172.21.15.175:0/24843309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1041126d0 0x7fb104114ac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 --2- 172.21.15.175:0/24843309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb104074ba0 0x7fb104112190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 --2- 172.21.15.175:0/24843309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb104074280 0x7fb104074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 -- 172.21.15.175:0/24843309 >> 172.21.15.175:0/24843309 conn(0x7fb1040fd400 msgr2=0x7fb1040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 -- 172.21.15.175:0/24843309 shutdown_connections 2024-07-29T00:47:57.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.868+0000 7fb103200640 1 -- 172.21.15.175:0/24843309 wait complete. 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.869+0000 7fb103200640 1 Processor -- start 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.869+0000 7fb103200640 1 -- start start 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb103200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb104074280 0x7fb10419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb103200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb104074ba0 0x7fb10419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb103200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 0x7fb1041a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb103200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb104118070 con 0x7fb1041126d0 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb103200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb104117ef0 con 0x7fb104074280 2024-07-29T00:47:57.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb103200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041181f0 con 0x7fb104074ba0 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb102800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 0x7fb1041a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb102800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 0x7fb1041a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38000/0 (socket says 172.21.15.175:38000) 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb102800640 1 -- 172.21.15.175:0/1601443942 learned_addr learned my addr 172.21.15.175:0/1601443942 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb101e00640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb104074280 0x7fb10419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.870+0000 7fb0f9400640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb104074ba0 0x7fb10419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb102800640 1 -- 172.21.15.175:0/1601443942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb104074ba0 msgr2=0x7fb10419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb102800640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb104074ba0 0x7fb10419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:57.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb102800640 1 -- 172.21.15.175:0/1601443942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb104074280 msgr2=0x7fb10419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb102800640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb104074280 0x7fb10419e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb102800640 1 -- 172.21.15.175:0/1601443942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1041a3510 con 0x7fb1041126d0 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb0f9400640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb104074ba0 0x7fb10419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb102800640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 0x7fb1041a2e30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009b30 tx=0x7fb0fc0305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.871+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc041020 con 0x7fb1041126d0 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.872+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041a37a0 con 0x7fb1041126d0 2024-07-29T00:47:57.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.872+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0fc003e50 con 0x7fb1041126d0 2024-07-29T00:47:57.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.872+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1041ab0e0 con 0x7fb1041126d0 2024-07-29T00:47:57.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.872+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc036040 con 0x7fb1041126d0 2024-07-29T00:47:57.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.873+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb104074660 con 0x7fb1041126d0 2024-07-29T00:47:57.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.875+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb0fc044020 con 0x7fb1041126d0 2024-07-29T00:47:57.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.876+0000 7fb0fb400640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb0e80a2cf0 0x7fb0e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:47:57.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.876+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fb0fc0f0070 con 0x7fb1041126d0 2024-07-29T00:47:57.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.876+0000 7fb101e00640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb0e80a2cf0 0x7fb0e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:47:57.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.877+0000 7fb101e00640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb0e80a2cf0 0x7fb0e80a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005d00 tx=0x7fb0f400b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:47:57.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:57.880+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb0fc04b020 con 0x7fb1041126d0 2024-07-29T00:47:58.034 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:57 smithi175 ceph-mon[32138]: pgmap v356: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:58.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.083+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1041ab340 con 0x7fb1041126d0 2024-07-29T00:47:58.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.084+0000 7fb0fb400640 1 -- 172.21.15.175:0/1601443942 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+957 (secure 0 0 0) 0x7fb0fc0bcbe0 con 0x7fb1041126d0 2024-07-29T00:47:58.084 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/1189171033 2024-07-30T00:36:29.909303+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:6801/3246788131 2024-07-30T00:36:29.909303+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/2775747625 2024-07-30T00:26:20.140387+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:6800/2801743500 2024-07-30T00:26:20.140387+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:6801/2801743500 2024-07-30T00:26:20.140387+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/257505941 2024-07-30T00:36:29.909303+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/1820809193 2024-07-30T00:26:20.140387+0000 2024-07-29T00:47:58.085 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/1808586297 2024-07-30T00:26:49.412212+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/1674524668 2024-07-30T00:26:49.412212+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/2019123994 2024-07-30T00:36:29.909303+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/1779316541 2024-07-30T00:26:20.140387+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:6800/61876509 2024-07-30T00:26:49.412212+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/4008532755 2024-07-30T00:36:29.909303+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:6801/61876509 2024-07-30T00:26:49.412212+0000 2024-07-29T00:47:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:0/3344352688 2024-07-30T00:26:49.412212+0000 2024-07-29T00:47:58.087 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175:6800/3246788131 2024-07-30T00:36:29.909303+0000 2024-07-29T00:47:58.087 INFO:teuthology.orchestra.run.smithi175.stderr:listed 16 entries 2024-07-29T00:47:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb0e80a2cf0 msgr2=0x7fb0e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb0e80a2cf0 0x7fb0e80a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005d00 tx=0x7fb0f400b410 comp rx=0 tx=0).stop 2024-07-29T00:47:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 msgr2=0x7fb1041a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:47:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 0x7fb1041a2e30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009b30 tx=0x7fb0fc0305b0 comp rx=0 tx=0).stop 2024-07-29T00:47:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 shutdown_connections 2024-07-29T00:47:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fb0e80a2cf0 0x7fb0e80a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:58.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1041126d0 0x7fb1041a2e30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:58.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb104074ba0 0x7fb10419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:58.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 --2- 172.21.15.175:0/1601443942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb104074280 0x7fb10419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:47:58.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 >> 172.21.15.175:0/1601443942 conn(0x7fb1040fd400 msgr2=0x7fb1040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:47:58.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 shutdown_connections 2024-07-29T00:47:58.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:47:58.087+0000 7fb103200640 1 -- 172.21.15.175:0/1601443942 wait complete. 2024-07-29T00:47:58.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:57 smithi181 ceph-mon[36706]: pgmap v356: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:58.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:57 smithi204 ceph-mon[37368]: pgmap v356: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-07-29T00:47:59.033 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-29T00:47:59.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:47:59.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-29T00:47:59.089 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:58 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/1601443942' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-29T00:47:59.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:47:59.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:47:59.242 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-29T00:47:59.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:58 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/1601443942' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-29T00:47:59.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:58 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/1601443942' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-29T00:47:59.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:47:59.345 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-29T00:47:59.421 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-29T00:47:59.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:47:59.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-29T00:47:59.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-29T00:47:59.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-07-29T00:47:59.846 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-29T00:47:59.858 INFO:tasks.cephadm:Teardown begin 2024-07-29T00:47:59.858 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-29T00:47:59.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:48:00.164 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:59 smithi181 ceph-mon[36706]: pgmap v357: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:48:00.165 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:59 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:48:00.165 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:59 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.165 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:59 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.165 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:59 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.165 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:47:59 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:59 smithi175 ceph-mon[32138]: pgmap v357: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:48:00.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:59 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:48:00.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:59 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:59 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:59 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:47:59 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.303 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:00.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:59 smithi204 ceph-mon[37368]: pgmap v357: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:48:00.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:59 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-29T00:48:00.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:59 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:59 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:59 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:00.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:47:59 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 -- 172.21.15.175:0/3745654872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68109aa0 msgr2=0x7f5e68109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/3745654872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68109aa0 0x7f5e68109e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009890 tx=0x7f5e5c02a380 comp rx=0 tx=0).stop 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 -- 172.21.15.175:0/3745654872 shutdown_connections 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/3745654872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68104440 0x7f5e68110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/3745654872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e68103aa0 0x7f5e68103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/3745654872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68109aa0 0x7f5e68109e80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.001 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 -- 172.21.15.175:0/3745654872 >> 172.21.15.175:0/3745654872 conn(0x7f5e680ff740 msgr2=0x7f5e68101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 -- 172.21.15.175:0/3745654872 shutdown_connections 2024-07-29T00:48:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:00.999+0000 7f5e6fa00640 1 -- 172.21.15.175:0/3745654872 wait complete. 2024-07-29T00:48:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.000+0000 7f5e6fa00640 1 Processor -- start 2024-07-29T00:48:01.002 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.000+0000 7f5e6fa00640 1 -- start start 2024-07-29T00:48:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6fa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68103aa0 0x7f5e6819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e68104440 0x7f5e6819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 0x7f5e6819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68113e60 con 0x7f5e68104440 2024-07-29T00:48:01.003 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68113ce0 con 0x7f5e68109aa0 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6fa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68113fe0 con 0x7f5e68103aa0 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 0x7f5e6819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e67e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e68104440 0x7f5e6819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 0x7f5e6819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:49186/0 (socket says 172.21.15.175:49186) 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6da00640 1 -- 172.21.15.175:0/787127175 learned_addr learned my addr 172.21.15.175:0/787127175 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.001+0000 7f5e6d000640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68103aa0 0x7f5e6819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6da00640 1 -- 172.21.15.175:0/787127175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68103aa0 msgr2=0x7f5e6819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:01.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6da00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68103aa0 0x7f5e6819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6da00640 1 -- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e68104440 msgr2=0x7f5e6819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6da00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e68104440 0x7f5e6819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6da00640 1 -- 172.21.15.175:0/787127175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6819f320 con 0x7f5e68109aa0 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6da00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 0x7f5e6819ec40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000eb10 tx=0x7f5e6000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000ce20 con 0x7f5e68109aa0 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e60005020 con 0x7f5e68109aa0 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60010690 con 0x7f5e68109aa0 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.002+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6819f610 con 0x7f5e68109aa0 2024-07-29T00:48:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.003+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681a6ef0 con 0x7f5e68109aa0 2024-07-29T00:48:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.004+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e68103f00 con 0x7f5e68109aa0 2024-07-29T00:48:01.006 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.005+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5e60005540 con 0x7f5e68109aa0 2024-07-29T00:48:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.007+0000 7f5e65600640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e500a2d70 0x7f5e500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.007+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f5e600cf9a0 con 0x7f5e68109aa0 2024-07-29T00:48:01.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.007+0000 7f5e6d000640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e500a2d70 0x7f5e500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:01.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.008+0000 7f5e6d000640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e500a2d70 0x7f5e500a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0097b0 tx=0x7f5e5c0058d0 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:01.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.011+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5e6009c510 con 0x7f5e68109aa0 2024-07-29T00:48:01.241 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.241+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e6810c070 con 0x7f5e68109aa0 2024-07-29T00:48:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:00 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:48:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:00 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:48:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:00 smithi175 ceph-mon[32138]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:01.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.242+0000 7f5e65600640 1 -- 172.21.15.175:0/787127175 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11994 (secure 0 0 0) 0x7f5e6009be10 con 0x7f5e68109aa0 2024-07-29T00:48:01.243 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:48:01.244 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":43,"btime":"2024-07-29T00:43:57:876956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:57.442709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:48:01.244 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 43 2024-07-29T00:48:01.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e500a2d70 msgr2=0x7f5e500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:01.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e500a2d70 0x7f5e500a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0097b0 tx=0x7f5e5c0058d0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 msgr2=0x7f5e6819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:01.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 0x7f5e6819ec40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000eb10 tx=0x7f5e6000ef10 comp rx=0 tx=0).stop 2024-07-29T00:48:01.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 shutdown_connections 2024-07-29T00:48:01.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109aa0 0x7f5e6819ec40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f5e500a2d70 0x7f5e500a5230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e68104440 0x7f5e6819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 --2- 172.21.15.175:0/787127175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68103aa0 0x7f5e6819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:01.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 >> 172.21.15.175:0/787127175 conn(0x7f5e680ff740 msgr2=0x7f5e68100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:01.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 shutdown_connections 2024-07-29T00:48:01.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:01.246+0000 7f5e6fa00640 1 -- 172.21.15.175:0/787127175 wait complete. 2024-07-29T00:48:01.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:00 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:48:01.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:00 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:48:01.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:00 smithi181 ceph-mon[36706]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:01.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:00 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-29T00:48:01.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:00 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-29T00:48:01.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:00 smithi204 ceph-mon[37368]: from='mgr.15033 ' entity='mgr.x' 2024-07-29T00:48:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:01 smithi204 ceph-mon[37368]: pgmap v358: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:48:02.086 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:01 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/787127175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:48:02.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:01 smithi181 ceph-mon[36706]: pgmap v358: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:48:02.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:01 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/787127175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:48:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:01 smithi175 ceph-mon[32138]: pgmap v358: 97 pgs: 97 active+clean; 122 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-29T00:48:02.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:01 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/787127175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:48:02.412 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs dump --format=json 2024-07-29T00:48:02.876 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:03.596 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 -- 172.21.15.175:0/983424939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20112720 msgr2=0x7f6d20114b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 --2- 172.21.15.175:0/983424939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20112720 0x7f6d20114b10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009ca0 tx=0x7f6d1802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 -- 172.21.15.175:0/983424939 shutdown_connections 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 --2- 172.21.15.175:0/983424939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20112720 0x7f6d20114b10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 --2- 172.21.15.175:0/983424939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076d30 0x7f6d201121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 --2- 172.21.15.175:0/983424939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20076410 0x7f6d200767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 -- 172.21.15.175:0/983424939 >> 172.21.15.175:0/983424939 conn(0x7f6d200ff760 msgr2=0x7f6d20101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:03.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 -- 172.21.15.175:0/983424939 shutdown_connections 2024-07-29T00:48:03.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.595+0000 7f6d25000640 1 -- 172.21.15.175:0/983424939 wait complete. 2024-07-29T00:48:03.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.596+0000 7f6d25000640 1 Processor -- start 2024-07-29T00:48:03.598 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.596+0000 7f6d25000640 1 -- start start 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.596+0000 7f6d25000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 0x7f6d201a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d25000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20076d30 0x7f6d201a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d25000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20112720 0x7f6d2019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d25000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201182c0 con 0x7f6d20076410 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d25000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20118140 con 0x7f6d20076d30 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d25000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20118440 con 0x7f6d20112720 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 0x7f6d201a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:03.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 0x7f6d201a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53890/0 (socket says 172.21.15.175:53890) 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 -- 172.21.15.175:0/3336429520 learned_addr learned my addr 172.21.15.175:0/3336429520 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1e800640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20112720 0x7f6d2019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d15400640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20076d30 0x7f6d201a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 -- 172.21.15.175:0/3336429520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20112720 msgr2=0x7f6d2019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20112720 0x7f6d2019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 -- 172.21.15.175:0/3336429520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20076d30 msgr2=0x7f6d201a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20076d30 0x7f6d201a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.597+0000 7f6d1de00640 1 -- 172.21.15.175:0/3336429520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2019ece0 con 0x7f6d20076410 2024-07-29T00:48:03.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.598+0000 7f6d1de00640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 0x7f6d201a43f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e9f0 tx=0x7f6d1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:03.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.598+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000cd80 con 0x7f6d20076410 2024-07-29T00:48:03.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.598+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10005500 con 0x7f6d20076410 2024-07-29T00:48:03.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.598+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10010690 con 0x7f6d20076410 2024-07-29T00:48:03.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.598+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2019efd0 con 0x7f6d20076410 2024-07-29T00:48:03.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.598+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d201ab2c0 con 0x7f6d20076410 2024-07-29T00:48:03.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.599+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d200767f0 con 0x7f6d20076410 2024-07-29T00:48:03.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.602+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6d100046c0 con 0x7f6d20076410 2024-07-29T00:48:03.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.603+0000 7f6d17400640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d040a2d70 0x7f6d040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:03.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.603+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f6d100cfa90 con 0x7f6d20076410 2024-07-29T00:48:03.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.603+0000 7f6d15400640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d040a2d70 0x7f6d040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:03.604 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.604+0000 7f6d15400640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d040a2d70 0x7f6d040a5230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d20103360 tx=0x7f6d08005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:03.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.605+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6d10053df0 con 0x7f6d20076410 2024-07-29T00:48:03.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.833+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d2019fae0 con 0x7f6d20076410 2024-07-29T00:48:03.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.835+0000 7f6d17400640 1 -- 172.21.15.175:0/3336429520 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11994 (secure 0 0 0) 0x7f6d1009c600 con 0x7f6d20076410 2024-07-29T00:48:03.836 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:48:03.836 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":43,"btime":"2024-07-29T00:43:57:876956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1855710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1855710459},{"type":"v1","addr":"172.21.15.175:6837","nonce":1855710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/1735377936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1735377936},{"type":"v1","addr":"172.21.15.175:6841","nonce":1735377936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-29T00:33:58.041883+0000","modified":"2024-07-29T00:43:57.442709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24503,"mds_3":24499,"mds_4":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1301583270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1301583270},{"type":"v1","addr":"172.21.15.204:6833","nonce":1301583270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/4123110499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4123110499},{"type":"v1","addr":"172.21.15.181:6833","nonce":4123110499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/284852041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":284852041},{"type":"v1","addr":"172.21.15.175:6835","nonce":284852041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/568693704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":568693704},{"type":"v1","addr":"172.21.15.181:6835","nonce":568693704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1977765890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1977765890},{"type":"v1","addr":"172.21.15.204:6835","nonce":1977765890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6837/2487746199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2487746199},{"type":"v1","addr":"172.21.15.204:6837","nonce":2487746199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6839/2455267021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2455267021},{"type":"v1","addr":"172.21.15.204:6839","nonce":2455267021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6837/753661139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":753661139},{"type":"v1","addr":"172.21.15.181:6837","nonce":753661139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/1522099567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1522099567},{"type":"v1","addr":"172.21.15.175:6839","nonce":1522099567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/4150501134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4150501134},{"type":"v1","addr":"172.21.15.181:6839","nonce":4150501134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24515,24503,24499,24484]},"id":1}]} 2024-07-29T00:48:03.836 INFO:teuthology.orchestra.run.smithi175.stderr:dumped fsmap epoch 43 2024-07-29T00:48:03.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d040a2d70 msgr2=0x7f6d040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:03.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d040a2d70 0x7f6d040a5230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d20103360 tx=0x7f6d08005f90 comp rx=0 tx=0).stop 2024-07-29T00:48:03.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 msgr2=0x7f6d201a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:03.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 0x7f6d201a43f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e9f0 tx=0x7f6d1000edf0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 shutdown_connections 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20112720 0x7f6d2019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d20076d30 0x7f6d201a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f6d040a2d70 0x7f6d040a5230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 --2- 172.21.15.175:0/3336429520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d20076410 0x7f6d201a43f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.838+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 >> 172.21.15.175:0/3336429520 conn(0x7f6d200ff760 msgr2=0x7f6d20101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:03.840 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.839+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 shutdown_connections 2024-07-29T00:48:03.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:03.839+0000 7f6d25000640 1 -- 172.21.15.175:0/3336429520 wait complete. 2024-07-29T00:48:04.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:03 smithi181 ceph-mon[36706]: pgmap v359: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-07-29T00:48:04.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:03 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/3336429520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:48:04.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:03 smithi175 ceph-mon[32138]: pgmap v359: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-07-29T00:48:04.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:03 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/3336429520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:48:04.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:03 smithi204 ceph-mon[37368]: pgmap v359: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-07-29T00:48:04.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:03 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/3336429520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-29T00:48:04.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd dump --format=json 2024-07-29T00:48:05.409 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:06.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 -- 172.21.15.175:0/3775479375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988104420 msgr2=0x7f3988110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:06.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 --2- 172.21.15.175:0/3775479375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988104420 0x7f3988110950 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3980009860 tx=0x7f398002a340 comp rx=0 tx=0).stop 2024-07-29T00:48:06.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 -- 172.21.15.175:0/3775479375 shutdown_connections 2024-07-29T00:48:06.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 --2- 172.21.15.175:0/3775479375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988104420 0x7f3988110950 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 --2- 172.21.15.175:0/3775479375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3988103a80 0x7f3988103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 --2- 172.21.15.175:0/3775479375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 0x7f3988109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.095+0000 7f398f400640 1 -- 172.21.15.175:0/3775479375 >> 172.21.15.175:0/3775479375 conn(0x7f39880ff780 msgr2=0x7f3988101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:06.097 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:05 smithi175 ceph-mon[32138]: pgmap v360: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-07-29T00:48:06.099 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.096+0000 7f398f400640 1 -- 172.21.15.175:0/3775479375 shutdown_connections 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.096+0000 7f398f400640 1 -- 172.21.15.175:0/3775479375 wait complete. 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 Processor -- start 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 -- start start 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988103a80 0x7f398819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3988104420 0x7f398819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 0x7f398819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:06.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113e80 con 0x7f3988104420 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113d00 con 0x7f3988103a80 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.097+0000 7f398f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3988114000 con 0x7f3988109a80 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988103a80 0x7f398819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 0x7f398819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988103a80 0x7f398819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:49210/0 (socket says 172.21.15.175:49210) 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398ca00640 1 -- 172.21.15.175:0/424559438 learned_addr learned my addr 172.21.15.175:0/424559438 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f3987e00640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3988104420 0x7f398819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:06.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 -- 172.21.15.175:0/424559438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988103a80 msgr2=0x7f398819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988103a80 0x7f398819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 -- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3988104420 msgr2=0x7f398819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3988104420 0x7f398819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 -- 172.21.15.175:0/424559438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398819f2f0 con 0x7f3988109a80 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.098+0000 7f398d400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 0x7f398819ec10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f398002a860 tx=0x7f3980033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.099+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398003f020 con 0x7f3988109a80 2024-07-29T00:48:06.102 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.099+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398819f580 con 0x7f3988109a80 2024-07-29T00:48:06.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.099+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980004050 con 0x7f3988109a80 2024-07-29T00:48:06.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.099+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980037040 con 0x7f3988109a80 2024-07-29T00:48:06.103 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.099+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39881a6ec0 con 0x7f3988109a80 2024-07-29T00:48:06.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.100+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39881051c0 con 0x7f3988109a80 2024-07-29T00:48:06.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.102+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3980042070 con 0x7f3988109a80 2024-07-29T00:48:06.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.103+0000 7f3985600640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39700a2d70 0x7f39700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:06.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.104+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f39800ee620 con 0x7f3988109a80 2024-07-29T00:48:06.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.104+0000 7f398ca00640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39700a2d70 0x7f39700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:06.105 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.104+0000 7f398ca00640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39700a2d70 0x7f39700a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f397c009fa0 tx=0x7f397c010040 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:06.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.108+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f39800bb2a0 con 0x7f3988109a80 2024-07-29T00:48:06.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:05 smithi181 ceph-mon[36706]: pgmap v360: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-07-29T00:48:06.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.297+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f398810c010 con 0x7f3988109a80 2024-07-29T00:48:06.300 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.299+0000 7f3985600640 1 -- 172.21.15.175:0/424559438 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+23312 (secure 0 0 0) 0x7f39800baba0 con 0x7f3988109a80 2024-07-29T00:48:06.301 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-29T00:48:06.301 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":102,"fsid":"f484ff76-4d40-11ef-bcc8-c7b262605968","created":"2024-07-29T00:25:53.732185+0000","modified":"2024-07-29T00:47:15.764784+0000","last_up_change":"2024-07-29T00:32:53.941986+0000","last_in_change":"2024-07-29T00:32:34.471833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-29T00:29:04.051354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-29T00:33:52.192870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-29T00:33:55.363149+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"563e75c0-0a80-4c6d-948c-6a7d756fa193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6803","nonce":311033878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6805","nonce":311033878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6809","nonce":311033878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":311033878},{"type":"v1","addr":"172.21.15.175:6807","nonce":311033878}]},"public_addr":"172.21.15.175:6803/311033878","cluster_addr":"172.21.15.175:6805/311033878","heartbeat_back_addr":"172.21.15.175:6809/311033878","heartbeat_front_addr":"172.21.15.175:6807/311033878","state":["exists","up"]},{"osd":1,"uuid":"f644309b-2ac6-416d-bb60-a115681712b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6801","nonce":2146535950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6803","nonce":2146535950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6807","nonce":2146535950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2146535950},{"type":"v1","addr":"172.21.15.181:6805","nonce":2146535950}]},"public_addr":"172.21.15.181:6801/2146535950","cluster_addr":"172.21.15.181:6803/2146535950","heartbeat_back_addr":"172.21.15.181:6807/2146535950","heartbeat_front_addr":"172.21.15.181:6805/2146535950","state":["exists","up"]},{"osd":2,"uuid":"77a7c751-1c7b-45ae-87fc-91eb99bdced4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6801","nonce":3684418621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6803","nonce":3684418621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6807","nonce":3684418621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3684418621},{"type":"v1","addr":"172.21.15.204:6805","nonce":3684418621}]},"public_addr":"172.21.15.204:6801/3684418621","cluster_addr":"172.21.15.204:6803/3684418621","heartbeat_back_addr":"172.21.15.204:6807/3684418621","heartbeat_front_addr":"172.21.15.204:6805/3684418621","state":["exists","up"]},{"osd":3,"uuid":"ec28d659-e23c-4abf-bc28-9dae71c447c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6811","nonce":2700469757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6813","nonce":2700469757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6817","nonce":2700469757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2700469757},{"type":"v1","addr":"172.21.15.175:6815","nonce":2700469757}]},"public_addr":"172.21.15.175:6811/2700469757","cluster_addr":"172.21.15.175:6813/2700469757","heartbeat_back_addr":"172.21.15.175:6817/2700469757","heartbeat_front_addr":"172.21.15.175:6815/2700469757","state":["exists","up"]},{"osd":4,"uuid":"708e65da-400e-40c5-8991-52201ddf57b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6809","nonce":1605621704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6811","nonce":1605621704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6815","nonce":1605621704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1605621704},{"type":"v1","addr":"172.21.15.181:6813","nonce":1605621704}]},"public_addr":"172.21.15.181:6809/1605621704","cluster_addr":"172.21.15.181:6811/1605621704","heartbeat_back_addr":"172.21.15.181:6815/1605621704","heartbeat_front_addr":"172.21.15.181:6813/1605621704","state":["exists","up"]},{"osd":5,"uuid":"f6258db1-4eca-4859-bca9-9291b0c40d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6809","nonce":377725660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6811","nonce":377725660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6815","nonce":377725660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":377725660},{"type":"v1","addr":"172.21.15.204:6813","nonce":377725660}]},"public_addr":"172.21.15.204:6809/377725660","cluster_addr":"172.21.15.204:6811/377725660","heartbeat_back_addr":"172.21.15.204:6815/377725660","heartbeat_front_addr":"172.21.15.204:6813/377725660","state":["exists","up"]},{"osd":6,"uuid":"58d77f5c-49a7-4727-99a5-c86d7bf62e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6819","nonce":3660975891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6821","nonce":3660975891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6825","nonce":3660975891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3660975891},{"type":"v1","addr":"172.21.15.175:6823","nonce":3660975891}]},"public_addr":"172.21.15.175:6819/3660975891","cluster_addr":"172.21.15.175:6821/3660975891","heartbeat_back_addr":"172.21.15.175:6825/3660975891","heartbeat_front_addr":"172.21.15.175:6823/3660975891","state":["exists","up"]},{"osd":7,"uuid":"393b8cdf-a78f-4d38-b132-f45237f8f088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6817","nonce":591220240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6819","nonce":591220240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6823","nonce":591220240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":591220240},{"type":"v1","addr":"172.21.15.181:6821","nonce":591220240}]},"public_addr":"172.21.15.181:6817/591220240","cluster_addr":"172.21.15.181:6819/591220240","heartbeat_back_addr":"172.21.15.181:6823/591220240","heartbeat_front_addr":"172.21.15.181:6821/591220240","state":["exists","up"]},{"osd":8,"uuid":"6f7f413a-1907-4fd0-8482-293e00ba2c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6817","nonce":1532855581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6819","nonce":1532855581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6823","nonce":1532855581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1532855581},{"type":"v1","addr":"172.21.15.204:6821","nonce":1532855581}]},"public_addr":"172.21.15.204:6817/1532855581","cluster_addr":"172.21.15.204:6819/1532855581","heartbeat_back_addr":"172.21.15.204:6823/1532855581","heartbeat_front_addr":"172.21.15.204:6821/1532855581","state":["exists","up"]},{"osd":9,"uuid":"56c85017-3133-4b51-90da-455c226a0d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6827","nonce":2466426097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6829","nonce":2466426097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6833","nonce":2466426097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2466426097},{"type":"v1","addr":"172.21.15.175:6831","nonce":2466426097}]},"public_addr":"172.21.15.175:6827/2466426097","cluster_addr":"172.21.15.175:6829/2466426097","heartbeat_back_addr":"172.21.15.175:6833/2466426097","heartbeat_front_addr":"172.21.15.175:6831/2466426097","state":["exists","up"]},{"osd":10,"uuid":"d33f82ef-0e35-45ca-ba0c-27da92a81a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6825","nonce":2035344536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6827","nonce":2035344536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6831","nonce":2035344536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2035344536},{"type":"v1","addr":"172.21.15.181:6829","nonce":2035344536}]},"public_addr":"172.21.15.181:6825/2035344536","cluster_addr":"172.21.15.181:6827/2035344536","heartbeat_back_addr":"172.21.15.181:6831/2035344536","heartbeat_front_addr":"172.21.15.181:6829/2035344536","state":["exists","up"]},{"osd":11,"uuid":"a28bd9a6-023c-4dd3-8acd-441f4bc106b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6825","nonce":945517137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6827","nonce":945517137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6831","nonce":945517137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":945517137},{"type":"v1","addr":"172.21.15.204:6829","nonce":945517137}]},"public_addr":"172.21.15.204:6825/945517137","cluster_addr":"172.21.15.204:6827/945517137","heartbeat_back_addr":"172.21.15.204:6831/945517137","heartbeat_front_addr":"172.21.15.204:6829/945517137","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:18.980206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:28:40.885893+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:01.345809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:31.457901+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:29:54.712790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:16.378688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:30:46.675865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:10.780531+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:31:32.996270+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:04.087608+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:28.621094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-29T00:32:51.583687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1189171033":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/3246788131":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/2775747625":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6801/2801743500":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/257505941":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1820809193":"2024-07-30T00:26:20.140387+0000","172.21.15.175:0/1808586297":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/1674524668":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/2019123994":"2024-07-30T00:36:29.909303+0000","172.21.15.175:0/1779316541":"2024-07-30T00:26:20.140387+0000","172.21.15.175:6800/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/4008532755":"2024-07-30T00:36:29.909303+0000","172.21.15.175:6801/61876509":"2024-07-30T00:26:49.412212+0000","172.21.15.175:0/3344352688":"2024-07-30T00:26:49.412212+0000","172.21.15.175:6800/3246788131":"2024-07-30T00:36:29.909303+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39700a2d70 msgr2=0x7f39700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39700a2d70 0x7f39700a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f397c009fa0 tx=0x7f397c010040 comp rx=0 tx=0).stop 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 msgr2=0x7f398819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 0x7f398819ec10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f398002a860 tx=0x7f3980033ae0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 shutdown_connections 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3988109a80 0x7f398819ec10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f39700a2d70 0x7f39700a5230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3988104420 0x7f398819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 --2- 172.21.15.175:0/424559438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3988103a80 0x7f398819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:06.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 >> 172.21.15.175:0/424559438 conn(0x7f39880ff780 msgr2=0x7f3988100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:06.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.302+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 shutdown_connections 2024-07-29T00:48:06.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:06.303+0000 7f398f400640 1 -- 172.21.15.175:0/424559438 wait complete. 2024-07-29T00:48:06.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:05 smithi204 ceph-mon[37368]: pgmap v360: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-07-29T00:48:07.207 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:06 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/424559438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:48:07.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs fail cephfs 2024-07-29T00:48:07.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:06 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/424559438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:48:07.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:06 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/424559438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-29T00:48:07.664 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:08.073 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:07 smithi175 ceph-mon[32138]: pgmap v361: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-07-29T00:48:08.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:07 smithi181 ceph-mon[36706]: pgmap v361: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-07-29T00:48:08.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 -- 172.21.15.175:0/2444841692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd64102060 msgr2=0x7fdd6410e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 --2- 172.21.15.175:0/2444841692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd64102060 0x7fdd6410e590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009cd0 tx=0x7fdd5c02a8e0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 -- 172.21.15.175:0/2444841692 shutdown_connections 2024-07-29T00:48:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 --2- 172.21.15.175:0/2444841692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd64102060 0x7fdd6410e590 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 --2- 172.21.15.175:0/2444841692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd641016c0 0x7fdd64101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 --2- 172.21.15.175:0/2444841692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641076c0 0x7fdd64107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 -- 172.21.15.175:0/2444841692 >> 172.21.15.175:0/2444841692 conn(0x7fdd640fd400 msgr2=0x7fdd640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 -- 172.21.15.175:0/2444841692 shutdown_connections 2024-07-29T00:48:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.319+0000 7fdd63200640 1 -- 172.21.15.175:0/2444841692 wait complete. 2024-07-29T00:48:08.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.320+0000 7fdd63200640 1 Processor -- start 2024-07-29T00:48:08.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.320+0000 7fdd63200640 1 -- start start 2024-07-29T00:48:08.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd63200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641016c0 0x7fdd640741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:08.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd63200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 0x7fdd64074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:08.323 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd63200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd641076c0 0x7fdd64074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd63200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111ac0 con 0x7fdd641076c0 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd63200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111940 con 0x7fdd641016c0 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd63200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111c40 con 0x7fdd64102060 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd61400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 0x7fdd64074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd62800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd641076c0 0x7fdd64074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd61e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641016c0 0x7fdd640741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd61400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 0x7fdd64074720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:34614/0 (socket says 172.21.15.175:34614) 2024-07-29T00:48:08.324 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.321+0000 7fdd61400640 1 -- 172.21.15.175:0/636715784 learned_addr learned my addr 172.21.15.175:0/636715784 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61400640 1 -- 172.21.15.175:0/636715784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641016c0 msgr2=0x7fdd640741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61400640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641016c0 0x7fdd640741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61400640 1 -- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd641076c0 msgr2=0x7fdd64074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61400640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd641076c0 0x7fdd64074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61400640 1 -- 172.21.15.175:0/636715784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd641a6440 con 0x7fdd64102060 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61e00640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641016c0 0x7fdd640741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd61400640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 0x7fdd64074720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00d770 tx=0x7fdd4c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:08.325 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c017070 con 0x7fdd64102060 2024-07-29T00:48:08.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641a6730 con 0x7fdd64102060 2024-07-29T00:48:08.326 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c005500 con 0x7fdd64102060 2024-07-29T00:48:08.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.322+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004e00 con 0x7fdd64102060 2024-07-29T00:48:08.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.323+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a6c70 con 0x7fdd64102060 2024-07-29T00:48:08.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.324+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64101b20 con 0x7fdd64102060 2024-07-29T00:48:08.327 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.326+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fdd4c004fa0 con 0x7fdd64102060 2024-07-29T00:48:08.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.328+0000 7fdd5aa00640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdd480a2d70 0x7fdd480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:08.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.328+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8644+0+0 (secure 0 0 0) 0x7fdd4c0cee50 con 0x7fdd64102060 2024-07-29T00:48:08.328 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.328+0000 7fdd61e00640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdd480a2d70 0x7fdd480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:08.329 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.329+0000 7fdd61e00640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdd480a2d70 0x7fdd480a5230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005d00 tx=0x7fdd5400b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:08.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.331+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdd4c09b340 con 0x7fdd64102060 2024-07-29T00:48:08.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:07 smithi204 ceph-mon[37368]: pgmap v361: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-07-29T00:48:08.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.559+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdd641a6ed0 con 0x7fdd64102060 2024-07-29T00:48:08.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.571+0000 7fdd5aa00640 1 -- 172.21.15.175:0/636715784 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7fdd4c09f640 con 0x7fdd64102060 2024-07-29T00:48:08.572 INFO:teuthology.orchestra.run.smithi175.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-29T00:48:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdd480a2d70 msgr2=0x7fdd480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdd480a2d70 0x7fdd480a5230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005d00 tx=0x7fdd5400b410 comp rx=0 tx=0).stop 2024-07-29T00:48:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 msgr2=0x7fdd64074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 0x7fdd64074720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00d770 tx=0x7fdd4c00dc30 comp rx=0 tx=0).stop 2024-07-29T00:48:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 shutdown_connections 2024-07-29T00:48:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fdd480a2d70 0x7fdd480a5230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd641076c0 0x7fdd64074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd64102060 0x7fdd64074720 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 --2- 172.21.15.175:0/636715784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd641016c0 0x7fdd640741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:08.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 >> 172.21.15.175:0/636715784 conn(0x7fdd640fd400 msgr2=0x7fdd640fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:08.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.575+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 shutdown_connections 2024-07-29T00:48:08.577 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:08.576+0000 7fdd63200640 1 -- 172.21.15.175:0/636715784 wait complete. 2024-07-29T00:48:08.952 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.952 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.953 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mds[66450]: -- 172.21.15.181:0/3403403269 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592736a8680 con 0x55927367d000 2024-07-29T00:48:08.953 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mds[66450]: -- 172.21.15.181:0/3403403269 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55927369f860 con 0x55927367d000 2024-07-29T00:48:08.953 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: starting mds.b at 2024-07-29T00:48:08.953 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f[67003]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.953 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f[67003]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.954 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mds[67007]: -- 172.21.15.204:0/1241415662 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55885358f860 con 0x55885356c800 2024-07-29T00:48:08.954 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f[67003]: starting mds.f at 2024-07-29T00:48:08.954 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h[66144]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.954 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h[66144]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.954 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mds[66148]: -- 172.21.15.181:0/4102994411 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636d343d860 con 0x5636d341b400 2024-07-29T00:48:08.955 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h[66144]: starting mds.h at 2024-07-29T00:48:08.955 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i[67611]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.955 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i[67611]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.955 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mds[67615]: -- 172.21.15.204:0/81037304 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0fd2df860 con 0x55f0fd2bd000 2024-07-29T00:48:08.955 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i[67611]: starting mds.i at 2024-07-29T00:48:08.956 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e[65824]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.956 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e[65824]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.956 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mds[65828]: -- 172.21.15.181:0/1365917829 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a99b57860 con 0x562a99b35000 2024-07-29T00:48:08.956 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e[65824]: starting mds.e at 2024-07-29T00:48:08.956 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l[66691]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.956 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l[66691]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.957 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mds[66695]: -- 172.21.15.204:0/164671755 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605454ab860 con 0x560545489000 2024-07-29T00:48:08.957 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l[66691]: starting mds.l at 2024-07-29T00:48:08.957 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k[65510]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.957 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k[65510]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.957 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mds[65514]: -- 172.21.15.181:0/3473012627 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645afed1860 con 0x5645afeaf000 2024-07-29T00:48:08.957 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k[65510]: starting mds.k at 2024-07-29T00:48:08.958 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c[67304]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.958 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c[67304]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.958 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mds[67308]: -- 172.21.15.204:0/727006957 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dd0ee9c20 con 0x558dd0ec7400 2024-07-29T00:48:08.958 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c[67304]: starting mds.c at 2024-07-29T00:48:08.958 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a[32134]: 2024-07-29T00:48:08.561+0000 7fa2f4400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-29T00:48:08.959 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g[72522]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.959 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g[72522]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.959 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mds[72546]: -- 172.21.15.175:0/4191392817 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a924899860 con 0x55a924876800 2024-07-29T00:48:08.959 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g[72522]: starting mds.g at 2024-07-29T00:48:08.960 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j[71312]: ignoring --setuser ceph since I am not root 2024-07-29T00:48:08.960 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j[71312]: ignoring --setgroup ceph since I am not root 2024-07-29T00:48:08.960 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mds[71316]: -- 172.21.15.175:0/3310856232 <== mon.0 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560513ae3c20 con 0x560513ac0800 2024-07-29T00:48:08.960 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j[71312]: starting mds.j at 2024-07-29T00:48:09.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: osdmap e103: 12 total, 12 up, 12 in 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/636715784' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: osdmap e104: 12 total, 12 up, 12 in 2024-07-29T00:48:09.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:08 smithi181 ceph-mon[36706]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-29T00:48:09.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: osdmap e103: 12 total, 12 up, 12 in 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/636715784' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: osdmap e104: 12 total, 12 up, 12 in 2024-07-29T00:48:09.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:08 smithi175 ceph-mon[32138]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-29T00:48:09.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: osdmap e103: 12 total, 12 up, 12 in 2024-07-29T00:48:09.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/636715784' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-29T00:48:09.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-29T00:48:09.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-29T00:48:09.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-29T00:48:09.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-29T00:48:09.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: osdmap e104: 12 total, 12 up, 12 in 2024-07-29T00:48:09.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:08 smithi204 ceph-mon[37368]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-29T00:48:09.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-07-29T00:48:09.978 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:10.130 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: pgmap v363: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6832/2884364930,v1:172.21.15.181:6834/2884364930] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6833/1402954413,v1:172.21.15.181:6835/1402954413] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6832/3527636247,v1:172.21.15.204:6834/3527636247] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6833/2633696167,v1:172.21.15.204:6835/2633696167] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6836/2161805938,v1:172.21.15.181:6837/2161805938] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6837/2044000542,v1:172.21.15.204:6839/2044000542] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.204:6836/2012500029,v1:172.21.15.204:6838/2012500029] up:boot 2024-07-29T00:48:10.131 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.181:6838/3337180467,v1:172.21.15.181:6839/3337180467] up:boot 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6834/3421375910,v1:172.21.15.175:6838/3421375910] up:boot 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: mds.? [v2:172.21.15.175:6835/4059582397,v1:172.21.15.175:6839/4059582397] up:boot 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:48:10.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:48:10.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:09 smithi175 ceph-mon[32138]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:48:10.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: pgmap v363: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-07-29T00:48:10.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6832/2884364930,v1:172.21.15.181:6834/2884364930] up:boot 2024-07-29T00:48:10.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6833/1402954413,v1:172.21.15.181:6835/1402954413] up:boot 2024-07-29T00:48:10.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6832/3527636247,v1:172.21.15.204:6834/3527636247] up:boot 2024-07-29T00:48:10.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6833/2633696167,v1:172.21.15.204:6835/2633696167] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6836/2161805938,v1:172.21.15.181:6837/2161805938] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6837/2044000542,v1:172.21.15.204:6839/2044000542] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.204:6836/2012500029,v1:172.21.15.204:6838/2012500029] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.181:6838/3337180467,v1:172.21.15.181:6839/3337180467] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6834/3421375910,v1:172.21.15.175:6838/3421375910] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: mds.? [v2:172.21.15.175:6835/4059582397,v1:172.21.15.175:6839/4059582397] up:boot 2024-07-29T00:48:10.251 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:48:10.252 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:48:10.253 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:48:10.253 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:48:10.253 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:48:10.253 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:09 smithi181 ceph-mon[36706]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:48:10.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: pgmap v363: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-07-29T00:48:10.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6832/2884364930,v1:172.21.15.181:6834/2884364930] up:boot 2024-07-29T00:48:10.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6833/1402954413,v1:172.21.15.181:6835/1402954413] up:boot 2024-07-29T00:48:10.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6832/3527636247,v1:172.21.15.204:6834/3527636247] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6833/2633696167,v1:172.21.15.204:6835/2633696167] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6836/2161805938,v1:172.21.15.181:6837/2161805938] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6837/2044000542,v1:172.21.15.204:6839/2044000542] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.204:6836/2012500029,v1:172.21.15.204:6838/2012500029] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.181:6838/3337180467,v1:172.21.15.181:6839/3337180467] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6834/3421375910,v1:172.21.15.175:6838/3421375910] up:boot 2024-07-29T00:48:10.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: mds.? [v2:172.21.15.175:6835/4059582397,v1:172.21.15.175:6839/4059582397] up:boot 2024-07-29T00:48:10.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-29T00:48:10.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-29T00:48:10.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-29T00:48:10.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-29T00:48:10.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-29T00:48:10.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-29T00:48:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-29T00:48:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-29T00:48:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-29T00:48:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-29T00:48:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:48:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:09 smithi204 ceph-mon[37368]: from='mgr.15033 172.21.15.175:0/2245820264' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-29T00:48:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 -- 172.21.15.175:0/465018670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 msgr2=0x7fd39810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 --2- 172.21.15.175:0/465018670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39810d9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd380009900 tx=0x7fd38002a480 comp rx=0 tx=0).stop 2024-07-29T00:48:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 -- 172.21.15.175:0/465018670 shutdown_connections 2024-07-29T00:48:10.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 --2- 172.21.15.175:0/465018670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39810df00 0x7fd398110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 --2- 172.21.15.175:0/465018670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39810d9c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 --2- 172.21.15.175:0/465018670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd398074310 0x7fd3980746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 -- 172.21.15.175:0/465018670 >> 172.21.15.175:0/465018670 conn(0x7fd3980fd400 msgr2=0x7fd3980ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.673+0000 7fd39e000640 1 -- 172.21.15.175:0/465018670 shutdown_connections 2024-07-29T00:48:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.674+0000 7fd39e000640 1 -- 172.21.15.175:0/465018670 wait complete. 2024-07-29T00:48:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.674+0000 7fd39e000640 1 Processor -- start 2024-07-29T00:48:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 -- start start 2024-07-29T00:48:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd398074310 0x7fd39819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd39810df00 0x7fd39819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd398113d00 con 0x7fd398074c30 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd398113b80 con 0x7fd39810df00 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd39e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd398113e80 con 0x7fd398074310 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd38fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd38fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55636/0 (socket says 172.21.15.175:55636) 2024-07-29T00:48:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.675+0000 7fd38fe00640 1 -- 172.21.15.175:0/320375882 learned_addr learned my addr 172.21.15.175:0/320375882 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd39cc00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd398074310 0x7fd39819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd39d600640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd39810df00 0x7fd39819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd38fe00640 1 -- 172.21.15.175:0/320375882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd398074310 msgr2=0x7fd39819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd38fe00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd398074310 0x7fd39819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd38fe00640 1 -- 172.21.15.175:0/320375882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd39810df00 msgr2=0x7fd39819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd38fe00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd39810df00 0x7fd39819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd38fe00640 1 -- 172.21.15.175:0/320375882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39819f1f0 con 0x7fd398074c30 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd39cc00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd398074310 0x7fd39819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-29T00:48:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.676+0000 7fd38fe00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39819a780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd380004770 tx=0x7fd380003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.677+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38003f070 con 0x7fd398074c30 2024-07-29T00:48:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.677+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39819f480 con 0x7fd398074c30 2024-07-29T00:48:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.677+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380031ca0 con 0x7fd398074c30 2024-07-29T00:48:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.677+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3981a6dc0 con 0x7fd398074c30 2024-07-29T00:48:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.677+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380049060 con 0x7fd398074c30 2024-07-29T00:48:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.678+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3980746f0 con 0x7fd398074c30 2024-07-29T00:48:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.680+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd380042070 con 0x7fd398074c30 2024-07-29T00:48:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.681+0000 7fd396000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd37c0a2cf0 0x7fd37c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:10.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.681+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9030+0+0 (secure 0 0 0) 0x7fd3800efc50 con 0x7fd398074c30 2024-07-29T00:48:10.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.682+0000 7fd39cc00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd37c0a2cf0 0x7fd37c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:10.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.683+0000 7fd39cc00640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd37c0a2cf0 0x7fd37c0a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd388009fc0 tx=0x7fd388005f90 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.686+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd3800bc080 con 0x7fd398074c30 2024-07-29T00:48:10.915 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.915+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd398109080 con 0x7fd398074c30 2024-07-29T00:48:10.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.968+0000 7fd396000640 1 -- 172.21.15.175:0/320375882 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7fd3800c0380 con 0x7fd398074c30 2024-07-29T00:48:10.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd37c0a2cf0 msgr2=0x7fd37c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:10.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd37c0a2cf0 0x7fd37c0a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd388009fc0 tx=0x7fd388005f90 comp rx=0 tx=0).stop 2024-07-29T00:48:10.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 msgr2=0x7fd39819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:10.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39819a780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd380004770 tx=0x7fd380003c10 comp rx=0 tx=0).stop 2024-07-29T00:48:10.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 shutdown_connections 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd39810df00 0x7fd39819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fd37c0a2cf0 0x7fd37c0a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd398074c30 0x7fd39819a780 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 --2- 172.21.15.175:0/320375882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd398074310 0x7fd39819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.971+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 >> 172.21.15.175:0/320375882 conn(0x7fd3980fd400 msgr2=0x7fd3980fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.972+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 shutdown_connections 2024-07-29T00:48:10.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:10.972+0000 7fd39e000640 1 -- 172.21.15.175:0/320375882 wait complete. 2024-07-29T00:48:11.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:10 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/320375882' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:48:11.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:10 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/320375882' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:48:11.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:10 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/320375882' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-29T00:48:11.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-29T00:48:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:11 smithi181 ceph-mon[36706]: pgmap v365: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-07-29T00:48:12.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:11 smithi181 ceph-mon[36706]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-29T00:48:12.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:11 smithi181 ceph-mon[36706]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-29T00:48:12.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:11 smithi181 ceph-mon[36706]: Cluster is now healthy 2024-07-29T00:48:12.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:11 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/320375882' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-29T00:48:12.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:11 smithi181 ceph-mon[36706]: fsmap 12 up:standby 2024-07-29T00:48:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:11 smithi175 ceph-mon[32138]: pgmap v365: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-07-29T00:48:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:11 smithi175 ceph-mon[32138]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-29T00:48:12.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:11 smithi175 ceph-mon[32138]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-29T00:48:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:11 smithi175 ceph-mon[32138]: Cluster is now healthy 2024-07-29T00:48:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:11 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/320375882' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-29T00:48:12.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:11 smithi175 ceph-mon[32138]: fsmap 12 up:standby 2024-07-29T00:48:12.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:11 smithi204 ceph-mon[37368]: pgmap v365: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-07-29T00:48:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:11 smithi204 ceph-mon[37368]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-29T00:48:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:11 smithi204 ceph-mon[37368]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-29T00:48:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:11 smithi204 ceph-mon[37368]: Cluster is now healthy 2024-07-29T00:48:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:11 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/320375882' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-29T00:48:12.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:11 smithi204 ceph-mon[37368]: fsmap 12 up:standby 2024-07-29T00:48:12.349 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:13.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.024+0000 7fce35e00640 1 -- 172.21.15.175:0/897904231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30112560 msgr2=0x7fce30114950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.024+0000 7fce35e00640 1 --2- 172.21.15.175:0/897904231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30112560 0x7fce30114950 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce28009ca0 tx=0x7fce2802a8c0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 -- 172.21.15.175:0/897904231 shutdown_connections 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 --2- 172.21.15.175:0/897904231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30112560 0x7fce30114950 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 --2- 172.21.15.175:0/897904231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce30076d30 0x7fce30112020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 --2- 172.21.15.175:0/897904231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce300767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 -- 172.21.15.175:0/897904231 >> 172.21.15.175:0/897904231 conn(0x7fce300ff5c0 msgr2=0x7fce301019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:13.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 -- 172.21.15.175:0/897904231 shutdown_connections 2024-07-29T00:48:13.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.025+0000 7fce35e00640 1 -- 172.21.15.175:0/897904231 wait complete. 2024-07-29T00:48:13.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 Processor -- start 2024-07-29T00:48:13.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 -- start start 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce3019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30076d30 0x7fce3019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce30112560 0x7fce301a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce301180e0 con 0x7fce30076d30 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce30117f60 con 0x7fce30076410 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.026+0000 7fce35e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce30118260 con 0x7fce30112560 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce3019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce27e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30076d30 0x7fce3019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce3019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:35246/0 (socket says 172.21.15.175:35246) 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 -- 172.21.15.175:0/2033091592 learned_addr learned my addr 172.21.15.175:0/2033091592 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce35400640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce30112560 0x7fce301a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 -- 172.21.15.175:0/2033091592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce30112560 msgr2=0x7fce301a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce30112560 0x7fce301a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 -- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30076d30 msgr2=0x7fce3019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30076d30 0x7fce3019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 -- 172.21.15.175:0/2033091592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce301a3510 con 0x7fce30076410 2024-07-29T00:48:13.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.027+0000 7fce34a00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce3019e560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e9f0 tx=0x7fce2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:13.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.028+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000cd80 con 0x7fce30076410 2024-07-29T00:48:13.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.028+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20010910 con 0x7fce30076410 2024-07-29T00:48:13.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.028+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce301a3800 con 0x7fce30076410 2024-07-29T00:48:13.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.028+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20018990 con 0x7fce30076410 2024-07-29T00:48:13.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.028+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce301ab0e0 con 0x7fce30076410 2024-07-29T00:48:13.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.029+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce300767f0 con 0x7fce30076410 2024-07-29T00:48:13.032 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.031+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fce20010ac0 con 0x7fce30076410 2024-07-29T00:48:13.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.032+0000 7fce2e000640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce100a2cf0 0x7fce100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:13.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.032+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9030+0+0 (secure 0 0 0) 0x7fce200cf9c0 con 0x7fce30076410 2024-07-29T00:48:13.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.032+0000 7fce27e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce100a2cf0 0x7fce100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:13.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.033+0000 7fce27e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce100a2cf0 0x7fce100a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce30078f80 tx=0x7fce1800b410 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:13.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.036+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fce2009bd30 con 0x7fce30076410 2024-07-29T00:48:13.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.226+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fce3019f8c0 con 0x7fce30076410 2024-07-29T00:48:13.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.981+0000 7fce2e000640 1 -- 172.21.15.175:0/2033091592 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7fce200a0030 con 0x7fce30076410 2024-07-29T00:48:13.983 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'cephfs_metadata' removed 2024-07-29T00:48:13.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.986+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce100a2cf0 msgr2=0x7fce100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:13.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.986+0000 7fce35e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce100a2cf0 0x7fce100a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce30078f80 tx=0x7fce1800b410 comp rx=0 tx=0).stop 2024-07-29T00:48:13.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.986+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 msgr2=0x7fce3019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:13.987 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.986+0000 7fce35e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce3019e560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e9f0 tx=0x7fce2000eeb0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 shutdown_connections 2024-07-29T00:48:13.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce30112560 0x7fce301a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7fce100a2cf0 0x7fce100a51b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30076d30 0x7fce3019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.988 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 --2- 172.21.15.175:0/2033091592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30076410 0x7fce3019e560 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:13.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 >> 172.21.15.175:0/2033091592 conn(0x7fce300ff5c0 msgr2=0x7fce301019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:13.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.987+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 shutdown_connections 2024-07-29T00:48:13.989 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:13.988+0000 7fce35e00640 1 -- 172.21.15.175:0/2033091592 wait complete. 2024-07-29T00:48:14.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:13 smithi181 ceph-mon[36706]: pgmap v366: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-29T00:48:14.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:13 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2033091592' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:14.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:13 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:14.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:13 smithi175 ceph-mon[32138]: pgmap v366: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-29T00:48:14.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:13 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2033091592' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:14.285 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:13 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:14.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:13 smithi204 ceph-mon[37368]: pgmap v366: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-29T00:48:14.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:13 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2033091592' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:14.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:13 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:14.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701 shell --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-29T00:48:15.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:14 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-29T00:48:15.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:14 smithi181 ceph-mon[36706]: osdmap e105: 12 total, 12 up, 12 in 2024-07-29T00:48:15.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:14 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-29T00:48:15.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:14 smithi175 ceph-mon[32138]: osdmap e105: 12 total, 12 up, 12 in 2024-07-29T00:48:15.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:14 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-29T00:48:15.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:14 smithi204 ceph-mon[37368]: osdmap e105: 12 total, 12 up, 12 in 2024-07-29T00:48:15.445 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/mon.a/config 2024-07-29T00:48:16.130 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 -- 172.21.15.175:0/322902689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0108670 msgr2=0x7f36f0114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 --2- 172.21.15.175:0/322902689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0108670 0x7f36f0114b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009cd0 tx=0x7f36e802a8e0 comp rx=0 tx=0).stop 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 -- 172.21.15.175:0/322902689 shutdown_connections 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 --2- 172.21.15.175:0/322902689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0108670 0x7f36f0114b90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 --2- 172.21.15.175:0/322902689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f0107d10 0x7f36f0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 --2- 172.21.15.175:0/322902689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f010dd50 0x7f36f010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 -- 172.21.15.175:0/322902689 >> 172.21.15.175:0/322902689 conn(0x7f36f0077670 msgr2=0x7f36f0079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:16.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 -- 172.21.15.175:0/322902689 shutdown_connections 2024-07-29T00:48:16.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.130+0000 7f36f5200640 1 -- 172.21.15.175:0/322902689 wait complete. 2024-07-29T00:48:16.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 Processor -- start 2024-07-29T00:48:16.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 -- start start 2024-07-29T00:48:16.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 0x7f36f019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f0108670 0x7f36f019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f010dd50 0x7f36f01a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0118140 con 0x7f36f010dd50 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0117fc0 con 0x7f36f0108670 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.131+0000 7f36f5200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01182c0 con 0x7f36f0107d10 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 0x7f36f019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36efe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f010dd50 0x7f36f01a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 0x7f36f019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.175:44164/0 (socket says 172.21.15.175:44164) 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 -- 172.21.15.175:0/2230641967 learned_addr learned my addr 172.21.15.175:0/2230641967 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36eea00640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f0108670 0x7f36f019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 -- 172.21.15.175:0/2230641967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f0108670 msgr2=0x7f36f019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f0108670 0x7f36f019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 -- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f010dd50 msgr2=0x7f36f01a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f010dd50 0x7f36f01a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:16.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 -- 172.21.15.175:0/2230641967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f01a3650 con 0x7f36f0107d10 2024-07-29T00:48:16.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.132+0000 7f36ef400640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 0x7f36f019e6a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f36e000b4a0 tx=0x7f36e000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:16.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.133+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0014070 con 0x7f36f0107d10 2024-07-29T00:48:16.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.133+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f01a3940 con 0x7f36f0107d10 2024-07-29T00:48:16.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.133+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e0010af0 con 0x7f36f0107d10 2024-07-29T00:48:16.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.133+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017d10 con 0x7f36f0107d10 2024-07-29T00:48:16.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.133+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f01ab220 con 0x7f36f0107d10 2024-07-29T00:48:16.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.134+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f0108130 con 0x7f36f0107d10 2024-07-29T00:48:16.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.136+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f36e0017070 con 0x7f36f0107d10 2024-07-29T00:48:16.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.137+0000 7f36e7e00640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36d40a2cf0 0x7f36d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-29T00:48:16.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.138+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8476+0+0 (secure 0 0 0) 0x7f36e00ce680 con 0x7f36f0107d10 2024-07-29T00:48:16.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.138+0000 7f36eea00640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36d40a2cf0 0x7f36d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-29T00:48:16.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.138+0000 7f36eea00640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36d40a2cf0 0x7f36d40a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f36f010d320 tx=0x7f36dc00b390 comp rx=0 tx=0).ready entity=mgr.15033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-29T00:48:16.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.139+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f36e009ad30 con 0x7f36f0107d10 2024-07-29T00:48:16.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:15 smithi181 ceph-mon[36706]: pgmap v368: 65 pgs: 65 active+clean; 22 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:48:16.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:15 smithi181 ceph-mon[36706]: osdmap e106: 12 total, 12 up, 12 in 2024-07-29T00:48:16.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:15 smithi175 ceph-mon[32138]: pgmap v368: 65 pgs: 65 active+clean; 22 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:48:16.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:15 smithi175 ceph-mon[32138]: osdmap e106: 12 total, 12 up, 12 in 2024-07-29T00:48:16.335 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:16.333+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36f019fa00 con 0x7f36f0107d10 2024-07-29T00:48:16.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:15 smithi204 ceph-mon[37368]: pgmap v368: 65 pgs: 65 active+clean; 22 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:48:16.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:15 smithi204 ceph-mon[37368]: osdmap e106: 12 total, 12 up, 12 in 2024-07-29T00:48:17.007 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.002+0000 7f36e7e00640 1 -- 172.21.15.175:0/2230641967 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7f36f019fa00 con 0x7f36f0107d10 2024-07-29T00:48:17.008 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'cephfs_data' removed 2024-07-29T00:48:17.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36d40a2cf0 msgr2=0x7f36d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:17.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36d40a2cf0 0x7f36d40a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f36f010d320 tx=0x7f36dc00b390 comp rx=0 tx=0).stop 2024-07-29T00:48:17.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 msgr2=0x7f36f019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-29T00:48:17.010 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 0x7f36f019e6a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f36e000b4a0 tx=0x7f36e000b960 comp rx=0 tx=0).stop 2024-07-29T00:48:17.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 shutdown_connections 2024-07-29T00:48:17.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:6800/1625031332,v1:172.21.15.175:6801/1625031332] conn(0x7f36d40a2cf0 0x7f36d40a51b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:17.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f010dd50 0x7f36f01a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:17.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f0108670 0x7f36f019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:17.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 --2- 172.21.15.175:0/2230641967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0107d10 0x7f36f019e6a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-29T00:48:17.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 >> 172.21.15.175:0/2230641967 conn(0x7f36f0077670 msgr2=0x7f36f0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-29T00:48:17.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.009+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 shutdown_connections 2024-07-29T00:48:17.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-29T00:48:17.010+0000 7f36f5200640 1 -- 172.21.15.175:0/2230641967 wait complete. 2024-07-29T00:48:17.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:16 smithi181 ceph-mon[36706]: from='client.? 172.21.15.175:0/2230641967' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:17.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:16 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:17.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:17 smithi175 ceph-mon[32138]: from='client.? 172.21.15.175:0/2230641967' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:17.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:17 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:17.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:16 smithi204 ceph-mon[37368]: from='client.? 172.21.15.175:0/2230641967' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:17.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:16 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-29T00:48:18.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:48:18.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:48:18.165 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:48:18.196 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-29T00:48:18.196 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-29T00:48:18.252 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-29T00:48:18.269 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-29T00:48:18.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 ceph-mon[32138]: pgmap v370: 65 pgs: 65 active+clean; 22 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:48:18.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 ceph-mon[32138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-29T00:48:18.284 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 ceph-mon[32138]: osdmap e107: 12 total, 12 up, 12 in 2024-07-29T00:48:18.286 INFO:tasks.cephadm:Stopping all daemons... 2024-07-29T00:48:18.286 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-07-29T00:48:18.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a 2024-07-29T00:48:18.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:18 smithi204 ceph-mon[37368]: pgmap v370: 65 pgs: 65 active+clean; 22 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:48:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:18 smithi204 ceph-mon[37368]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-29T00:48:18.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:18 smithi204 ceph-mon[37368]: osdmap e107: 12 total, 12 up, 12 in 2024-07-29T00:48:18.499 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:18 smithi181 ceph-mon[36706]: pgmap v370: 65 pgs: 65 active+clean; 22 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-29T00:48:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:18 smithi181 ceph-mon[36706]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-29T00:48:18.500 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:18 smithi181 ceph-mon[36706]: osdmap e107: 12 total, 12 up, 12 in 2024-07-29T00:48:18.784 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 systemd[1]: Stopping Ceph mon.a for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:19.275 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a[32134]: 2024-07-29T00:48:18.913+0000 7fa2fb200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:19.275 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a[32134]: 2024-07-29T00:48:18.913+0000 7fa2fb200640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-07-29T00:48:19.275 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:18 smithi175 podman[111132]: 2024-07-29 00:48:18.957689717 +0000 UTC m=+0.240491081 container died 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-29T00:48:19.872 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:19 smithi175 podman[111132]: 2024-07-29 00:48:19.552740229 +0000 UTC m=+0.835541630 container cleanup 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-07-29T00:48:19.872 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:19 smithi175 bash[111132]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a 2024-07-29T00:48:20.133 INFO:journalctl@ceph.mon.a.smithi175.stdout:Jul 29 00:48:19 smithi175 podman[111147]: 2024-07-29 00:48:19.871518383 +0000 UTC m=+0.897744287 container remove 50d2691b7e35ad1d9d0ebb51e72e70caa4e586ae7ae27fc94c42d0089db6a6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-07-29T00:48:20.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.a.service' 2024-07-29T00:48:20.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:20.307 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-07-29T00:48:20.308 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-07-29T00:48:20.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.b 2024-07-29T00:48:20.625 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:20 smithi181 systemd[1]: Stopping Ceph mon.b for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-b[36702]: 2024-07-29T00:48:20.624+0000 7f4c80bf7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:20.999 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-b[36702]: 2024-07-29T00:48:20.624+0000 7f4c80bf7640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-07-29T00:48:21.000 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:20 smithi181 podman[72606]: 2024-07-29 00:48:20.706937923 +0000 UTC m=+0.207183541 container died 6ebc6a9cf208fcd9965b3c95b3b6cc378d9f12ad4e7339cdf69d95e00c766720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True) 2024-07-29T00:48:21.368 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:21 smithi181 podman[72606]: 2024-07-29 00:48:21.125880763 +0000 UTC m=+0.626126335 container cleanup 6ebc6a9cf208fcd9965b3c95b3b6cc378d9f12ad4e7339cdf69d95e00c766720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-07-29T00:48:21.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:21 smithi181 bash[72606]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-b 2024-07-29T00:48:21.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.b.service' 2024-07-29T00:48:21.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:21 smithi181 podman[72620]: 2024-07-29 00:48:21.369298321 +0000 UTC m=+0.655088836 container remove 6ebc6a9cf208fcd9965b3c95b3b6cc378d9f12ad4e7339cdf69d95e00c766720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-29T00:48:21.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:21 smithi181 systemd[1]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.b.service: Deactivated successfully. 2024-07-29T00:48:21.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:21 smithi181 systemd[1]: Stopped Ceph mon.b for f484ff76-4d40-11ef-bcc8-c7b262605968. 2024-07-29T00:48:21.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jul 29 00:48:21 smithi181 systemd[1]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.b.service: Consumed 14.236s CPU time. 2024-07-29T00:48:21.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:21.645 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-07-29T00:48:21.645 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-07-29T00:48:21.645 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.c 2024-07-29T00:48:21.940 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:21 smithi204 systemd[1]: Stopping Ceph mon.c for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:22.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-c[37364]: 2024-07-29T00:48:21.968+0000 7f130edc9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:22.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-c[37364]: 2024-07-29T00:48:21.968+0000 7f130edc9640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-07-29T00:48:22.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:21 smithi204 podman[73932]: 2024-07-29 00:48:21.988812785 +0000 UTC m=+0.151309341 container died 0a892ddf45e65019ddbb803ad62840844f03b2a54810d137a6dfe36be046f4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-29T00:48:22.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:22 smithi204 podman[73932]: 2024-07-29 00:48:22.36927369 +0000 UTC m=+0.531770244 container cleanup 0a892ddf45e65019ddbb803ad62840844f03b2a54810d137a6dfe36be046f4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-29T00:48:22.586 INFO:journalctl@ceph.mon.c.smithi204.stdout:Jul 29 00:48:22 smithi204 bash[73932]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mon-c 2024-07-29T00:48:22.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mon.c.service' 2024-07-29T00:48:22.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:22.840 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-07-29T00:48:22.840 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-07-29T00:48:22.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x 2024-07-29T00:48:23.284 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:22 smithi175 systemd[1]: Stopping Ceph mgr.x for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:23.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:48:23.472+0000 7f7249000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:23.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x[32367]: 2024-07-29T00:48:23.472+0000 7f7249000640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-07-29T00:48:23.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:23 smithi175 podman[111252]: 2024-07-29 00:48:23.568840388 +0000 UTC m=+0.277482116 container died 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-07-29T00:48:24.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:24 smithi175 podman[111252]: 2024-07-29 00:48:24.308717832 +0000 UTC m=+1.017359519 container cleanup 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-07-29T00:48:24.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:24 smithi175 bash[111252]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x 2024-07-29T00:48:24.785 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Jul 29 00:48:24 smithi175 podman[111266]: 2024-07-29 00:48:24.543960972 +0000 UTC m=+0.952473971 container remove 8df856417f2b161e70c19ac75610a2f7b373c98799c05305e391697aafb9aaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba82afec3628bf830c0830558bf3d3c3aea66701, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:48:25.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.x.service' 2024-07-29T00:48:25.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:25.156 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-07-29T00:48:25.157 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-07-29T00:48:25.157 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.y 2024-07-29T00:48:25.499 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:48:25 smithi181 systemd[1]: Stopping Ceph mgr.y for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:25.832 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:48:25 smithi181 podman[72730]: 2024-07-29 00:48:25.501324539 +0000 UTC m=+0.150272410 container died 613bd09f3e12ffaecf445092d8bfb960e6933b86c9065e3f54176296ac3cfb36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-29T00:48:26.502 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:48:26 smithi181 podman[72730]: 2024-07-29 00:48:26.148312767 +0000 UTC m=+0.797260638 container cleanup 613bd09f3e12ffaecf445092d8bfb960e6933b86c9065e3f54176296ac3cfb36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y, ceph=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-07-29T00:48:26.502 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jul 29 00:48:26 smithi181 bash[72730]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-y 2024-07-29T00:48:26.749 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.y.service' 2024-07-29T00:48:26.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:26.783 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-07-29T00:48:26.783 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-07-29T00:48:26.783 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.z 2024-07-29T00:48:27.085 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:48:26 smithi204 systemd[1]: Stopping Ceph mgr.z for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:27.467 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:48:27 smithi204 podman[74066]: 2024-07-29 00:48:27.170107127 +0000 UTC m=+0.182348039 container died 71ba458fb5a267b942fe01a29dac6b97c6f107203bb13071d65aa6c525689d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-29T00:48:27.826 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:48:27 smithi204 podman[74066]: 2024-07-29 00:48:27.558162816 +0000 UTC m=+0.570403727 container cleanup 71ba458fb5a267b942fe01a29dac6b97c6f107203bb13071d65aa6c525689d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-29T00:48:27.826 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Jul 29 00:48:27 smithi204 bash[74066]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mgr-z 2024-07-29T00:48:27.973 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mgr.z.service' 2024-07-29T00:48:28.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:28.008 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-07-29T00:48:28.008 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-29T00:48:28.008 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.0 2024-07-29T00:48:28.284 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:28 smithi175 systemd[1]: Stopping Ceph osd.0 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:29.034 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0[46025]: 2024-07-29T00:48:28.578+0000 7f043b400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:29.035 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0[46025]: 2024-07-29T00:48:28.578+0000 7f043b400640 -1 osd.0 107 *** Got signal Terminated *** 2024-07-29T00:48:29.035 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0[46025]: 2024-07-29T00:48:28.578+0000 7f043b400640 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:48:34.503 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:34 smithi175 podman[111374]: 2024-07-29 00:48:34.156641882 +0000 UTC m=+5.719335135 container died fc26bc4272fdfb3903ccfe338205a7cc5fa91c623ef9e4c62f650399d9aabd8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-29T00:48:34.784 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:34 smithi175 podman[111374]: 2024-07-29 00:48:34.622468805 +0000 UTC m=+6.185162073 container cleanup fc26bc4272fdfb3903ccfe338205a7cc5fa91c623ef9e4c62f650399d9aabd8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:48:34.785 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:34 smithi175 bash[111374]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0 2024-07-29T00:48:35.285 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:34 smithi175 podman[111393]: 2024-07-29 00:48:34.823966926 +0000 UTC m=+0.653207621 container remove fc26bc4272fdfb3903ccfe338205a7cc5fa91c623ef9e4c62f650399d9aabd8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS) 2024-07-29T00:48:35.970 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:35 smithi175 podman[111481]: 2024-07-29 00:48:35.543460574 +0000 UTC m=+0.036181970 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:48:35.971 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:35 smithi175 podman[111481]: 2024-07-29 00:48:35.683395574 +0000 UTC m=+0.176116983 container create 919682c4cf1eaa75e08181ec370178d9b19afb5776dea15c2524eaff5c44638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-07-29T00:48:36.285 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:35 smithi175 podman[111481]: 2024-07-29 00:48:35.970423756 +0000 UTC m=+0.463145165 container init 919682c4cf1eaa75e08181ec370178d9b19afb5776dea15c2524eaff5c44638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-07-29T00:48:36.285 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:35 smithi175 podman[111481]: 2024-07-29 00:48:35.981687351 +0000 UTC m=+0.474408760 container start 919682c4cf1eaa75e08181ec370178d9b19afb5776dea15c2524eaff5c44638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-29T00:48:36.285 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:36 smithi175 podman[111481]: 2024-07-29 00:48:36.102289304 +0000 UTC m=+0.595010756 container attach 919682c4cf1eaa75e08181ec370178d9b19afb5776dea15c2524eaff5c44638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-07-29T00:48:36.651 INFO:journalctl@ceph.osd.0.smithi175.stdout:Jul 29 00:48:36 smithi175 podman[111481]: 2024-07-29 00:48:36.285465487 +0000 UTC m=+0.778186865 container died 919682c4cf1eaa75e08181ec370178d9b19afb5776dea15c2524eaff5c44638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:48:37.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.0.service' 2024-07-29T00:48:37.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:37.128 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-29T00:48:37.129 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-29T00:48:37.129 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.1 2024-07-29T00:48:37.499 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:37 smithi181 systemd[1]: Stopping Ceph osd.1 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:37.999 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1[41140]: 2024-07-29T00:48:37.502+0000 7f41fa7d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:38.000 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1[41140]: 2024-07-29T00:48:37.502+0000 7f41fa7d8640 -1 osd.1 107 *** Got signal Terminated *** 2024-07-29T00:48:38.000 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1[41140]: 2024-07-29T00:48:37.502+0000 7f41fa7d8640 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:48:43.222 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:42 smithi181 podman[72869]: 2024-07-29 00:48:42.909195424 +0000 UTC m=+5.579411466 container died 201e808b1afeb4c5ddc3d673ec6d595d700e7f74bb9066e5f35b481ec77699be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-ba82afe) 2024-07-29T00:48:43.499 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:43 smithi181 podman[72869]: 2024-07-29 00:48:43.320034973 +0000 UTC m=+5.990251016 container cleanup 201e808b1afeb4c5ddc3d673ec6d595d700e7f74bb9066e5f35b481ec77699be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-29T00:48:43.500 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:43 smithi181 bash[72869]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1 2024-07-29T00:48:43.989 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:43 smithi181 podman[72884]: 2024-07-29 00:48:43.546463049 +0000 UTC m=+0.636972895 container remove 201e808b1afeb4c5ddc3d673ec6d595d700e7f74bb9066e5f35b481ec77699be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-29T00:48:44.250 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:43 smithi181 podman[72975]: 2024-07-29 00:48:43.89282079 +0000 UTC m=+0.017245099 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:48:44.250 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:44 smithi181 podman[72975]: 2024-07-29 00:48:44.030741842 +0000 UTC m=+0.155166193 container create 35dd12c2c9c91f9864ff328d114aecbe198e9be8653724ede1e543e86d501a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-29T00:48:44.750 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:44 smithi181 podman[72975]: 2024-07-29 00:48:44.270762914 +0000 UTC m=+0.395187229 container init 35dd12c2c9c91f9864ff328d114aecbe198e9be8653724ede1e543e86d501a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-07-29T00:48:44.750 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:44 smithi181 podman[72975]: 2024-07-29 00:48:44.273799686 +0000 UTC m=+0.398223994 container start 35dd12c2c9c91f9864ff328d114aecbe198e9be8653724ede1e543e86d501a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-07-29T00:48:44.750 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:44 smithi181 podman[72975]: 2024-07-29 00:48:44.383832805 +0000 UTC m=+0.508257129 container attach 35dd12c2c9c91f9864ff328d114aecbe198e9be8653724ede1e543e86d501a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True) 2024-07-29T00:48:44.750 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jul 29 00:48:44 smithi181 podman[72975]: 2024-07-29 00:48:44.462824626 +0000 UTC m=+0.587248935 container died 35dd12c2c9c91f9864ff328d114aecbe198e9be8653724ede1e543e86d501a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-29T00:48:45.190 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.1.service' 2024-07-29T00:48:45.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:45.229 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-29T00:48:45.229 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-29T00:48:45.229 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.2 2024-07-29T00:48:45.546 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:45 smithi204 systemd[1]: Stopping Ceph osd.2 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:45.835 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2[42343]: 2024-07-29T00:48:45.544+0000 7f8e4fed7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:45.908 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2[42343]: 2024-07-29T00:48:45.544+0000 7f8e4fed7640 -1 osd.2 107 *** Got signal Terminated *** 2024-07-29T00:48:45.908 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2[42343]: 2024-07-29T00:48:45.544+0000 7f8e4fed7640 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:48:51.336 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:51 smithi204 podman[74193]: 2024-07-29 00:48:51.077422972 +0000 UTC m=+5.649098326 container died 220f060e895fa8a643fddf3d12af1756a6b4a58ab588aea5efe412a2d3439982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe) 2024-07-29T00:48:51.714 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:51 smithi204 podman[74193]: 2024-07-29 00:48:51.463193211 +0000 UTC m=+6.034868572 container cleanup 220f060e895fa8a643fddf3d12af1756a6b4a58ab588aea5efe412a2d3439982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:48:51.715 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:51 smithi204 bash[74193]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2 2024-07-29T00:48:52.061 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:51 smithi204 podman[74208]: 2024-07-29 00:48:51.714830243 +0000 UTC m=+0.639743924 container remove 220f060e895fa8a643fddf3d12af1756a6b4a58ab588aea5efe412a2d3439982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=squid-ba82afe, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-07-29T00:48:52.335 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:52 smithi204 podman[74309]: 2024-07-29 00:48:51.964834841 +0000 UTC m=+0.017109516 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:48:52.336 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:52 smithi204 podman[74309]: 2024-07-29 00:48:52.091216281 +0000 UTC m=+0.143490954 container create df1ebfb5c2f878ea02bef5d713e0285871fbb81882218b80105cedb1a12ab63c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, io.buildah.version=1.36.0) 2024-07-29T00:48:52.657 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:52 smithi204 podman[74309]: 2024-07-29 00:48:52.460851264 +0000 UTC m=+0.513125952 container init df1ebfb5c2f878ea02bef5d713e0285871fbb81882218b80105cedb1a12ab63c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:48:52.658 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:52 smithi204 podman[74309]: 2024-07-29 00:48:52.46402866 +0000 UTC m=+0.516303345 container start df1ebfb5c2f878ea02bef5d713e0285871fbb81882218b80105cedb1a12ab63c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe) 2024-07-29T00:48:52.658 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:52 smithi204 podman[74309]: 2024-07-29 00:48:52.561006848 +0000 UTC m=+0.613281537 container attach df1ebfb5c2f878ea02bef5d713e0285871fbb81882218b80105cedb1a12ab63c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:48:52.964 INFO:journalctl@ceph.osd.2.smithi204.stdout:Jul 29 00:48:52 smithi204 podman[74309]: 2024-07-29 00:48:52.658272312 +0000 UTC m=+0.710546987 container died df1ebfb5c2f878ea02bef5d713e0285871fbb81882218b80105cedb1a12ab63c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-07-29T00:48:53.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.2.service' 2024-07-29T00:48:53.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:48:53.278 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-29T00:48:53.278 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-29T00:48:53.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.3 2024-07-29T00:48:53.784 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:53 smithi175 systemd[1]: Stopping Ceph osd.3 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:48:54.285 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:53 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:53.866+0000 7f678e400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:48:54.285 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:53 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:53.866+0000 7f678e400640 -1 osd.3 107 *** Got signal Terminated *** 2024-07-29T00:48:54.286 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:53 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:53.866+0000 7f678e400640 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:48:55.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:48:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:48:54.767+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:48:55.284 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:54 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:54.985+0000 7f6784e00640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:31.653408+0000 front 2024-07-29T00:48:31.653262+0000 (oldest deadline 2024-07-29T00:48:54.454810+0000) 2024-07-29T00:48:55.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:48:55 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:48:55.031+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:48:55.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:48:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:48:55.350+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:48:56.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:48:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:48:55.728+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:48:56.284 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:55 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:55.981+0000 7f6784e00640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:31.653408+0000 front 2024-07-29T00:48:31.653262+0000 (oldest deadline 2024-07-29T00:48:54.454810+0000) 2024-07-29T00:48:56.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:48:56 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:48:56.053+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:48:56.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:48:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:48:56.308+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:48:57.085 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:48:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:48:56.754+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:48:57.284 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:56 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:56.932+0000 7f6784e00640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:31.653408+0000 front 2024-07-29T00:48:31.653262+0000 (oldest deadline 2024-07-29T00:48:54.454810+0000) 2024-07-29T00:48:57.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:48:57 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:48:57.018+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:48:57.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:48:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:48:57.358+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:48:58.085 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:48:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:48:57.766+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:48:58.284 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:57 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3[52894]: 2024-07-29T00:48:57.946+0000 7f6784e00640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:31.653408+0000 front 2024-07-29T00:48:31.653262+0000 (oldest deadline 2024-07-29T00:48:54.454810+0000) 2024-07-29T00:48:58.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:48:58 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:48:58.044+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:48:58.332 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:48:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:48:58.016+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:48:58.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:48:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:48:58.332+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:48:59.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:48:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:48:58.752+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:48:59.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:48:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:48:58.851+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:48:59.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:48:59 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:48:59.027+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:48:59.285 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:48:59 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:48:59.254+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:48:59.334 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:48:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:48:59.003+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:48:59.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:48:59 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:48:59.334+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:48:59.784 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:48:59 smithi175 podman[111607]: 2024-07-29 00:48:59.502142925 +0000 UTC m=+5.766765347 container died c2cbfa08dc0eb5da7fa7d54577602e79d51bcb4752301021ecc482852feb1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:49:00.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:48:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:48:59.859+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:00.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:48:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:48:59.736+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:00.219 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:00 smithi175 podman[111607]: 2024-07-29 00:49:00.014513664 +0000 UTC m=+6.279136110 container cleanup c2cbfa08dc0eb5da7fa7d54577602e79d51bcb4752301021ecc482852feb1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, ceph=True, GIT_CLEAN=True) 2024-07-29T00:49:00.220 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:00 smithi175 bash[111607]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3 2024-07-29T00:49:00.220 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:00.003+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:00.335 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:00 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:00.039+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:00.534 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:00 smithi175 podman[111626]: 2024-07-29 00:49:00.218775848 +0000 UTC m=+0.703240982 container remove c2cbfa08dc0eb5da7fa7d54577602e79d51bcb4752301021ecc482852feb1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724) 2024-07-29T00:49:00.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:00 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:00.285+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:00.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:00 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:00.335+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:01.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:00 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:00.880+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:01.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:00 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:00.771+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:01.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:01 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:01.028+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:01.286 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:01 smithi175 podman[111717]: 2024-07-29 00:49:00.953786769 +0000 UTC m=+0.033477216 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:01.286 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:01 smithi175 podman[111717]: 2024-07-29 00:49:01.0779752 +0000 UTC m=+0.157665668 container create aab1436bc73c1a6894715ce4f64f118f1f16641f75e39a0a5645e2672bfadc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:49:01.502 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:01 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:01.038+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:01.502 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:01 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:01.287+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:01.685 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:01 smithi175 podman[111717]: 2024-07-29 00:49:01.373235428 +0000 UTC m=+0.452925983 container init aab1436bc73c1a6894715ce4f64f118f1f16641f75e39a0a5645e2672bfadc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:49:01.685 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:01 smithi175 podman[111717]: 2024-07-29 00:49:01.386039059 +0000 UTC m=+0.465729530 container start aab1436bc73c1a6894715ce4f64f118f1f16641f75e39a0a5645e2672bfadc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe) 2024-07-29T00:49:01.685 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:01 smithi175 podman[111717]: 2024-07-29 00:49:01.539055042 +0000 UTC m=+0.618745560 container attach aab1436bc73c1a6894715ce4f64f118f1f16641f75e39a0a5645e2672bfadc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-07-29T00:49:01.685 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:01 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:01.299+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:01.990 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:01 smithi175 podman[111740]: 2024-07-29 00:49:01.743956007 +0000 UTC m=+0.038860192 container died aab1436bc73c1a6894715ce4f64f118f1f16641f75e39a0a5645e2672bfadc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3-deactivate, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:49:02.068 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:01.870+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:02.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:01.804+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:02.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:01 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:01.990+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:02.326 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:02 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:02.033+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:02.557 INFO:journalctl@ceph.osd.3.smithi175.stdout:Jul 29 00:49:02 smithi175 podman[111740]: 2024-07-29 00:49:02.526284155 +0000 UTC m=+0.821188368 container remove aab1436bc73c1a6894715ce4f64f118f1f16641f75e39a0a5645e2672bfadc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:49:02.557 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:02 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:02.295+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:02.574 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.3.service' 2024-07-29T00:49:02.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:02.689 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-29T00:49:02.690 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-29T00:49:02.690 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.4 2024-07-29T00:49:02.717 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:02 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:02.326+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:02.915 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:02 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:02.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:02.999 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:02 smithi181 systemd[1]: Stopping Ceph osd.4 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:03.249 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:02 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:02.999+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:03.250 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:03 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:03.014+0000 7fbe14426640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:03.250 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:03 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:03.014+0000 7fbe14426640 -1 osd.4 107 *** Got signal Terminated *** 2024-07-29T00:49:03.250 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:03 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:03.014+0000 7fbe14426640 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:03.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:02 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:02.975+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:03.336 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:02 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:02.915+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:03.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:03 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:03.362+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:03.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:03.307+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:04.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:03.924+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:04.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:03.948+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:04.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:03.773+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:04.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:03.773+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:04.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:03.954+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:04.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:03 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:03.954+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:04.344 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:04 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:04.020+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:04.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:04 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:04.344+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:04.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:04 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:04.344+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:04.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:04 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:04.355+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:05.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:04.974+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:05.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:04.986+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:05.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:04.749+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:05.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:04.749+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:05.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:04 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:04.910+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:05.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:04 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:04.910+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:05.297 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:05 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:05.030+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:05.616 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:05 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:05.484+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:05.617 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:05 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:05.297+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:05.617 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:05 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:05.297+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:05.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:05.332+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:06.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:05.953+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:06.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:05.994+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:06.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:05.745+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:06.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:05.745+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:06.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:05.942+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:06.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:05 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:05.942+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:06.337 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:06 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:06.016+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:06.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:06 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:06.448+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:06.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:06 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:06.336+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:06.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:06 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:06.336+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:06.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:06 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:06.375+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:06.983 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:06.942+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:06.983 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:06.942+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:06.984 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:06.731+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:06.984 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:06.731+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:07.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:06 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:06.941+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:07.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:06 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:06.941+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:07.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:06.983+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:07.342 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:07 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:07.007+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:07.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:07 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:07.466+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:07.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:07 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:07.342+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:07.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:07 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:07.342+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:07.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:07.333+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:07.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:07.333+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:08.015 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:07 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:07.941+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:08.015 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:07 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:07.941+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:08.015 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:07 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:07.762+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:08.015 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:07 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:07.762+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:08.249 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:07 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4[47679]: 2024-07-29T00:49:07.980+0000 7fbe10a3c640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.306713+0000 front 2024-07-29T00:48:33.306939+0000 (oldest deadline 2024-07-29T00:48:58.006751+0000) 2024-07-29T00:49:08.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:07.979+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:08.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:07 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:07.979+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:08.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:08.014+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:08.750 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:08 smithi181 podman[73081]: 2024-07-29 00:49:08.546451232 +0000 UTC m=+5.659577816 container died c58385a8f539954b7a05d467c1077c5fed9efb4ba9ea8a521cdbf364df0efe8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-07-29T00:49:08.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:08.456+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:08.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:08.299+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:08.751 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:08 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:08.300+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:08.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:08.352+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:08.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:08.352+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:09.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:08.988+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:09.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:08.982+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:09.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:08.982+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:09.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:08.777+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:09.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:08 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:08.777+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:09.184 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:08 smithi181 podman[73081]: 2024-07-29 00:49:08.932491103 +0000 UTC m=+6.045617691 container cleanup c58385a8f539954b7a05d467c1077c5fed9efb4ba9ea8a521cdbf364df0efe8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-07-29T00:49:09.184 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:08 smithi181 bash[73081]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4 2024-07-29T00:49:09.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:08.948+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:09.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:08 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:08.948+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:09.454 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:09 smithi181 podman[73107]: 2024-07-29 00:49:09.183803499 +0000 UTC m=+0.635037475 container remove c58385a8f539954b7a05d467c1077c5fed9efb4ba9ea8a521cdbf364df0efe8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:49:09.455 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:09 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:09.347+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:09.455 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:09 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:09.348+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:09.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:09 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:09.454+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:09.750 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:09 smithi181 podman[73198]: 2024-07-29 00:49:09.515698012 +0000 UTC m=+0.017019866 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:09.750 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:09 smithi181 podman[73198]: 2024-07-29 00:49:09.642893922 +0000 UTC m=+0.144215767 container create 1ace793e9f8ce6e47e759c418c8b512436025d068d36bea833f05b3ae43372cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-07-29T00:49:09.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:09.323+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:09.785 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:09.323+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:10.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:10 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:10.002+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:10.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:09 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:09.815+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:10.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:09 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:09.815+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:10.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:09 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:09.976+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:10.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:09 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:09.976+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:10.093 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:09 smithi181 podman[73198]: 2024-07-29 00:49:09.904463827 +0000 UTC m=+0.405785676 container init 1ace793e9f8ce6e47e759c418c8b512436025d068d36bea833f05b3ae43372cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe) 2024-07-29T00:49:10.094 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:09 smithi181 podman[73198]: 2024-07-29 00:49:09.907934943 +0000 UTC m=+0.409256790 container start 1ace793e9f8ce6e47e759c418c8b512436025d068d36bea833f05b3ae43372cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-29T00:49:10.094 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:09 smithi181 podman[73198]: 2024-07-29 00:49:09.987300028 +0000 UTC m=+0.488621881 container attach 1ace793e9f8ce6e47e759c418c8b512436025d068d36bea833f05b3ae43372cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe) 2024-07-29T00:49:10.284 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:09.991+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:10.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:09.991+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:10.364 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jul 29 00:49:10 smithi181 podman[73198]: 2024-07-29 00:49:10.094419452 +0000 UTC m=+0.595741300 container died 1ace793e9f8ce6e47e759c418c8b512436025d068d36bea833f05b3ae43372cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-29T00:49:10.699 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:10 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:10.454+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:10.700 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:10 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:10.363+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:10.700 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:10 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:10.363+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:10.727 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.4.service' 2024-07-29T00:49:10.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:10 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:10.312+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:10.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:10 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:10.312+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:10.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:10.809 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-29T00:49:10.810 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-29T00:49:10.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.5 2024-07-29T00:49:11.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:11.051+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:11.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:10 smithi204 systemd[1]: Stopping Ceph osd.5 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:11.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.007+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:11.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.007+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:11.088 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:10 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:10.808+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:11.088 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:10 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:10.808+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:11.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:11 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:11.287+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:11.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:11 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:11.287+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:11.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:11 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:11.040+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:11.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:11 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:11.040+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:11.586 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.100+0000 7f1376531640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:11.586 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.100+0000 7f1376531640 -1 osd.5 107 *** Got signal Terminated *** 2024-07-29T00:49:11.586 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.100+0000 7f1376531640 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:11.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:11 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:11.476+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:11.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:11 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:11.343+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:11.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:11 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:11.343+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:12.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:12 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:12.042+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:12.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:11.793+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:12.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:11.793+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:12.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.959+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:12.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.959+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:12.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:11 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:11.959+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:50.377055+0000 front 2024-07-29T00:48:50.377081+0000 (oldest deadline 2024-07-29T00:49:11.377876+0000) 2024-07-29T00:49:12.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:12 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:12.056+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:12.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:12 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:12.056+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:12.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:12 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:12.239+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:12.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:12 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:12.239+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:12.536 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:12 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:12.239+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:12.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:12 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:12.448+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:12.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:12 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:12.448+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:12.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:12 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:12.295+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:12.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:12 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:12.295+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:13.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:13 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:13.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:13.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:12 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:12.995+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:13.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:12 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:12.995+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:13.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:12 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:12.995+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:50.377055+0000 front 2024-07-29T00:48:50.377081+0000 (oldest deadline 2024-07-29T00:49:11.377876+0000) 2024-07-29T00:49:13.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:12 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:12.783+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:13.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:12 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:12.783+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:13.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:13.069+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:13.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:13.069+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:13.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:13.268+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:13.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:13.268+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:13.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:13.268+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:13.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:13 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:13.430+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:13.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:13 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:13.430+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:13.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:13 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:13.274+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:13.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:13 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:13.274+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:14.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:13 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:13.764+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:14.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:13 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:13.764+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:14.027 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:13 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:13.973+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:14.027 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:13 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:13.973+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:14.027 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:13 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:13.973+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:50.377055+0000 front 2024-07-29T00:48:50.377081+0000 (oldest deadline 2024-07-29T00:49:11.377876+0000) 2024-07-29T00:49:14.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:14.026+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:14.499 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:14 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:14.389+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:14.500 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:14 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:14.389+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:14.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:14 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:14.238+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:14.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:14 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:14.238+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:14.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:14 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:14.238+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:14.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:14.270+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:14.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:14.270+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:14.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:14.270+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:14.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:14.071+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:14.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:14.071+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:14.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:14 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:14.071+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:15.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:14.757+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:15.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:14.757+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:15.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:14.981+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:15.086 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:14.981+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:15.087 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:14.981+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:50.377055+0000 front 2024-07-29T00:48:50.377081+0000 (oldest deadline 2024-07-29T00:49:11.377876+0000) 2024-07-29T00:49:15.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:14.986+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:15.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:14 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:14.986+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:15.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:15.266+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:15.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:15.266+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:15.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:15.266+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:15.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:15.085+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:15.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:15.085+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:15.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:15 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:15.085+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:15.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:15 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:15.412+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:15.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:15 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:15.412+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:15.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:15 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:15.265+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:15.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:15 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:15.265+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:15.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:15 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:15.265+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:16.020 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:15.729+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:16.020 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:15.729+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:16.020 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:15.729+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:16.020 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:15.952+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:32.175483+0000 front 2024-07-29T00:48:32.175495+0000 (oldest deadline 2024-07-29T00:48:58.075226+0000) 2024-07-29T00:49:16.021 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:15.952+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.176188+0000 front 2024-07-29T00:48:42.176247+0000 (oldest deadline 2024-07-29T00:49:06.876199+0000) 2024-07-29T00:49:16.021 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:15 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5[49026]: 2024-07-29T00:49:15.952+0000 7f136eb3f640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:50.377055+0000 front 2024-07-29T00:48:50.377081+0000 (oldest deadline 2024-07-29T00:49:11.377876+0000) 2024-07-29T00:49:16.309 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:16.037+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:16.309 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:16.037+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:16.309 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:16.037+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:16.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:16 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:16.020+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:16.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:16 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:16.020+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:16.449 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:16 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:16.230+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:16.450 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:16 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:16.230+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:16.450 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:16 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:16.230+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:16.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:16 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:16.449+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:16.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:16 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:16.449+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:16.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:16.308+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:16.785 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:16.308+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:16.785 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:16 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:16.308+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:16.942 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:16 smithi204 podman[74415]: 2024-07-29 00:49:16.659155224 +0000 UTC m=+5.657772620 container died ac54ffbaaabdb9af1ade82e55b7b1852d69b917db501101d1342dd2eced87481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS) 2024-07-29T00:49:16.942 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:16 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:16.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:16.943 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:16 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:16.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:16.943 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:16 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:16.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:17.210 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:16 smithi204 podman[74415]: 2024-07-29 00:49:16.942849292 +0000 UTC m=+5.941466685 container cleanup ac54ffbaaabdb9af1ade82e55b7b1852d69b917db501101d1342dd2eced87481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-29T00:49:17.211 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:16 smithi204 bash[74415]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5 2024-07-29T00:49:17.211 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:17.020+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:17.211 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:17.020+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:17.319 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:17.052+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:17.319 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:17.052+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:17.320 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:17.052+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:17.491 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:17 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:17.236+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:17.491 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:17 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:17.236+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:17.492 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:17 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:17.236+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:17.572 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:17 smithi204 podman[74432]: 2024-07-29 00:49:17.210898331 +0000 UTC m=+0.548563800 container remove ac54ffbaaabdb9af1ade82e55b7b1852d69b917db501101d1342dd2eced87481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-29T00:49:17.695 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:17.318+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:17.695 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:17.318+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:17.696 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:17 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:17.318+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:17.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:17 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:17.491+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:17.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:17 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:17.491+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:17.835 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:17 smithi204 podman[74518]: 2024-07-29 00:49:17.47523292 +0000 UTC m=+0.017180455 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:17.836 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:17 smithi204 podman[74518]: 2024-07-29 00:49:17.587270335 +0000 UTC m=+0.129217856 container create 4c3c6ba9f1bda90a2212c2f47ecee02ca1108ddef8287dd0767c2ead8f41cff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe) 2024-07-29T00:49:17.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:17.782+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:17.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:17.782+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:17.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:17 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:17.782+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:18.279 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:18.035+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:18.279 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:18.035+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:18.279 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:18.035+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:18.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:18.039+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:18.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:18.039+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:18.336 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:17 smithi204 podman[74518]: 2024-07-29 00:49:17.839765842 +0000 UTC m=+0.381713378 container init 4c3c6ba9f1bda90a2212c2f47ecee02ca1108ddef8287dd0767c2ead8f41cff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , ceph=True) 2024-07-29T00:49:18.336 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:17 smithi204 podman[74518]: 2024-07-29 00:49:17.842897229 +0000 UTC m=+0.384844756 container start 4c3c6ba9f1bda90a2212c2f47ecee02ca1108ddef8287dd0767c2ead8f41cff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5-deactivate, RELEASE=squid-ba82afe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD) 2024-07-29T00:49:18.336 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:17 smithi204 podman[74518]: 2024-07-29 00:49:17.923371737 +0000 UTC m=+0.465319277 container attach 4c3c6ba9f1bda90a2212c2f47ecee02ca1108ddef8287dd0767c2ead8f41cff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-29T00:49:18.336 INFO:journalctl@ceph.osd.5.smithi204.stdout:Jul 29 00:49:18 smithi204 podman[74518]: 2024-07-29 00:49:18.029273235 +0000 UTC m=+0.571220772 container died 4c3c6ba9f1bda90a2212c2f47ecee02ca1108ddef8287dd0767c2ead8f41cff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-07-29T00:49:18.489 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:18 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:18.235+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:18.489 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:18 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:18.235+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:18.489 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:18 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:18.235+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:18.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:18.277+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:18.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:18.277+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:18.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:18 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:18.277+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:18.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.5.service' 2024-07-29T00:49:18.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:18.726 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-29T00:49:18.726 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-07-29T00:49:18.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.6 2024-07-29T00:49:18.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:18 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:18.489+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:18.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:18 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:18.489+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:18.995 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:18.995+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:18.995 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:18.995+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:18.995 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:18.815+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:18.995 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:18.815+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:18.996 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:18 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:18.815+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:19.267 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:18 smithi175 systemd[1]: Stopping Ceph osd.6 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:19.268 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:19.019+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:19.268 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:19.019+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:19.268 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:19.019+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:19.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:19 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:19.210+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:19.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:19 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:19.210+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:19.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:19 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:19.210+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:19.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:19.266+0000 7f9363c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:19.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:19.266+0000 7f9363c00640 -1 osd.6 107 *** Got signal Terminated *** 2024-07-29T00:49:19.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:19.266+0000 7f9363c00640 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:19.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:19.319+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:19.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:19.319+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:19.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:19.319+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:20.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:19 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:19.501+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:20.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:19 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:19.501+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:20.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:19 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:19.812+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:20.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:19 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:19.812+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:20.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:19 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:19.812+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:20.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:19 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:19.954+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:20.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:19 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:19.954+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:20.319 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:20.053+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:20.319 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:20.053+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:20.319 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:20.053+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:20.484 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:20.170+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:20.484 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:20.170+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:20.484 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:20.170+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:20.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:20.484+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:20.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:20.484+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:20.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:20 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:20.484+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:20.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:20.318+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:20.784 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:20.318+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:20.785 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:20 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:20.318+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:21.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:20 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:20.959+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:21.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:20 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:20.959+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:21.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:20 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:20.959+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:21.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:20 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:20.819+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:21.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:20 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:20.819+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:21.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:20 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:20.819+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:21.451 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:21 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:21.134+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:21.451 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:21 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:21.134+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:21.452 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:21 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:21.134+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:21.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:21.278+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:21.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:21.278+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:21.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:21.278+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:21.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:21.093+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:21.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:21.093+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:21.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:21.093+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:21.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:21 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:21.093+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:21.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:21 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:21.451+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:21.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:21 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:21.451+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:21.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:21 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:21.451+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:22.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:21.814+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:22.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:21.814+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:22.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:21.814+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:22.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:21.940+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:22.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:21.940+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:22.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:21 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:21.940+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:22.493 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:22.123+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:22.493 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:22.123+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:22.493 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:22.123+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:22.493 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:22.123+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:22.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:22.252+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:22.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:22.252+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:22.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:22.252+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:22.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:22.252+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:58.166311+0000 front 2024-07-29T00:48:58.166359+0000 (oldest deadline 2024-07-29T00:49:21.666409+0000) 2024-07-29T00:49:22.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:22.120+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:22.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:22.120+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:22.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:22.120+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:22.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:22.120+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:22.749 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:22.492+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:22.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:22.492+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:22.750 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:22 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:22.492+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:23.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:22 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:22.915+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:23.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:22 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:22.915+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:23.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:22 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:22.915+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:23.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:22 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:22.784+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:23.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:22 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:22.784+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:23.087 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:22 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:22.784+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:23.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:23.119+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:23.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:23.119+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:23.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:23.119+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:23.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:23.119+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:23.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:23.271+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:33.060445+0000 front 2024-07-29T00:48:33.059863+0000 (oldest deadline 2024-07-29T00:48:58.359571+0000) 2024-07-29T00:49:23.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:23.271+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.261710+0000 front 2024-07-29T00:48:41.262243+0000 (oldest deadline 2024-07-29T00:49:06.561862+0000) 2024-07-29T00:49:23.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:23.271+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.863453+0000 front 2024-07-29T00:48:48.863257+0000 (oldest deadline 2024-07-29T00:49:11.763365+0000) 2024-07-29T00:49:23.535 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6[59796]: 2024-07-29T00:49:23.271+0000 7f935f600640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:58.166311+0000 front 2024-07-29T00:48:58.166359+0000 (oldest deadline 2024-07-29T00:49:21.666409+0000) 2024-07-29T00:49:23.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:23.105+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:23.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:23.105+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:23.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:23.105+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:23.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:23 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:23.105+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:23.999 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:23.509+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:24.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:23.509+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:24.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:23 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:23.509+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:24.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:23.749+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:24.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:23.749+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:24.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:23.749+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:24.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:23.935+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:24.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:23.935+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:24.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:23 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:23.935+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:24.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:24.117+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:24.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:24.117+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:24.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:24.117+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:24.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:24.117+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:24.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:24.097+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:24.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:24.097+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:24.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:24.097+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:24.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:24 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:24.097+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:24.899 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:24 smithi175 podman[111833]: 2024-07-29 00:49:24.628185588 +0000 UTC m=+5.506782216 container died 249bc4729b24a8eed303e28378b7c56c06283616e40b571716bda706393f8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:49:24.999 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:24.516+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:24.999 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:24.516+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:25.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:24 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:24.516+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:25.085 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:24.797+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:25.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:24.797+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:25.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:24.797+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:25.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:24.961+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:25.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:24.961+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:25.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:24 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:24.961+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:25.154 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:25 smithi175 podman[111833]: 2024-07-29 00:49:25.135707281 +0000 UTC m=+6.014303949 container cleanup 249bc4729b24a8eed303e28378b7c56c06283616e40b571716bda706393f8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_CLEAN=True, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-29T00:49:25.154 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:25 smithi175 bash[111833]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6 2024-07-29T00:49:25.155 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:25.096+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:25.155 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:25.096+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:25.155 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:25.096+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:25.155 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:25 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:25.096+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:25.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:25.091+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:25.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:25.091+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:25.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:25.091+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:25.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:25.091+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:25.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:25 smithi175 podman[111850]: 2024-07-29 00:49:25.370658461 +0000 UTC m=+0.729049218 container remove 249bc4729b24a8eed303e28378b7c56c06283616e40b571716bda706393f8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-07-29T00:49:26.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:25.548+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:26.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:25.548+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:26.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:25 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:25.548+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:26.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:25.838+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:26.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:25.838+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:26.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:25.838+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:26.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:25.977+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:26.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:25.977+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:26.337 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:25 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:25.977+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:26.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:26.132+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:26.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:26.132+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:26.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:26.132+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:26.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:26.132+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:26.500 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:26.127+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:26.501 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:26.127+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:26.501 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:26.127+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:26.501 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:26 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:26.127+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:26.501 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:26 smithi175 podman[111938]: 2024-07-29 00:49:26.149855481 +0000 UTC m=+0.030563484 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:26.501 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:26 smithi175 podman[111938]: 2024-07-29 00:49:26.329990868 +0000 UTC m=+0.210698891 container create bc07d3e18995d2e4450bfc16b16394e14ad785eef99759cb1347f06b5eaa3fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6-deactivate, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-29T00:49:26.785 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:26 smithi175 podman[111938]: 2024-07-29 00:49:26.633606898 +0000 UTC m=+0.514314947 container init bc07d3e18995d2e4450bfc16b16394e14ad785eef99759cb1347f06b5eaa3fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-29T00:49:26.785 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:26 smithi175 podman[111938]: 2024-07-29 00:49:26.645235355 +0000 UTC m=+0.525943398 container start bc07d3e18995d2e4450bfc16b16394e14ad785eef99759cb1347f06b5eaa3fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6-deactivate, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-29T00:49:26.786 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:26 smithi175 podman[111938]: 2024-07-29 00:49:26.73229559 +0000 UTC m=+0.613003676 container attach bc07d3e18995d2e4450bfc16b16394e14ad785eef99759cb1347f06b5eaa3fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:49:26.999 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:26.592+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:27.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:26.592+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:27.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:26.592+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:27.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:26.798+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:27.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:26.798+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:27.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:26.798+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:27.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:26.990+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:27.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:26.990+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:27.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:26 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:26.990+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:27.285 INFO:journalctl@ceph.osd.6.smithi175.stdout:Jul 29 00:49:26 smithi175 podman[111938]: 2024-07-29 00:49:26.944199771 +0000 UTC m=+0.824907854 container died bc07d3e18995d2e4450bfc16b16394e14ad785eef99759cb1347f06b5eaa3fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-29T00:49:27.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:27.156+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:27.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:27.156+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:27.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:27.156+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:27.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:27 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:27.156+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:27.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:27.108+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:27.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:27.108+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:27.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:27.108+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:27.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:27.108+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:27.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.6.service' 2024-07-29T00:49:27.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:27.826 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-07-29T00:49:27.827 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-07-29T00:49:27.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.7 2024-07-29T00:49:27.852 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:27.546+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:27.852 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:27.546+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:27.853 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:27 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:27.546+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:28.038 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:27.775+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:28.038 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:27.775+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:28.038 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:27 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:27.775+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:28.143 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:27 smithi181 systemd[1]: Stopping Ceph osd.7 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:28.143 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:28.088+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:28.143 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:28.088+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:28.143 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:28.088+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:28.143 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:28.088+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:28.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:28.038+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:28.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:28.038+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:28.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:28.038+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:28.380 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:28.117+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:28.380 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:28.117+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:28.380 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:28.117+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:28.380 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:28 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:28.117+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:28.499 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:28.142+0000 7f54f4fcc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:28.500 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:28.142+0000 7f54f4fcc640 -1 osd.7 107 *** Got signal Terminated *** 2024-07-29T00:49:28.500 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:28.142+0000 7f54f4fcc640 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:29.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:28.545+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:29.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:28.545+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:29.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:28 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:28.545+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:29.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:28.796+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:29.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:28.796+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:29.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:28 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:28.796+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:29.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:29 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:29.040+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:29.086 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:29 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:29.040+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:29.087 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:29 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:29.040+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:29.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:29.126+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:29.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:29.126+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:29.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:29.126+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:29.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:29.126+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:29.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:29 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:29.126+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:29.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:29.052+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:29.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:29.052+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:29.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:29.052+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:29.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:29.052+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:29.999 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:29.553+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:30.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:29.553+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:30.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:29.553+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:30.037 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:29 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:29.786+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:30.037 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:29 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:29.786+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:30.037 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:29 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:29.786+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:30.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:30.037+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:30.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:30.037+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:30.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:30.037+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:30.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:30.037+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:30.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:30.094+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:30.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:30.094+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:30.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:30.094+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:30.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:30.094+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:30.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:30.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:30.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:30.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:30.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:30.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:30.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:30.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:30.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:30 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:30.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:30.999 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:30.575+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:31.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:30.575+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:31.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:30 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:30.575+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:31.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:30.785+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:31.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:30.785+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:31.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:30 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:30.785+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:31.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:31.068+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:31.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:31.068+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:31.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:31.068+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:31.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:31.068+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:31.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:31.086+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:31.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:31.086+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:31.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:31.086+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:31.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:31.086+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:31.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:31.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:31.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:31.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:31.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:31.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:31.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:31.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:31.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:31 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:31.144+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:32.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:31.600+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:32.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:31.600+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:32.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:31 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:31.600+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:32.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:31.737+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:32.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:31.737+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:32.063 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:31 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:31.737+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:32.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:32.062+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:32.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:32.062+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:32.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:32.062+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:32.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:32.062+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:32.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:32.082+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:32.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:32.082+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:32.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:32.082+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:32.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:32.082+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:32.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:32.082+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:32.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:32 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:32.150+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:32.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:32 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:32.150+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:32.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:32 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:32.150+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:32.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:32 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:32.150+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:32.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:32 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:32.150+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:33.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:32.571+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.814245+0000 front 2024-07-29T00:48:41.814221+0000 (oldest deadline 2024-07-29T00:49:05.314316+0000) 2024-07-29T00:49:33.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:32.571+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.716008+0000 front 2024-07-29T00:48:48.715880+0000 (oldest deadline 2024-07-29T00:49:11.615620+0000) 2024-07-29T00:49:33.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:32.571+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:55.116869+0000 front 2024-07-29T00:48:55.116840+0000 (oldest deadline 2024-07-29T00:49:19.816593+0000) 2024-07-29T00:49:33.000 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:32 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7[54579]: 2024-07-29T00:49:32.571+0000 7f54f15e2640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:05.717454+0000 front 2024-07-29T00:49:05.717475+0000 (oldest deadline 2024-07-29T00:49:31.617622+0000) 2024-07-29T00:49:33.030 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:32.705+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:33.030 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:32.705+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:33.031 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:32.705+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:33.031 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:32 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:32.705+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:33.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:33.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:33.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:33.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:33.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:33.042+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:33.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:33.042+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:33.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:33.042+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:33.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:33.042+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:33.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:33.042+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:33.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:33.145+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:33.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:33.145+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:33.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:33.145+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:33.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:33.145+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:33.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:33 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:33.145+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:33.860 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:33 smithi181 podman[73302]: 2024-07-29 00:49:33.565662389 +0000 UTC m=+5.541357611 container died dd035fa9de7de9b33462c631fbcace3d8a1d8131df8d0c6fe304079802509688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-ba82afe) 2024-07-29T00:49:33.995 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:33.697+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:33.995 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:33.697+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:33.995 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:33.697+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:33.995 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:33.697+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:34.227 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:33 smithi181 podman[73302]: 2024-07-29 00:49:33.984891784 +0000 UTC m=+5.960587005 container cleanup dd035fa9de7de9b33462c631fbcace3d8a1d8131df8d0c6fe304079802509688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-07-29T00:49:34.228 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:33 smithi181 bash[73302]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7 2024-07-29T00:49:34.228 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:34 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:34.083+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:34.228 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:34 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:34.083+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:34.228 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:34 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:34.083+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:34.228 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:34 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:34.083+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:34.228 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:34 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:34.083+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:34.263 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.995+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:34.263 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.995+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:34.263 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.995+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:34.263 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:33 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:33.995+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:34.499 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:34 smithi181 podman[73324]: 2024-07-29 00:49:34.227983607 +0000 UTC m=+0.654516732 container remove dd035fa9de7de9b33462c631fbcace3d8a1d8131df8d0c6fe304079802509688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-07-29T00:49:34.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:34 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:34.182+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:34.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:34 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:34.182+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:34.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:34 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:34.182+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:34.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:34 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:34.182+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:34.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:34 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:34.182+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:34.973 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:34 smithi181 podman[73410]: 2024-07-29 00:49:34.559441414 +0000 UTC m=+0.017264555 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:34.973 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:34 smithi181 podman[73410]: 2024-07-29 00:49:34.68720206 +0000 UTC m=+0.145025201 container create 057980d577f29f4c7765f64c0768f96ebdecc2ff181c2dd4e5e36530c8578d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-ba82afe) 2024-07-29T00:49:34.973 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:34 smithi181 podman[73410]: 2024-07-29 00:49:34.973289359 +0000 UTC m=+0.431112508 container init 057980d577f29f4c7765f64c0768f96ebdecc2ff181c2dd4e5e36530c8578d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:49:35.014 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:34 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:34.721+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:35.015 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:34 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:34.721+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:35.015 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:34 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:34.721+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:35.015 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:34 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:34.721+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:35.249 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:34 smithi181 podman[73410]: 2024-07-29 00:49:34.976711631 +0000 UTC m=+0.434534772 container start 057980d577f29f4c7765f64c0768f96ebdecc2ff181c2dd4e5e36530c8578d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe) 2024-07-29T00:49:35.250 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:35 smithi181 podman[73410]: 2024-07-29 00:49:35.081411046 +0000 UTC m=+0.539234192 container attach 057980d577f29f4c7765f64c0768f96ebdecc2ff181c2dd4e5e36530c8578d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe) 2024-07-29T00:49:35.250 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jul 29 00:49:35 smithi181 podman[73410]: 2024-07-29 00:49:35.166543686 +0000 UTC m=+0.624366832 container died 057980d577f29f4c7765f64c0768f96ebdecc2ff181c2dd4e5e36530c8578d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, GIT_CLEAN=True, ceph=True) 2024-07-29T00:49:35.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:35 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:35.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:35.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:35 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:35.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:35.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:35 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:35.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:35.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:35 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:35.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:35.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:35 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:35.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:35.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:35.014+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:35.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:35.014+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:35.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:35.014+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:35.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:35.014+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:35.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:35 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:35.186+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:35.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:35 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:35.186+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:35.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:35 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:35.186+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:35.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:35 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:35.186+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:35.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:35 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:35.186+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:35.870 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.7.service' 2024-07-29T00:49:35.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:35.949 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-07-29T00:49:35.949 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-07-29T00:49:35.949 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.8 2024-07-29T00:49:35.976 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:35.692+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:35.976 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:35.692+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:35.976 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:35.692+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:35.976 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:35 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:35.692+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:36.263 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 systemd[1]: Stopping Ceph osd.8 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.263+0000 7f4855ceb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.263+0000 7f4855ceb640 -1 osd.8 107 *** Got signal Terminated *** 2024-07-29T00:49:36.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:36.263+0000 7f4855ceb640 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:36.377 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:36 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:36.120+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:36.378 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:36 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:36.120+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:36.378 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:36 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:36.120+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:36.378 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:36 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:36.120+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:36.378 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:36 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:36.120+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:36.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:36 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:36.170+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:36.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:36 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:36.170+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:36.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:36 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:36.170+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:36.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:36 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:36.170+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:36.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:36 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:36.170+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:37.004 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:36.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:37.004 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:36.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:37.004 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:36.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:37.004 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:36 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:36.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:37.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.004+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:37.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.004+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:37.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.004+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:37.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.004+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:37.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:37.075+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:37.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:37.075+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:37.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:37.075+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:37.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:37.075+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:37.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:37.075+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:37.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:37.075+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:37.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:37 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:37.184+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:37.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:37 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:37.184+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:37.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:37 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:37.184+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:37.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:37 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:37.184+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:37.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:37 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:37.184+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:37.982 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:37.711+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:37.983 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:37.711+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:37.983 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:37.711+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:37.983 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:37.711+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:38.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.982+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:38.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.982+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:38.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.982+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:38.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.982+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:38.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:37 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:37.982+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.758038+0000 front 2024-07-29T00:49:15.758098+0000 (oldest deadline 2024-07-29T00:49:37.958843+0000) 2024-07-29T00:49:38.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:38 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:38.080+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:38.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:38 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:38.080+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:38.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:38 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:38.080+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:38.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:38 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:38.080+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:38.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:38 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:38.080+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:38.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:38 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:38.080+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:38.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:38 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:38.138+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:38.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:38 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:38.138+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:38.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:38 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:38.138+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:38.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:38 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:38.138+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:38.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:38 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:38.138+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:38.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:38 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:38.138+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:39.030 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:38 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:38.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:39.031 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:38 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:38.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:39.031 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:38 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:38.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:39.031 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:38 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:38.738+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:39.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:39.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:39.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:39.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:39.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:39.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:39.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:39.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:39.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:39.030+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.758038+0000 front 2024-07-29T00:49:15.758098+0000 (oldest deadline 2024-07-29T00:49:37.958843+0000) 2024-07-29T00:49:39.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:39.076+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:39.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:39.076+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:39.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:39.076+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:39.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:39.076+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:39.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:39.076+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:39.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:39 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:39.076+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:39.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:39.159+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:39.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:39.159+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:39.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:39.159+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:39.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:39.159+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:39.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:39.159+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:39.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:39 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:39.159+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:39.709+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:39.709+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:39.709+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:40.026 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:39 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:39.709+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:40.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:40.026+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:40.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:40.026+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:40.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:40.026+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:40.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:40.026+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:40.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:40.026+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.758038+0000 front 2024-07-29T00:49:15.758098+0000 (oldest deadline 2024-07-29T00:49:37.958843+0000) 2024-07-29T00:49:40.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:40 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:40.065+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:40.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:40 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:40.065+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:40.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:40 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:40.065+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:40.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:40 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:40.065+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:40.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:40 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:40.065+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:40.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:40 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:40.065+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:40.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:40 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:40.136+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:40.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:40 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:40.136+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:40.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:40 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:40.136+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:40.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:40 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:40.136+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:40.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:40 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:40.136+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:40.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:40 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:40.136+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:41.057 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:40.734+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:41.057 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:40.734+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:41.058 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:40.734+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:41.058 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:40.734+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:41.058 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:40 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:40.734+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:41.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:41.057+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:42.052191+0000 front 2024-07-29T00:48:42.052331+0000 (oldest deadline 2024-07-29T00:49:03.752299+0000) 2024-07-29T00:49:41.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:41.057+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:48.953831+0000 front 2024-07-29T00:48:48.953974+0000 (oldest deadline 2024-07-29T00:49:14.154626+0000) 2024-07-29T00:49:41.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:41.057+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.655995+0000 front 2024-07-29T00:48:54.655703+0000 (oldest deadline 2024-07-29T00:49:20.555474+0000) 2024-07-29T00:49:41.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:41.057+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:04.557273+0000 front 2024-07-29T00:49:04.557211+0000 (oldest deadline 2024-07-29T00:49:29.857299+0000) 2024-07-29T00:49:41.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8[55515]: 2024-07-29T00:49:41.057+0000 7f484e2f9640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.758038+0000 front 2024-07-29T00:49:15.758098+0000 (oldest deadline 2024-07-29T00:49:37.958843+0000) 2024-07-29T00:49:41.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.046+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:41.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.046+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:41.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.046+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:41.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.046+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:41.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.046+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:41.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.046+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:41.534 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:41 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:41.140+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:41.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:41 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:41.140+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:41.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:41 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:41.140+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:41.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:41 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:41.140+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:41.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:41 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:41.140+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:41.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:41 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:41.140+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:41.938 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:41 smithi204 podman[74631]: 2024-07-29 00:49:41.634565912 +0000 UTC m=+5.486631026 container died db03915296790bd764ca4a271e8507544c92f64b8ba952cfc57de168740b9db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-07-29T00:49:41.939 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:41.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:41.939 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:41.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:41.939 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:41.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:41.939 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:41.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:41.939 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:41.772+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:42.239 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 podman[74631]: 2024-07-29 00:49:42.04682462 +0000 UTC m=+5.898889724 container cleanup db03915296790bd764ca4a271e8507544c92f64b8ba952cfc57de168740b9db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-29T00:49:42.239 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 bash[74631]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8 2024-07-29T00:49:42.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.998+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:42.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.998+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:42.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.998+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:42.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.998+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:42.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.998+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:42.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:41 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:41.998+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:42.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:42 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:42.147+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:42.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:42 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:42.147+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:42.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:42 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:42.147+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:42.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:42 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:42.147+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:42.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:42 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:42.147+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:42.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:42 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:42.147+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:42.585 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 podman[74646]: 2024-07-29 00:49:42.239787645 +0000 UTC m=+0.605937640 container remove db03915296790bd764ca4a271e8507544c92f64b8ba952cfc57de168740b9db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-07-29T00:49:42.960 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 podman[74739]: 2024-07-29 00:49:42.565992555 +0000 UTC m=+0.017168759 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:42.960 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 podman[74739]: 2024-07-29 00:49:42.719180716 +0000 UTC m=+0.170356905 container create 8fb0996f318872f58eb1fb2203fdd9c30745bd7ddf1ab02c295bccc26445b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-07-29T00:49:42.960 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:42 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:42.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:42.961 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:42 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:42.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:42.961 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:42 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:42.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:42.961 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:42 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:42.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:42.961 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:42 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:42.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:43.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:42.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:43.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:42.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:43.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:42.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:43.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:42.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:43.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:42.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:43.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:42 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:42.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:43.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 podman[74739]: 2024-07-29 00:49:42.960430623 +0000 UTC m=+0.411606813 container init 8fb0996f318872f58eb1fb2203fdd9c30745bd7ddf1ab02c295bccc26445b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:49:43.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:42 smithi204 podman[74739]: 2024-07-29 00:49:42.963952068 +0000 UTC m=+0.415128257 container start 8fb0996f318872f58eb1fb2203fdd9c30745bd7ddf1ab02c295bccc26445b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8-deactivate, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-29T00:49:43.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:43 smithi204 podman[74739]: 2024-07-29 00:49:43.027013028 +0000 UTC m=+0.478189219 container attach 8fb0996f318872f58eb1fb2203fdd9c30745bd7ddf1ab02c295bccc26445b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724) 2024-07-29T00:49:43.336 INFO:journalctl@ceph.osd.8.smithi204.stdout:Jul 29 00:49:43 smithi204 podman[74739]: 2024-07-29 00:49:43.154299466 +0000 UTC m=+0.605475656 container died 8fb0996f318872f58eb1fb2203fdd9c30745bd7ddf1ab02c295bccc26445b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:49:43.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:43.106+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:43.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:43.106+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:43.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:43.106+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:43.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:43.106+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:43.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:43.106+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:43.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:43 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:43.106+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:43.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.8.service' 2024-07-29T00:49:43.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:43.913 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-07-29T00:49:43.913 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-07-29T00:49:43.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.9 2024-07-29T00:49:44.085 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:43 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:43.861+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:44.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:43 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:43.861+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:44.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:43 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:43.861+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:44.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:43 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:43.861+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:44.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:43 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:43.861+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:44.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:43 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:43.987+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:44.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:43 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:43.987+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:44.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:43 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:43.987+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:44.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:43 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:43.987+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:44.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:43 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:43.987+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:44.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:43 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:43.987+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 systemd[1]: Stopping Ceph osd.9 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.155+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.155+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.155+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.155+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.155+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:44.285 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.155+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:44.785 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.468+0000 7f1ddfc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:44.785 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.468+0000 7f1ddfc00640 -1 osd.9 107 *** Got signal Terminated *** 2024-07-29T00:49:44.785 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:44 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:44.468+0000 7f1ddfc00640 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:45.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:44 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:44.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:45.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:44 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:44.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:45.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:44 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:44.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:45.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:44 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:44.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:45.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:44 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:44.822+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:45.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:44 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:44.982+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:45.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:44 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:44.982+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:45.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:44 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:44.982+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:45.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:44 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:44.982+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:45.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:44 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:44.982+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:45.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:44 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:44.982+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:45.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:45.135+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:45.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:45.135+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:45.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:45.135+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:45.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:45.135+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:45.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:45.135+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:45.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:45 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:45.135+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:46.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:46.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:46.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:46.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:46.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:46.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:46.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:45 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:45.939+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:46.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:45.837+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:46.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:45.837+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:46.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:45.837+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:46.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:45.837+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:46.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:45.837+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:46.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:46.103+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:46.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:46.103+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:46.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:46.103+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:46.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:46.103+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:46.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:46.103+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:46.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:46 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:46.103+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:47.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:47.263 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:47.263 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:47.263 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:47.263 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:47.263 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:47.263 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:46 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:46.892+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:47.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:46 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:46.850+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:47.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:46 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:46.850+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:47.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:46 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:46.850+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:47.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:46 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:46.850+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:47.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:46 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:46.850+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:47.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:47.090+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:47.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:47.090+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:47.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:47.090+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:47.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:47.090+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:47.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:47.090+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:47.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:47 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:47.090+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:48.234 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:48.110+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:48.234 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:48.110+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:48.234 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:48.110+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:48.234 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:48.110+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:48.234 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:48.110+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:48.235 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:48 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:48.110+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:48.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:48.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:48.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:48.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:48.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:48.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:48.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:47 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:47.902+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:48.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:47 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:47.882+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:48.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:47 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:47.882+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:48.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:47 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:47.882+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:48.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:47 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:47.882+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:48.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:47 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:47.882+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:49.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:48 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:48.832+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:49.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:48 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:48.832+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:49.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:48 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:48.832+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:49.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:48 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:48.832+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:49.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:48 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:48.832+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:49.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:49.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:49.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:49.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:49.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:49.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:49.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:48 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:48.861+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:49.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:27.683283+0000 front 2024-07-29T00:48:27.682765+0000 (oldest deadline 2024-07-29T00:48:54.683615+0000) 2024-07-29T00:49:49.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:41.586715+0000 front 2024-07-29T00:48:41.586604+0000 (oldest deadline 2024-07-29T00:49:03.787601+0000) 2024-07-29T00:49:49.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.590271+0000 front 2024-07-29T00:48:49.590144+0000 (oldest deadline 2024-07-29T00:49:13.090185+0000) 2024-07-29T00:49:49.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:54.792462+0000 front 2024-07-29T00:48:54.791990+0000 (oldest deadline 2024-07-29T00:49:20.592324+0000) 2024-07-29T00:49:49.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.496279+0000 front 2024-07-29T00:49:07.496346+0000 (oldest deadline 2024-07-29T00:49:28.596253+0000) 2024-07-29T00:49:49.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:12.097981+0000 front 2024-07-29T00:49:12.098040+0000 (oldest deadline 2024-07-29T00:49:37.998140+0000) 2024-07-29T00:49:49.536 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9[66325]: 2024-07-29T00:49:49.116+0000 7f1ddb600640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.800959+0000 front 2024-07-29T00:49:23.801000+0000 (oldest deadline 2024-07-29T00:49:48.500748+0000) 2024-07-29T00:49:50.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:50.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:50.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:50.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:50.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:50.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:50.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:49 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:49.907+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:50.267 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:49 smithi175 podman[112039]: 2024-07-29 00:49:49.997796226 +0000 UTC m=+5.666662728 container died 6f1cd63a744d042e92bb3f8fca6cddac82e412f6c6a7644ea324bc2860599436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS) 2024-07-29T00:49:50.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:49.871+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:50.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:49.871+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:50.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:49.871+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:50.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:49.871+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:50.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:49.871+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:50.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:49.871+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:50.519 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:50 smithi175 podman[112039]: 2024-07-29 00:49:50.494447368 +0000 UTC m=+6.163313897 container cleanup 6f1cd63a744d042e92bb3f8fca6cddac82e412f6c6a7644ea324bc2860599436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe) 2024-07-29T00:49:50.519 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:50 smithi175 bash[112039]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9 2024-07-29T00:49:51.034 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:50 smithi175 podman[112068]: 2024-07-29 00:49:50.712589989 +0000 UTC m=+0.701526399 container remove 6f1cd63a744d042e92bb3f8fca6cddac82e412f6c6a7644ea324bc2860599436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2) 2024-07-29T00:49:51.085 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:50 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:50.823+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:51.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:50 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:50.823+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:51.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:50 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:50.823+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:51.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:50 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:50.823+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:51.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:50 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:50.823+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:51.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:50 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:50.823+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:51.249 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:51.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:51.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:51.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:51.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:51.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:51.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:50 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:50.935+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:51.875 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:51 smithi175 podman[112158]: 2024-07-29 00:49:51.475817514 +0000 UTC m=+0.036990513 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:49:51.875 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:51 smithi175 podman[112158]: 2024-07-29 00:49:51.61335344 +0000 UTC m=+0.174526430 container create 82fe5ec98e501091b5b5130208cb5931f4c7571697874f4b62aa230c4e21c132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-07-29T00:49:51.876 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:51 smithi175 podman[112158]: 2024-07-29 00:49:51.875043693 +0000 UTC m=+0.436216740 container init 82fe5ec98e501091b5b5130208cb5931f4c7571697874f4b62aa230c4e21c132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9-deactivate, RELEASE=squid-ba82afe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-29T00:49:52.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:51 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:51.810+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:52.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:51 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:51.810+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:52.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:51 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:51.810+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:52.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:51 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:51.810+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:52.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:51 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:51.810+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:52.086 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:51 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:51.810+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:52.178 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:51 smithi175 podman[112158]: 2024-07-29 00:49:51.885802699 +0000 UTC m=+0.446975733 container start 82fe5ec98e501091b5b5130208cb5931f4c7571697874f4b62aa230c4e21c132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-29T00:49:52.178 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:51 smithi175 podman[112158]: 2024-07-29 00:49:51.98249565 +0000 UTC m=+0.543668657 container attach 82fe5ec98e501091b5b5130208cb5931f4c7571697874f4b62aa230c4e21c132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-29T00:49:52.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:52.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:52.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:52.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:52.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:52.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:52.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:51 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:51.949+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:52.535 INFO:journalctl@ceph.osd.9.smithi175.stdout:Jul 29 00:49:52 smithi175 podman[112158]: 2024-07-29 00:49:52.178856916 +0000 UTC m=+0.740029952 container died 82fe5ec98e501091b5b5130208cb5931f4c7571697874f4b62aa230c4e21c132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-29T00:49:53.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.9.service' 2024-07-29T00:49:53.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:49:53.171 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-07-29T00:49:53.171 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-07-29T00:49:53.172 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.10 2024-07-29T00:49:53.250 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:53.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:53.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:53.251 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:53.252 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:53.252 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:53.252 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:52 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:52.996+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:53.253 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:53 smithi181 systemd[1]: Stopping Ceph osd.10 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:49:53.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:52.845+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:53.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:52.845+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:53.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:52.845+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:53.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:52.845+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:53.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:52.845+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:53.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:52 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:52.845+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:53.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:53 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:53.453+0000 7fa0cfa49640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:49:53.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:53 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:53.453+0000 7fa0cfa49640 -1 osd.10 107 *** Got signal Terminated *** 2024-07-29T00:49:53.750 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:53 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:53.453+0000 7fa0cfa49640 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:49:54.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:54.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:54.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:54.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:54.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:54.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:54.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:53.878+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:49:54.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:54.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:54.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:54.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:54.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:54.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:54.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:54 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:54.040+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:55.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:55.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:55.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:55.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:55.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:55.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:55.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:54 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:54.877+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:49:55.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:55.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:55.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:55.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:55.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:55.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:55.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:55 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:55.032+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:56.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:56.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:56.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:56.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:56.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:56.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:56.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:55 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:55.859+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:49:56.499 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:56.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:56.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:56.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:56.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:56.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:56.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:56 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:56.064+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:57.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:56 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:56.890+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:49:57.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:57.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:57.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:57.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:57.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:57.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:57.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:57 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:57.081+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:58.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:58.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:58.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:58.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:58.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:58.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:58.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:57 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:57.935+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:49:58.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:29.631426+0000 front 2024-07-29T00:48:29.631586+0000 (oldest deadline 2024-07-29T00:48:54.931161+0000) 2024-07-29T00:49:58.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:40.831835+0000 front 2024-07-29T00:48:40.831920+0000 (oldest deadline 2024-07-29T00:49:03.731958+0000) 2024-07-29T00:49:58.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.032762+0000 front 2024-07-29T00:48:49.032738+0000 (oldest deadline 2024-07-29T00:49:13.732680+0000) 2024-07-29T00:49:58.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6814 osd.3 since back 2024-07-29T00:48:57.233963+0000 front 2024-07-29T00:48:57.234117+0000 (oldest deadline 2024-07-29T00:49:21.933752+0000) 2024-07-29T00:49:58.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:07.134928+0000 front 2024-07-29T00:49:07.134989+0000 (oldest deadline 2024-07-29T00:49:31.835071+0000) 2024-07-29T00:49:58.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:15.336376+0000 front 2024-07-29T00:49:15.336432+0000 (oldest deadline 2024-07-29T00:49:37.036186+0000) 2024-07-29T00:49:58.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10[61096]: 2024-07-29T00:49:58.090+0000 7fa0cc05f640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:20.537408+0000 front 2024-07-29T00:49:20.537271+0000 (oldest deadline 2024-07-29T00:49:45.837043+0000) 2024-07-29T00:49:59.079 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:58 smithi181 podman[73524]: 2024-07-29 00:49:58.783771878 +0000 UTC m=+5.414579890 container died 705d762df3840d410cc37e36638c27fcb616fed0ad3f77c6b3d1435eb0e4a9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:49:59.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:49:59.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:49:59.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:49:59.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:49:59.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:49:59.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:49:59.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:58 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:58.949+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:49:59.412 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:59 smithi181 podman[73524]: 2024-07-29 00:49:59.203112872 +0000 UTC m=+5.833920884 container cleanup 705d762df3840d410cc37e36638c27fcb616fed0ad3f77c6b3d1435eb0e4a9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS) 2024-07-29T00:49:59.412 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:59 smithi181 bash[73524]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10 2024-07-29T00:49:59.749 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:59 smithi181 podman[73540]: 2024-07-29 00:49:59.412771957 +0000 UTC m=+0.620517198 container remove 705d762df3840d410cc37e36638c27fcb616fed0ad3f77c6b3d1435eb0e4a9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-29T00:50:00.090 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:59 smithi181 podman[73632]: 2024-07-29 00:49:59.732678919 +0000 UTC m=+0.016624132 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:50:00.090 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:49:59 smithi181 podman[73632]: 2024-07-29 00:49:59.863607842 +0000 UTC m=+0.147553042 container create 80b2b287c3f1e8d523cd5e1555c75ec6ec9c46d748460c8acb3c14e9e8e4b05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10-deactivate, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-29T00:50:00.090 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:50:00 smithi181 podman[73632]: 2024-07-29 00:50:00.090710717 +0000 UTC m=+0.374655917 container init 80b2b287c3f1e8d523cd5e1555c75ec6ec9c46d748460c8acb3c14e9e8e4b05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:50:00.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:00.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:00.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:00.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:00.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:00.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:00.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:49:59 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:49:59.967+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:00.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:50:00 smithi181 podman[73632]: 2024-07-29 00:50:00.093861827 +0000 UTC m=+0.377807029 container start 80b2b287c3f1e8d523cd5e1555c75ec6ec9c46d748460c8acb3c14e9e8e4b05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10-deactivate, RELEASE=squid-ba82afe, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-29T00:50:00.500 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:50:00 smithi181 podman[73632]: 2024-07-29 00:50:00.215978181 +0000 UTC m=+0.499923382 container attach 80b2b287c3f1e8d523cd5e1555c75ec6ec9c46d748460c8acb3c14e9e8e4b05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, ceph=True) 2024-07-29T00:50:00.501 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:50:00 smithi181 podman[73632]: 2024-07-29 00:50:00.290867922 +0000 UTC m=+0.574813133 container died 80b2b287c3f1e8d523cd5e1555c75ec6ec9c46d748460c8acb3c14e9e8e4b05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:50:00.887 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jul 29 00:50:00 smithi181 podman[73656]: 2024-07-29 00:50:00.885686867 +0000 UTC m=+0.588980647 container remove 80b2b287c3f1e8d523cd5e1555c75ec6ec9c46d748460c8acb3c14e9e8e4b05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-29T00:50:00.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.10.service' 2024-07-29T00:50:00.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:00.942 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-07-29T00:50:00.943 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-07-29T00:50:00.943 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.11 2024-07-29T00:50:01.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:00 smithi204 systemd[1]: Stopping Ceph osd.11 for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.006+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:01.268 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.265+0000 7f11acce4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:01.268 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.265+0000 7f11acce4640 -1 osd.11 107 *** Got signal Terminated *** 2024-07-29T00:50:01.268 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.265+0000 7f11acce4640 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-29T00:50:02.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:02.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:02.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:02.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:02.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:02.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:02.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:01 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:01.978+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:03.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:03.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:03.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:03.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:03.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:03.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:03.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:03 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:03.017+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:04.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:04.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:04.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:04.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:04.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:04.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:04.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:04 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:04.014+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:05.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:05.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:05.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:05.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:05.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:05.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:05.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:05.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:05 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:05.008+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-07-29T00:49:39.134008+0000 front 2024-07-29T00:49:39.134024+0000 (oldest deadline 2024-07-29T00:50:04.934671+0000) 2024-07-29T00:50:06.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-07-29T00:48:30.525873+0000 front 2024-07-29T00:48:30.526022+0000 (oldest deadline 2024-07-29T00:48:54.626014+0000) 2024-07-29T00:50:06.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-07-29T00:48:39.927160+0000 front 2024-07-29T00:48:39.927059+0000 (oldest deadline 2024-07-29T00:49:03.427106+0000) 2024-07-29T00:50:06.336 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-07-29T00:48:49.328144+0000 front 2024-07-29T00:48:49.328257+0000 (oldest deadline 2024-07-29T00:49:15.228369+0000) 2024-07-29T00:50:06.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-07-29T00:49:06.330692+0000 front 2024-07-29T00:49:06.330614+0000 (oldest deadline 2024-07-29T00:49:32.230755+0000) 2024-07-29T00:50:06.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-07-29T00:49:13.931449+0000 front 2024-07-29T00:49:13.931493+0000 (oldest deadline 2024-07-29T00:49:39.831509+0000) 2024-07-29T00:50:06.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.175:6822 osd.6 since back 2024-07-29T00:49:23.932551+0000 front 2024-07-29T00:49:23.932456+0000 (oldest deadline 2024-07-29T00:49:49.232335+0000) 2024-07-29T00:50:06.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-07-29T00:49:29.232991+0000 front 2024-07-29T00:49:29.232958+0000 (oldest deadline 2024-07-29T00:49:53.833242+0000) 2024-07-29T00:50:06.337 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11[62019]: 2024-07-29T00:50:06.050+0000 7f11a52f2640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-07-29T00:49:39.134008+0000 front 2024-07-29T00:49:39.134024+0000 (oldest deadline 2024-07-29T00:50:04.934671+0000) 2024-07-29T00:50:06.934 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:06 smithi204 podman[74855]: 2024-07-29 00:50:06.671552812 +0000 UTC m=+5.534717277 container died 203cedcef0dd63000dad7586efcc77baf4c48487bf9447fe5944bada4e3bf9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2) 2024-07-29T00:50:07.259 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74855]: 2024-07-29 00:50:07.024846292 +0000 UTC m=+5.888010765 container cleanup 203cedcef0dd63000dad7586efcc77baf4c48487bf9447fe5944bada4e3bf9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True) 2024-07-29T00:50:07.259 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 bash[74855]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11 2024-07-29T00:50:07.576 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74873]: 2024-07-29 00:50:07.259362021 +0000 UTC m=+0.583383034 container remove 203cedcef0dd63000dad7586efcc77baf4c48487bf9447fe5944bada4e3bf9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-29T00:50:07.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74964]: 2024-07-29 00:50:07.479120762 +0000 UTC m=+0.016703189 image pull 07a7c1195fa901b5db9c7c7877f7840c7727add127afcd28880ba9ed872b58ef quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6 2024-07-29T00:50:07.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74964]: 2024-07-29 00:50:07.585035822 +0000 UTC m=+0.122618248 container create f24ba80c105cedf6ed9fd96ec55ea5c959e4e510b5c655cdab97f7cde5ef7913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-29T00:50:07.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74964]: 2024-07-29 00:50:07.779096339 +0000 UTC m=+0.316678769 container init f24ba80c105cedf6ed9fd96ec55ea5c959e4e510b5c655cdab97f7cde5ef7913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-07-29T00:50:07.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74964]: 2024-07-29 00:50:07.782274541 +0000 UTC m=+0.319856966 container start f24ba80c105cedf6ed9fd96ec55ea5c959e4e510b5c655cdab97f7cde5ef7913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:50:08.256 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74964]: 2024-07-29 00:50:07.862375101 +0000 UTC m=+0.399957530 container attach f24ba80c105cedf6ed9fd96ec55ea5c959e4e510b5c655cdab97f7cde5ef7913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-29T00:50:08.256 INFO:journalctl@ceph.osd.11.smithi204.stdout:Jul 29 00:50:07 smithi204 podman[74964]: 2024-07-29 00:50:07.977587317 +0000 UTC m=+0.515169743 container died f24ba80c105cedf6ed9fd96ec55ea5c959e4e510b5c655cdab97f7cde5ef7913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, ceph=True, org.label-schema.build-date=20240724) 2024-07-29T00:50:08.529 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@osd.11.service' 2024-07-29T00:50:08.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:08.568 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-07-29T00:50:08.568 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-07-29T00:50:08.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.a 2024-07-29T00:50:09.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:08 smithi175 systemd[1]: Stopping Ceph mds.a for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:09.535 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a[72917]: 2024-07-29T00:50:09.087+0000 7fc3e6e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:09.535 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:09 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a[72917]: 2024-07-29T00:50:09.087+0000 7fc3e6e00640 -1 mds.a *** got signal Terminated *** 2024-07-29T00:50:12.035 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:11 smithi175 podman[112260]: 2024-07-29 00:50:11.760647059 +0000 UTC m=+2.819506401 container died c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-07-29T00:50:12.489 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:12 smithi175 podman[112260]: 2024-07-29 00:50:12.236006589 +0000 UTC m=+3.294865932 container cleanup c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:50:12.489 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:12 smithi175 bash[112260]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a 2024-07-29T00:50:12.785 INFO:journalctl@ceph.mds.a.smithi175.stdout:Jul 29 00:50:12 smithi175 podman[112273]: 2024-07-29 00:50:12.487256677 +0000 UTC m=+0.709775219 container remove c4911c99062abf8b82dbb9eec50a7c40c216634511a790255fe964c501a127a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:50:12.990 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.a.service' 2024-07-29T00:50:13.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:13.088 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-07-29T00:50:13.088 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-07-29T00:50:13.088 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.d 2024-07-29T00:50:13.285 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:13 smithi175 systemd[1]: Stopping Ceph mds.d for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:13.762 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d[71761]: 2024-07-29T00:50:13.629+0000 7f9b6f600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:13.762 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:13 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d[71761]: 2024-07-29T00:50:13.629+0000 7f9b6f600640 -1 mds.d *** got signal Terminated *** 2024-07-29T00:50:17.936 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:17 smithi175 podman[112382]: 2024-07-29 00:50:17.648519512 +0000 UTC m=+4.162854270 container died 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-29T00:50:18.257 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:18 smithi175 podman[112382]: 2024-07-29 00:50:18.053445059 +0000 UTC m=+4.567779837 container cleanup 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe) 2024-07-29T00:50:18.258 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:18 smithi175 bash[112382]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d 2024-07-29T00:50:18.534 INFO:journalctl@ceph.mds.d.smithi175.stdout:Jul 29 00:50:18 smithi175 podman[112407]: 2024-07-29 00:50:18.257082017 +0000 UTC m=+0.592112212 container remove 067d1842d19731faec487545913af9296c6f76a6136de48d840b2e4287b8bd27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:50:18.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.d.service' 2024-07-29T00:50:18.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:18.891 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-07-29T00:50:18.892 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-07-29T00:50:18.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.g 2024-07-29T00:50:19.477 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:19 smithi175 systemd[1]: Stopping Ceph mds.g for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:19.785 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g[72522]: 2024-07-29T00:50:19.476+0000 7ff655800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:19.785 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:19 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g[72522]: 2024-07-29T00:50:19.476+0000 7ff655800640 -1 mds.g *** got signal Terminated *** 2024-07-29T00:50:20.946 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:20 smithi175 podman[112514]: 2024-07-29 00:50:20.68277164 +0000 UTC m=+1.333718937 container died e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-29T00:50:21.245 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:21 smithi175 podman[112514]: 2024-07-29 00:50:21.06837278 +0000 UTC m=+1.719320055 container cleanup e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux ) 2024-07-29T00:50:21.245 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:21 smithi175 bash[112514]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g 2024-07-29T00:50:21.534 INFO:journalctl@ceph.mds.g.smithi175.stdout:Jul 29 00:50:21 smithi175 podman[112526]: 2024-07-29 00:50:21.244350914 +0000 UTC m=+0.543082045 container remove e048746e0637f2f3bfae4a4fb587434561b708370d0c7846942b28e5d68e5bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-g, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:50:21.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.g.service' 2024-07-29T00:50:21.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:21.818 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-07-29T00:50:21.818 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-07-29T00:50:21.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.j 2024-07-29T00:50:22.287 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:21 smithi175 systemd[1]: Stopping Ceph mds.j for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:22.785 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j[71312]: 2024-07-29T00:50:22.396+0000 7fa6cce00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:22.785 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:22 smithi175 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j[71312]: 2024-07-29T00:50:22.396+0000 7fa6cce00640 -1 mds.j *** got signal Terminated *** 2024-07-29T00:50:25.001 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:24 smithi175 podman[112634]: 2024-07-29 00:50:24.677547667 +0000 UTC m=+2.395641287 container stop ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-29T00:50:25.001 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:24 smithi175 podman[112634]: 2024-07-29 00:50:24.703629693 +0000 UTC m=+2.421723308 container died ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0) 2024-07-29T00:50:25.285 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:25 smithi175 podman[112634]: 2024-07-29 00:50:25.113468816 +0000 UTC m=+2.831562471 container cleanup ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0) 2024-07-29T00:50:25.285 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:25 smithi175 bash[112634]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j 2024-07-29T00:50:25.784 INFO:journalctl@ceph.mds.j.smithi175.stdout:Jul 29 00:50:25 smithi175 podman[112646]: 2024-07-29 00:50:25.347522017 +0000 UTC m=+0.650962636 container remove ade123c4916ac5ef0fbc2a3c3e7cd940356e1703183ae7bd0aa3e46e1ef16805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-j, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-07-29T00:50:25.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.j.service' 2024-07-29T00:50:25.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:25.979 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-07-29T00:50:25.979 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-07-29T00:50:25.979 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.b 2024-07-29T00:50:26.311 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:50:26 smithi181 systemd[1]: Stopping Ceph mds.b for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:26.572 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:50:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: 2024-07-29T00:50:26.310+0000 7f76dfa14640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:26.572 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:50:26 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b[66446]: 2024-07-29T00:50:26.310+0000 7f76dfa14640 -1 mds.b *** got signal Terminated *** 2024-07-29T00:50:28.910 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:50:28 smithi181 podman[73736]: 2024-07-29 00:50:28.626338086 +0000 UTC m=+2.445322839 container died 84ff2af4a35aaf2c3377bd63ee11baf0384f043ce71b5c4399e7cff90ba30949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-29T00:50:29.249 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:50:28 smithi181 podman[73736]: 2024-07-29 00:50:28.977108388 +0000 UTC m=+2.796093143 container cleanup 84ff2af4a35aaf2c3377bd63ee11baf0384f043ce71b5c4399e7cff90ba30949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-07-29T00:50:29.250 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jul 29 00:50:28 smithi181 bash[73736]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-b 2024-07-29T00:50:29.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.b.service' 2024-07-29T00:50:29.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:29.459 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-07-29T00:50:29.459 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-07-29T00:50:29.459 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.e 2024-07-29T00:50:30.000 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:50:29 smithi181 systemd[1]: Stopping Ceph mds.e for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:30.000 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:50:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e[65824]: 2024-07-29T00:50:29.763+0000 7f8968cd1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:30.000 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:50:29 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e[65824]: 2024-07-29T00:50:29.763+0000 7f8968cd1640 -1 mds.e *** got signal Terminated *** 2024-07-29T00:50:32.914 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:50:32 smithi181 podman[73869]: 2024-07-29 00:50:32.641217247 +0000 UTC m=+2.968015213 container died 14b08d6e5b5234f6c388014f2d8e501066cecfcae16629ca199bd58b028b368c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724) 2024-07-29T00:50:33.248 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:50:32 smithi181 podman[73869]: 2024-07-29 00:50:32.989133825 +0000 UTC m=+3.315931780 container cleanup 14b08d6e5b5234f6c388014f2d8e501066cecfcae16629ca199bd58b028b368c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD) 2024-07-29T00:50:33.248 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jul 29 00:50:32 smithi181 bash[73869]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-e 2024-07-29T00:50:33.421 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.e.service' 2024-07-29T00:50:33.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:33.459 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-07-29T00:50:33.460 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-07-29T00:50:33.460 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.h 2024-07-29T00:50:33.850 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:33 smithi181 systemd[1]: Stopping Ceph mds.h for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:34.249 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h[66144]: 2024-07-29T00:50:33.850+0000 7ff33da2e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:34.250 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:33 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h[66144]: 2024-07-29T00:50:33.850+0000 7ff33da2e640 -1 mds.h *** got signal Terminated *** 2024-07-29T00:50:36.966 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:36 smithi181 podman[73991]: 2024-07-29 00:50:36.635169042 +0000 UTC m=+2.959136878 container died 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True) 2024-07-29T00:50:36.967 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:36 smithi181 podman[73991]: 2024-07-29 00:50:36.967292908 +0000 UTC m=+3.291260740 container cleanup 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-ba82afe, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-29T00:50:37.249 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:36 smithi181 bash[73991]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h 2024-07-29T00:50:37.250 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jul 29 00:50:37 smithi181 podman[74005]: 2024-07-29 00:50:37.168701197 +0000 UTC m=+0.528426690 container remove 8e376d404a9d3feb9c9f69f8264603a927fe88c4c3ebdfc0d7b57af85d1145ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-29T00:50:37.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.h.service' 2024-07-29T00:50:37.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:37.427 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-07-29T00:50:37.428 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-07-29T00:50:37.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.k 2024-07-29T00:50:37.749 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:50:37 smithi181 systemd[1]: Stopping Ceph mds.k for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:38.249 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:50:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k[65510]: 2024-07-29T00:50:37.770+0000 7f75ce899640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:38.250 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:50:37 smithi181 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k[65510]: 2024-07-29T00:50:37.770+0000 7f75ce899640 -1 mds.k *** got signal Terminated *** 2024-07-29T00:50:40.887 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:50:40 smithi181 podman[74112]: 2024-07-29 00:50:40.627195721 +0000 UTC m=+2.975380301 container died c9c81f1f128478150efee1c521bb8fa9fe9c5cb3fbf0203e966d5ec77fbe3e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-29T00:50:41.188 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:50:40 smithi181 podman[74112]: 2024-07-29 00:50:40.979112985 +0000 UTC m=+3.327297553 container cleanup c9c81f1f128478150efee1c521bb8fa9fe9c5cb3fbf0203e966d5ec77fbe3e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-ba82afe, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-29T00:50:41.188 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jul 29 00:50:40 smithi181 bash[74112]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-k 2024-07-29T00:50:41.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.k.service' 2024-07-29T00:50:41.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:41.353 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-07-29T00:50:41.354 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-07-29T00:50:41.354 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.c 2024-07-29T00:50:41.657 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:50:41 smithi204 systemd[1]: Stopping Ceph mds.c for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:41.917 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:50:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c[67304]: 2024-07-29T00:50:41.657+0000 7f400db55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:41.917 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:50:41 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c[67304]: 2024-07-29T00:50:41.657+0000 7f400db55640 -1 mds.c *** got signal Terminated *** 2024-07-29T00:50:44.900 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:50:44 smithi204 podman[75067]: 2024-07-29 00:50:44.63327271 +0000 UTC m=+3.076793550 container died 48119f7fae9d83e86e892dd5d433cd5579de7ca7933bfd996e42b6990f8bd9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-29T00:50:45.283 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:50:44 smithi204 podman[75067]: 2024-07-29 00:50:44.990527525 +0000 UTC m=+3.434048365 container cleanup 48119f7fae9d83e86e892dd5d433cd5579de7ca7933bfd996e42b6990f8bd9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-07-29T00:50:45.283 INFO:journalctl@ceph.mds.c.smithi204.stdout:Jul 29 00:50:44 smithi204 bash[75067]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-c 2024-07-29T00:50:45.420 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.c.service' 2024-07-29T00:50:45.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:45.455 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-07-29T00:50:45.455 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-07-29T00:50:45.455 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.f 2024-07-29T00:50:45.836 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:45 smithi204 systemd[1]: Stopping Ceph mds.f for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:45.836 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f[67003]: 2024-07-29T00:50:45.761+0000 7fea7fe51640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:45.836 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:45 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f[67003]: 2024-07-29T00:50:45.761+0000 7fea7fe51640 -1 mds.f *** got signal Terminated *** 2024-07-29T00:50:48.969 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:48 smithi204 podman[75200]: 2024-07-29 00:50:48.636452646 +0000 UTC m=+2.959033497 container died f0a1c0c1381d544fc0c1e04e6f8dc19dbd0c1a3dfd437721558e174556f3f95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-ba82afe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-29T00:50:48.969 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:48 smithi204 podman[75200]: 2024-07-29 00:50:48.969656405 +0000 UTC m=+3.292237256 container cleanup f0a1c0c1381d544fc0c1e04e6f8dc19dbd0c1a3dfd437721558e174556f3f95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-29T00:50:49.335 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:48 smithi204 bash[75200]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f 2024-07-29T00:50:49.336 INFO:journalctl@ceph.mds.f.smithi204.stdout:Jul 29 00:50:49 smithi204 podman[75214]: 2024-07-29 00:50:49.187707004 +0000 UTC m=+0.544265205 container remove f0a1c0c1381d544fc0c1e04e6f8dc19dbd0c1a3dfd437721558e174556f3f95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-ba82afe, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-29T00:50:49.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.f.service' 2024-07-29T00:50:49.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:49.477 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-07-29T00:50:49.478 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-07-29T00:50:49.478 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.i 2024-07-29T00:50:49.790 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:49 smithi204 systemd[1]: Stopping Ceph mds.i for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:50.086 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i[67611]: 2024-07-29T00:50:49.790+0000 7fbffbd55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:50.086 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:49 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i[67611]: 2024-07-29T00:50:49.790+0000 7fbffbd55640 -1 mds.i *** got signal Terminated *** 2024-07-29T00:50:52.922 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:52 smithi204 podman[75319]: 2024-07-29 00:50:52.636531879 +0000 UTC m=+2.938018441 container died 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-29T00:50:53.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:52 smithi204 podman[75319]: 2024-07-29 00:50:52.977085274 +0000 UTC m=+3.278571837 container cleanup 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-29T00:50:53.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:52 smithi204 bash[75319]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i 2024-07-29T00:50:53.336 INFO:journalctl@ceph.mds.i.smithi204.stdout:Jul 29 00:50:53 smithi204 podman[75331]: 2024-07-29 00:50:53.15803483 +0000 UTC m=+0.519131042 container remove 6b310ef961e94b191045172bf716b23d48b79006a57bb6010b92494913859cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-ba82afe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-29T00:50:53.408 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.i.service' 2024-07-29T00:50:53.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:53.439 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-07-29T00:50:53.439 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-07-29T00:50:53.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.l 2024-07-29T00:50:53.777 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:50:53 smithi204 systemd[1]: Stopping Ceph mds.l for f484ff76-4d40-11ef-bcc8-c7b262605968... 2024-07-29T00:50:54.085 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:50:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l[66691]: 2024-07-29T00:50:53.776+0000 7fe0c0013640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-29T00:50:54.086 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:50:53 smithi204 ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l[66691]: 2024-07-29T00:50:53.776+0000 7fe0c0013640 -1 mds.l *** got signal Terminated *** 2024-07-29T00:50:56.944 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:50:56 smithi204 podman[75442]: 2024-07-29 00:50:56.644665292 +0000 UTC m=+2.990141124 container died cce0a62972a38f5fd3eb147fcb049e5b2344edffef21f9649ca6a3b5bdd59f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-ba82afe, org.label-schema.build-date=20240724) 2024-07-29T00:50:57.270 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:50:57 smithi204 podman[75442]: 2024-07-29 00:50:57.002602779 +0000 UTC m=+3.348078614 container cleanup cce0a62972a38f5fd3eb147fcb049e5b2344edffef21f9649ca6a3b5bdd59f56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:43fa27c3c61b4d69c281ea6d9da48c632f0228cec8979fa720d27a945290f2f6, name=ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-ba82afe, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-29T00:50:57.270 INFO:journalctl@ceph.mds.l.smithi204.stdout:Jul 29 00:50:57 smithi204 bash[75442]: ceph-f484ff76-4d40-11ef-bcc8-c7b262605968-mds-l 2024-07-29T00:50:57.414 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f484ff76-4d40-11ef-bcc8-c7b262605968@mds.l.service' 2024-07-29T00:50:57.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-29T00:50:57.445 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-07-29T00:50:57.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --force --keep-logs 2024-07-29T00:50:57.777 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:51:02.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --force --keep-logs 2024-07-29T00:51:02.918 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:51:05.409 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --force --keep-logs 2024-07-29T00:51:05.615 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:51:07.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:51:08.043 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:51:08.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-29T00:51:08.100 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-29T00:51:08.102 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi175/crash 2024-07-29T00:51:08.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash -- . 2024-07-29T00:51:08.203 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash: Cannot open: No such file or directory 2024-07-29T00:51:08.203 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-07-29T00:51:08.208 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi181/crash 2024-07-29T00:51:08.209 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash -- . 2024-07-29T00:51:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash: Cannot open: No such file or directory 2024-07-29T00:51:08.252 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-07-29T00:51:08.254 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi204/crash 2024-07-29T00:51:08.255 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash -- . 2024-07-29T00:51:08.287 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/crash: Cannot open: No such file or directory 2024-07-29T00:51:08.287 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-07-29T00:51:08.288 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-29T00:51:08.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-29T00:51:08.394 INFO:tasks.cephadm:Compressing logs... 2024-07-29T00:51:08.395 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-29T00:51:08.398 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-29T00:51:08.401 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-29T00:51:08.424 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log 2024-07-29T00:51:08.424 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-07-29T00:51:08.424 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.2.log 2024-07-29T00:51:08.426 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.f.log 2024-07-29T00:51:08.426 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log 2024-07-29T00:51:08.426 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.5.log 2024-07-29T00:51:08.426 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.c.log 2024-07-29T00:51:08.431 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.c.log 2024-07-29T00:51:08.431 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.z.log 2024-07-29T00:51:08.431 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.c.log: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.i.log 2024-07-29T00:51:08.431 INFO:teuthology.orchestra.run.smithi204.stderr: 87.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log.gz 2024-07-29T00:51:08.431 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log 2024-07-29T00:51:08.432 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log.gz 2024-07-29T00:51:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log 2024-07-29T00:51:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:: No such file or directory 2024-07-29T00:51:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log 2024-07-29T00:51:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.7.log 2024-07-29T00:51:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.b.log 2024-07-29T00:51:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.4.log 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.1.log 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.b.log: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.y.log 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.h.log 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.10.log 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log.gz 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log.gz 2024-07-29T00:51:08.434 INFO:teuthology.orchestra.run.smithi181.stderr: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log 2024-07-29T00:51:08.437 INFO:teuthology.orchestra.run.smithi204.stderr: 92.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.z.log.gz 2024-07-29T00:51:08.441 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.11.log 2024-07-29T00:51:08.442 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.8.log 2024-07-29T00:51:08.442 INFO:teuthology.orchestra.run.smithi204.stderr: 82.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log.gz 2024-07-29T00:51:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.10.log: 92.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.y.log.gz 2024-07-29T00:51:08.445 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.e.log 2024-07-29T00:51:08.446 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log 2024-07-29T00:51:08.446 INFO:teuthology.orchestra.run.smithi181.stderr: 82.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log.gz/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.e.log: 2024-07-29T00:51:08.449 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log 2024-07-29T00:51:08.457 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-07-29T00:51:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.b.log 2024-07-29T00:51:08.459 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log 2024-07-29T00:51:08.460 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log 2024-07-29T00:51:08.460 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.g.log 2024-07-29T00:51:08.462 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.l.log 2024-07-29T00:51:08.462 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.j.log 2024-07-29T00:51:08.463 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.3.log 2024-07-29T00:51:08.463 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.j.log: 87.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.log.gz 2024-07-29T00:51:08.464 INFO:teuthology.orchestra.run.smithi175.stderr: 90.3% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.audit.log.gz 2024-07-29T00:51:08.464 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.0.log 2024-07-29T00:51:08.465 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.x.log 2024-07-29T00:51:08.466 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.a.log 2024-07-29T00:51:08.468 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.6.log 2024-07-29T00:51:08.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log 2024-07-29T00:51:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.k.log 2024-07-29T00:51:08.473 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.6.log: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log: 84.5%gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.a.log 2024-07-29T00:51:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log.gz 2024-07-29T00:51:08.474 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log 2024-07-29T00:51:08.475 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-29T00:51:08.475 INFO:teuthology.orchestra.run.smithi175.stderr: -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph.cephadm.log.gz 2024-07-29T00:51:08.477 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.d.log 2024-07-29T00:51:08.478 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-29T00:51:08.482 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log: /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.9.log 2024-07-29T00:51:08.487 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-29T00:51:08.491 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.9.log: 94.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.a.log.gz 2024-07-29T00:51:08.493 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.d.log.gz 2024-07-29T00:51:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-29T00:51:08.495 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.2% 91.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log.gz 2024-07-29T00:51:08.498 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-29T00:51:08.503 INFO:teuthology.orchestra.run.smithi175.stderr: 90.9% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-volume.log.gz 2024-07-29T00:51:08.519 INFO:teuthology.orchestra.run.smithi175.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-29T00:51:08.717 INFO:teuthology.orchestra.run.smithi204.stderr: 94.5% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.l.log.gz 2024-07-29T00:51:08.905 INFO:teuthology.orchestra.run.smithi175.stderr: 90.1% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mgr.x.log.gz 2024-07-29T00:51:08.974 INFO:teuthology.orchestra.run.smithi181.stderr: 95.5% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.e.log.gz 2024-07-29T00:51:09.034 INFO:teuthology.orchestra.run.smithi175.stderr: 95.1% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.j.log.gz 2024-07-29T00:51:09.045 INFO:teuthology.orchestra.run.smithi204.stderr: 91.5% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.c.log.gz 2024-07-29T00:51:09.094 INFO:teuthology.orchestra.run.smithi181.stderr: 91.6% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.b.log.gz 2024-07-29T00:51:09.160 INFO:teuthology.orchestra.run.smithi181.stderr: 95.1% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.k.log.gz 2024-07-29T00:51:09.559 INFO:teuthology.orchestra.run.smithi204.stderr: 95.6% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.f.log.gz 2024-07-29T00:51:09.965 INFO:teuthology.orchestra.run.smithi175.stderr: 90.6% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mon.a.log.gz 2024-07-29T00:51:11.230 INFO:teuthology.orchestra.run.smithi175.stderr: 91.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.9.log.gz 2024-07-29T00:51:11.302 INFO:teuthology.orchestra.run.smithi181.stderr: 91.3% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.4.log.gz 2024-07-29T00:51:11.332 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.c.log.gz 2024-07-29T00:51:12.021 INFO:teuthology.orchestra.run.smithi175.stderr: 92.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.0.log.gz 2024-07-29T00:51:12.050 INFO:teuthology.orchestra.run.smithi175.stderr: 91.8% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.6.log.gz 2024-07-29T00:51:12.145 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.10.log.gz 2024-07-29T00:51:12.459 INFO:teuthology.orchestra.run.smithi181.stderr: 92.0% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.1.log.gz 2024-07-29T00:51:13.021 INFO:teuthology.orchestra.run.smithi175.stderr: 91.6% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.3.log.gz 2024-07-29T00:51:13.201 INFO:teuthology.orchestra.run.smithi204.stderr: 91.9% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.8.log.gz 2024-07-29T00:51:13.730 INFO:teuthology.orchestra.run.smithi204.stderr: 91.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.11.log.gz 2024-07-29T00:51:13.825 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.2.log.gz 2024-07-29T00:51:13.837 INFO:teuthology.orchestra.run.smithi204.stderr: 91.4% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.5.log.gz 2024-07-29T00:51:14.449 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-osd.7.log.gz 2024-07-29T00:51:15.739 INFO:teuthology.orchestra.run.smithi181.stderr: 94.1% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.h.log.gz 2024-07-29T00:51:16.530 INFO:teuthology.orchestra.run.smithi204.stderr: 94.1% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.i.log.gz 2024-07-29T00:51:16.532 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-07-29T00:51:16.532 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m8.117s 2024-07-29T00:51:16.532 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m32.610s 2024-07-29T00:51:16.532 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.976s 2024-07-29T00:51:17.093 INFO:teuthology.orchestra.run.smithi175.stderr: 94.1% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.g.log.gz 2024-07-29T00:51:17.097 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-07-29T00:51:17.097 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m8.664s 2024-07-29T00:51:17.097 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m23.671s 2024-07-29T00:51:17.097 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m1.983s 2024-07-29T00:51:27.972 INFO:teuthology.orchestra.run.smithi181.stderr: 93.8% -- replaced with /var/log/ceph/f484ff76-4d40-11ef-bcc8-c7b262605968/ceph-mds.b.log.gz 2024-07-29T00:51:27.973 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-07-29T00:51:27.974 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m19.557s 2024-07-29T00:51:27.974 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m43.412s 2024-07-29T00:51:27.974 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m1.388s 2024-07-29T00:51:27.974 INFO:tasks.cephadm:Archiving logs... 2024-07-29T00:51:27.975 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi175/log 2024-07-29T00:51:27.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-29T00:51:31.286 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi181/log 2024-07-29T00:51:31.286 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-29T00:51:36.785 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi204/log 2024-07-29T00:51:36.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-29T00:51:40.736 INFO:tasks.cephadm:Removing cluster... 2024-07-29T00:51:40.737 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --force 2024-07-29T00:51:41.072 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:51:41.772 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --force 2024-07-29T00:51:41.980 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:51:42.286 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f484ff76-4d40-11ef-bcc8-c7b262605968 --force 2024-07-29T00:51:42.490 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: f484ff76-4d40-11ef-bcc8-c7b262605968 2024-07-29T00:51:42.787 INFO:tasks.cephadm:Removing cephadm ... 2024-07-29T00:51:42.788 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-29T00:51:42.846 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-29T00:51:42.862 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-29T00:51:42.878 INFO:tasks.cephadm:Teardown complete 2024-07-29T00:51:42.878 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-29T00:51:42.890 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-29T00:51:42.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-29T00:51:42.896 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-29T00:51:42.904 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-29T00:51:43.047 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-29T00:51:43.058 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-29T00:51:43.058 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-29T00:51:43.066 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-29T00:51:43.068 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-29T00:51:43.080 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-07-29T00:51:43.082 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-07-29T00:51:43.085 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-29T00:51:43.085 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-07-29T00:51:43.085 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +738us[ +741us] +/- 55ms 2024-07-29T00:51:43.085 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -810us[ -810us] +/- 75ms 2024-07-29T00:51:43.085 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -101us[ -98us] +/- 70ms 2024-07-29T00:51:43.086 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:51:43.087 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-29T00:51:43.087 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-07-29T00:51:43.087 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +712us[ +710us] +/- 55ms 2024-07-29T00:51:43.088 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 -868us[ -869us] +/- 75ms 2024-07-29T00:51:43.088 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -142us[ -142us] +/- 70ms 2024-07-29T00:51:43.088 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:51:43.113 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-07-29T00:51:43.123 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-29T00:51:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-07-29T00:51:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +730us[ +745us] +/- 55ms 2024-07-29T00:51:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -868us[ -853us] +/- 75ms 2024-07-29T00:51:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -114us[ -99us] +/- 70ms 2024-07-29T00:51:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-29T00:51:43.125 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-29T00:51:43.136 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-29T00:51:43.137 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-29T00:51:43.146 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-29T00:51:43.359 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-29T00:51:43.428 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-29T00:51:43.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-29T00:51:43.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:51:43.588 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-29T00:51:43.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:51:43.617 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-29T00:51:43.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:51:43.644 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-29T00:51:43.654 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-29T00:51:43.664 INFO:teuthology.task.internal:Duration was 2131.565071 seconds 2024-07-29T00:51:43.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-29T00:51:43.675 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-29T00:51:43.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-29T00:51:43.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-29T00:51:43.685 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-29T00:51:43.721 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-29T00:51:43.722 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-29T00:51:43.821 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-29T00:51:44.124 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-29T00:51:44.124 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-07-29T00:51:44.124 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-29T00:51:44.198 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-07-29T00:51:44.198 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-29T00:51:44.227 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-07-29T00:51:44.227 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-29T00:51:44.253 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-29T00:51:44.253 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-29T00:51:44.260 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-29T00:51:44.269 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-29T00:51:44.341 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-29T00:51:44.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-29T00:51:44.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-29T00:51:44.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-29T00:51:45.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-29T00:51:45.719 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-29T00:51:45.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-29T00:51:45.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-29T00:51:45.828 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-29T00:51:45.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-29T00:51:45.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-29T00:51:45.876 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-29T00:51:45.879 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-29T00:51:45.902 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-07-29T00:51:45.923 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-07-29T00:51:45.967 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-07-29T00:51:46.011 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-29T00:51:46.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:51:46.059 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-29T00:51:46.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:51:46.076 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-29T00:51:46.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-29T00:51:46.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-29T00:51:46.105 INFO:teuthology.task.internal:Transferring archived files... 2024-07-29T00:51:46.105 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi175 2024-07-29T00:51:46.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-29T00:51:46.229 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi181 2024-07-29T00:51:46.230 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-29T00:51:46.269 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-07-28_21:24:02-fs-squid-distro-default-smithi/7823088/remote/smithi204 2024-07-29T00:51:46.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-29T00:51:46.308 INFO:teuthology.task.internal:Removing archive directory... 2024-07-29T00:51:46.308 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-29T00:51:46.316 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-29T00:51:46.318 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-29T00:51:46.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-29T00:51:46.380 INFO:teuthology.task.internal:Not uploading archives. 2024-07-29T00:51:46.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-29T00:51:46.406 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-29T00:51:46.406 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-29T00:51:46.411 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-29T00:51:46.414 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-29T00:51:46.428 INFO:teuthology.orchestra.run.smithi181.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 29 00:51 /home/ubuntu/cephtest 2024-07-29T00:51:46.430 INFO:teuthology.orchestra.run.smithi204.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 29 00:51 /home/ubuntu/cephtest 2024-07-29T00:51:46.460 INFO:teuthology.orchestra.run.smithi175.stdout: 393421 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 29 00:51 /home/ubuntu/cephtest 2024-07-29T00:51:46.463 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-29T00:51:46.475 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-29T00:51:46.566 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3e9255e907100f88049f320ca33d977e29155d12 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2131.5650708675385 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi175 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba82afec3628bf830c0830558bf3d3c3aea66701 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6f96dc749b3d46c6b8dff7b3d425de51 status: fail success: false 2024-07-29T00:51:46.566 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-29T00:51:46.661 INFO:teuthology.run:FAIL